Stochastic Signals and Systems
 
Prof. Dr.-Ing. Dieter Kraus     

1. Probability Theory (pdf)
   1.1 Terminology
   1.2 Definition of Probability
        1.2.1 Relative Frequency and Probability
        1.2.2 Axiomatic Approach to Probability
        1.2.3 Classical Definition of Probability
   1.3 Conditional Probability
   1.4 Random Variables
   1.5 Distribution Functions
        1.5.1 Purely discrete case
        1.5.2 Purely continuous case
   1.6 Some Special Distributions
        1.6.1 Discrete Distributions
        1.6.2 Continuous Distributions
   1.7 Bivariate Distribution
        1.7.1 Bivariate Distribution Function
        1.7.2 Bivariate Density Function
        1.7.3 Marginal Distribution and Density Function
        1.7.4 Conditional Distribution and Density Function
        1.7.5 Independent Random Variables
        1.7.6 Bivariate Normal Distribution
    1.8 Transformations of Random Variables
        1.8.1 Function of One Random Variable
        1.8.2 One Function of Two Random Variables
        1.8.3 Two Functions of Two Random Variables
   1.9 Expectation Operator
        1.9.1 Expectation for Univariate Distributions
        1.9.2 Expectation for Bivariate Distributions
        1.9.3 Mean Square Estimation
   1.10 Vector-valued Random Variables
        1.10.1 Multivariate Distributions
        1.10.2 Transformation of Vector-valued Random Variables
        1.10.3 Expectations for Vector-valued Random Variables
        1.10.4 Mean Square Estimation
        1.10.5 Multivariate Normal Distribution
   1.11 Sequences of Random Variables
        1.11.1 Convergence Concepts
        1.11.2 Laws of Large Numbers
        1.11.3 Central Limit Theorems

2. Stochastic Processes (pdf)
   2.1 Fundamentals 
        2.1.1 Definition of Stochastic Processes
        2.1.2 Sample Function and Ensembles
        2.1.3 Probabilistic Description of Stochastic Processes
        2.1.4 Complex Stochastic Processes
        2.1.5 Moment Functions
   2.2 Some Particular Processes
        2.2.1 Poisson Process
        2.2.2 Random Walk
        2.2.3 Wiener Process (Brownian motion)
        2.2.4 Markov Process
        2.2.5 Gauss Process
   2.3 Stationary Processes  
        2.3.1 Real Valued Stationary Processes
        2.3.2 Complex Valued Stationary Processes
        2.3.3 Moment Functions for Stationary Processes
   2.4 Stochastic Limiting Operations
        2.4.1 Stochastic Continuity
        2.4.2 Stochastic Differentiation
        2.4.3 Stochastic Integration
    2.5 Spectral Analysis of Stationary Processes
        2.5.1 Spectral Density Function
        2.5.2 Spectral Representation of Stationary Processes
    2.6 Systems with Stochastic Inputs
        2.6.1 Transformation of Stochastic Processes
        2.6.2 Memoryless Systems
        2.6.3 Linear Systems
    2.7 Special Discrete-time Parameter Models
        2.7.1 Purely Random Processes, White Noise
        2.7.2 Auto-Regressive (AR)-Processes
        2.7.3 Moving-Average (MA)-Processes
        2.7.4 Auto-Regressive-Moving Average (ARMA)-Processes

3. Parameter Estimation (pdf)
   3.1 Estimating Function and Estimator
   3.2 Sufficient Statistic, Exponential Families
   3.3 Linear Least Squares Estimation
   3.4 Confidence Intervals
   3.5 Cramer-Rao Lower Bound
   3.6 Maximum Likelihood Estimation
   3.7 Bayesian Estimation
        3.7.1 Minimum Mean Square Error Estimation
        3.7.2 Minimum Mean Absolute Error Estimation
        3.7.3 Maximum A Posteriori Estimation

4. Signal Detection (pdf)
   4.1 Introduction to Hypothesis Testing
   4.2 Simple Hypothesis Testing
        4.2.1 Neyman Pearson Hypothesis Testing
        4.2.2 Most Powerful Tests for Normally Distributed Data
        4.2.3 Bayes Hypothesis Testing
   4.3 Composite Hypothesis Testing
        4.3.1 Sufficient Statistic
        4.3.2 Bayesian Approach
        4.3.3 Monotone Likelihood Ratio and UMP Tests
        4.3.4 Invariance Principle and UMP Invariant Tests
        4.3.5 Maximum Likelihood Ratio Test
        4.3.6 Non Parametric Tests and Invariance

5. Spectrum Analysis (pdf)
   5.1 Estimation of Moment Functions
        5.1.1 Ergodicity
        5.1.2 Estimation of the Mean
        5.1.3 Estimation of the Covariance Function
        5.1.4 Estimation of the Cross-Covariance Function  
    5.2 Nonparametric Spectrum Estimation
        5.2.1 Finite Discrete-Time Fourier Transform
        5.2.2 Periodogram
        5.2.3 Smoothing and Averaging of Periodograms
        5.2.4 Consistent Spectrum Estimation
   5.3 Parametric Spectrum Estimation
        5.3.1 Parametric Models
        5.3.2 Consistent Parameter Estimators
        5.3.3 Asymtotically Efficient Paramter Estimators

6. Optimal Filtering (pdf)
   6.1 Introduction
   6.2 Matched Filtering
        6.2.1 Matched Filtering for White Noise
        6.2.2 Matched Filtering as Correlation Processing
   6.3 Wiener Filtering
        6.3.1 Wiener-Hopf Equation
        6.3.2 Finite Wiener Filtering
        6.3.3 Noncausal Wiener Filtering
        6.3.4 Causal Wiener Filtering
   6.4 Kalman Filtering
        6.4.1 State Space Model
        6.4.2 State Estimation
        6.4.3 Kalman Filter Approach

Assignments (pdf)

Questionnaire (pdf)