Stochastic Processes: Definition and Types

• A stochastic process is a collection of random variables used to model random processes.
• Represented as {X(t) | t ∈ T}, where T is the parameter space and S is the state space.
• X(t) denotes the random variables in the process.
• X₁ can be used instead of X(t) to indicate random variables in a stochastic process.
• For each t ∈ T, X(t) represents a different random variable.

Types of Stochastic Processes

• Stochastic processes are described by their parameter space T and state space S.
• Discrete-time: T is countable.
• Continuous-time: T is uncountable.
• Discrete-state: S is countable.
• Continuous-state: S is uncountable.
• Four types of stochastic processes exist, based on combinations of discrete/continuous time and state.

Example: Modeling Customers in a Shop

• Uses two options to model the number of customers in a shop during the day.
• Option 1: X(t) = Number of customers in the shop at time t.
• T = {t ∈ ℝ : 0 ≤ t ≤ 24} is uncountably infinite.
• S = {0, 1, 2, ...} is countably infinite.
• This option represents a continuous-time, discrete-state process.
• Option 2: X(t) = Number of customers in the shop after the nth customer leaves.
• T = {1, 2, 3, ...} is countably infinite.
• S = {0, 1, 2, 3, ...} is countably infinite.
• This option represents a discrete-time, discrete-space process.

Statistics of Stochastic Processes

• Three main points of interest when studying stochastic processes:
  • Dependencies that the sequences of values generated by the process exhibit.
  • Long-term averages of the generated sequence of values.
  • Characterization of the likelihood and frequency of certain boundary events.
• {X(t) | t ∈ T} represents a stochastic process with parameter space T and state space S.

Density Functions

• For a fixed element t of T:
  • The random variable X(t) has a cumulative distribution function F(x, t) = P(X(t) ≤ x) for every x ∈ ℝ.
  • Probability density function/probability mass function is given by:
    • Continuous-state: f(x, t) = ∂F(x, t) / ∂x
    • Discrete-state: f(x, t) = P(X(t) = x)
  • For values t₁, t₂ ∈ T with t₁ ≠ t₂, the joint cdf of random variables X(t₁) and X(t₂) is: F(x₁, x₂, t₁, t₂) = P(X(t₁) ≤ x₁ ∩ X(t₂) ≤ x₂).
  • Joint pdf/pmf:
    • S continuous: f(x₁, x₂, t₁, t₂) = ∂²F(x₁, x₂, t₁, t₂) / ∂x₁∂x₂
    • S discrete: f(x₁, x₂, t₁, t₂) = P(X(t₁) = x₁ ∩ X(t₂) = x₂)
  • Generalization to n random variables X(t₁), X(t₂), ..., X(tₙ) follows.
• If we treat X(t) as a random variable, then these definitions are the same as previous definitions.
• Fully knowing a stochastic process requires knowing f(x₁, x₂,... xₙ; t₁, t₂,... tₙ) for all n ∈ ℕ and parameters t₁, t₂, ... tₙ ∈ T, which is usually impossible.

Moments and Correlations

• For a fixed t ∈ T:

  • The mean or expectation of X(t) is given by:
    • S continuous: μ(t) = E(X(t)) = ∫₋∞ to ∞ x f(x, t) dx
    • S discrete: μ(t) = E(X(t)) = ∑ x f(x, t)
  • The variance of X(t) is given by:
    • S continuous: σ²(t) = Var(X(t)) = E((X(t) − μ(t))²) = ∫₋∞ to ∞ (x − μ(t))² f(x, t) dx
    • S discrete: σ²(t) = Var(X(t)) = E((X(t) − μ(t))²) = ∑ (x − μ(t))² f(x, t)

• For fixed t₁, t₂ ∈ T:

  • The autocorrelation of X(t₁) and X(t₂) is: R(t₁, t₂) = E(X(t₁)X(t₂)).
    • Measures the relationship between the current value and its past value.
    • S continuous: R(t₁, t₂) = ∫₋∞ to ∞ ∫₋∞ to ∞ x₁x₂ f(x₁, x₂; t₁, t₂) dx₁ dx₂
    • S discrete: R(t₁, t₂) = ∑ ∑ x₁x₂ f(x₁, x₂; t₁, t₂)
  • The autocorrelation of two different processes X(t) and Y(t) is: Rxy(t₁, t₂) = E(X(t₁)Y(t₂)).
    • Measures how common the processes are with themselves.
  • The autocovariance of X(t₁) and X(t₂) is: C(t₁, t₂) = E((X(t₁) – μ(t₁))(X(t₂) – μ(t₂))) = R(t₁, t₂) – μ(t₁)μ(t₂).
    • This applies for discrete-state and continuous-state processes.
    • Measures the joint variability of the two random variables.

Example: Tossing a Coin

• X(t) represents the outcome of the t-th toss: 1 if heads, 0 otherwise.
• T = {1, 2, 3, ...}
• S = {0, 1}
• Joint pmf is calculated using independance of coin tosses.
• p is the probability of getting heads
• If not independent, then P(X(t₁) = x₁) * P(X(t₂) = x₂) has no connection to the joint pmf

At The Station

• nth-order strict-sense stationary (SSS) for stochastic processes.
• The joint distribution of random variables at times t1, t2,...,tn is the same as that at times t1+s, t2+s, ..., tn+s.
• This definition applies for some n ∈ N, t1, t2, ..., tn ∈ T, and s ∈ R such that ti + s ∈ T for all i = 1, 2, ..., n.
• If the process is nth-order SSS for every n ∈ N, it is considered strict-sense stationary.

Consequences of Stationary SPs

Under Stationary SPs:

• If f(x, t) is independent of time, then E(X(t)) = μ is constant.
  • μ represents a constant.
• With a 2nd-order SSS process, f(x₁, x₂; t₁, t₂) = f(x₁, x₂; t₁ + s, t₂ + s) for all s.
  • If s = -t₁, f(x₁, x₂; t₁, t₂) = f(x₁, x₂; t₂ - t₁), becoming a function of t₂ - t₁.
  • This means μ(t₁) = E(X(t₁)) = μ(t₂ - t₁) and similarly for μ(t₂).
  • R(t₁, t₂) only depends on the difference t₂ - t₁ and not on t₁ itself.
  • C(t, t + s) = R(t, t + s) – μ(t)μ(t + s) only depends on the difference between t and t + s.
• A weaker definition of stationary is simpler than direct proof.

Definition of Wide-Sense Stationary

• A stochastic process {X(t) | t ∈ T} is wide-sense stationary (WSS).
• This applies if, for all t ∈ T and s ∈ ℝ such that t + s ∈ T, E(X(t)) and R(t, t + s) do NOT depend on t.
• Two processes {X(t) | t ∈ Tx} and {Y(t) | t ∈ Ty} are jointly wide-sense stationary when both processes are WSS.
• Rxy(t, t + s) shouldn't depend on t either.

Example: Wide-Sense Stationary

• X(t) defined as r cos(at + φ).
• r and a are constants.
• φ is a uniformally distributed random variable.
• E[X(t)] is a constant independent of time.

Increments of a Stochastic process

• An increment of a random process {X(t) | t ∈ Tz} is X(t₂) - X(t₁) for some t₂,t₁ ∈ T where t₂ > t₁.
• A stochastic process {X(t) | t ∈ Tx} has independent increments.
• For every n∈ N and t₀ < t₁ < ... < tₙ ∈ T, the random variables X(t₀), X(t₁) - X(t₀), ..., X(tₙ) - X(tₙ₋₁) are jointly independent.
  • I.e. the difference between values of the random process across non-overlapping time intervals are independent.

Stationary ones

• A stochastic process {X(t) | t = 1,2,...} has stationary increments if, for every fixed s > 0.
• The increment X (t + s) – X(t) has the same distribution Vt ∈ T.
  • The distribution of an increment only depends on the length of the time period it spans, i.e. s.

Example - Stationary increments

• Considers a stochastic process defined as flipping a coin with probability p of landing on heads.
• Y(.) Represents total # of heads
• Has stationary increments.

Explore stochastic processes, which model random phenomena using random variables. Learn about parameter and state spaces and their influence on process types. Understand discrete and continuous-time stochastic processes. See examples in real-world applications.