Statistical Inference and Point Estimation

Statistical Inference

• Statistical inference involves using sample data to make inferences about population parameters.

Point Estimation

• Point estimation involves finding a single value to estimate a population parameter.
• An estimator is a function of the sample used to estimate a parameter.
• Desirable properties of estimators include unbiasedness, small variance, consistency, and efficiency.
• An unbiased estimator has an expected value equal to the parameter it estimates: ( E[\hat{\Theta}] = \theta ).
• A consistent estimator converges in probability to the true parameter value: ( \hat{\Theta} \xrightarrow{p} \theta ).
• Efficiency refers to the estimator with the smallest variance among unbiased estimators.
• Bias is the difference between the expected value of the estimator and the true parameter: ( Bias(\hat{\Theta}) = E[\hat{\Theta}] - \theta ).
• Mean Squared Error (MSE) measures the average squared difference between the estimator and the parameter: ( MSE(\hat{\Theta}) = E[(\hat{\Theta} - \theta)^2] = Var(\hat{\Theta}) + Bias(\hat{\Theta})^2 ).
• Common methods for finding estimators include the Method of Moments and Maximum Likelihood Estimation (MLE).

Method of Moments

• The Method of Moments equates sample moments to population moments to solve for parameters.
• For example, if ( X_1,..., X_n \sim N(\mu, \sigma^2) ), then ( E[X] = \mu ) and ( \frac{1}{n}\sum_{i=1}^{n} X_i = \hat{\mu} ).

Maximum Likelihood Estimation (MLE)

• Maximum Likelihood Estimation (MLE) chooses the parameter value that maximizes the likelihood function.
• The likelihood function ( L(\theta; x_1,..., x_n) = \prod_{i=1}^{n} f(x_i; \theta) ) represents the probability of observing the given sample for a given parameter value.
• The log-likelihood function ( \ell(\theta; x_1,..., x_n) = \sum_{i=1}^{n} log(f(x_i; \theta)) ) is often used for easier computation.
• To find the MLE: write the likelihood function, take its logarithm, differentiate with respect to the parameter, set to zero, solve for the parameter, and verify that it's a maximum.
• For example, if ( X_1,..., X_n \sim Bernoulli(p) ), the MLE of ( p ) is ( \hat{p} = \frac{1}{n}\sum_{i=1}^{n} x_i = \bar{x} ).
• For ( X_1,..., X_n \sim N(\mu, \sigma^2) ) with ( \sigma^2 ) known, the MLE of ( \mu ) is ( \hat{\mu} = \frac{1}{n} \sum_{i=1}^{n} x_i = \bar{x} ).
• For ( X_1,..., X_n \sim N(\mu, \sigma^2) ) with ( \mu ) known, the MLE of ( \sigma^2 ) is ( \hat{\sigma}^2 = \frac{1}{n} \sum_{i=1}^{n} (x_i - \mu)^2 ).
• If both ( \mu ) and ( \sigma^2 ) are unknown, their MLEs are ( \hat{\mu} = \frac{1}{n} \sum_{i=1}^{n} x_i = \bar{x} ) and ( \hat{\sigma}^2 = \frac{1}{n} \sum_{i=1}^{n} (x_i - \hat{\mu})^2 = \frac{1}{n} \sum_{i=1}^{n} (x_i - \bar{x})^2 ), respectively.
• The Invariance Property of MLEs states that if ( \hat{\theta} ) is the MLE of ( \theta ), then ( g(\hat{\theta}) ) is the MLE of ( g(\theta) ).
• MLEs are consistent, asymptotically normal, and efficient under certain regularity conditions.

Cramer-Rao Lower Bound (CRLB)

• The Cramer-Rao Lower Bound (CRLB) provides a lower bound on the variance of any unbiased estimator: ( Var(\hat{\Theta}) \geq \frac{1}{nI(\theta)} ).
• ( I(\theta) = E[(\frac{\partial}{\partial \theta} log f(X; \theta))^2] = -E[\frac{\partial^2}{\partial \theta^2} log f(X; \theta)] ) is the Fisher Information.
• For example, if ( X_1,..., X_n \sim Bernoulli(p) ), then ( Var(\hat{p}) \geq \frac{1}{nI(p)} = \frac{p(1-p)}{n} ).

Matrices

• A matrix is a table of ( m \cdot n ) elements arranged in ( m ) rows and ( n ) columns.
• ( A = \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n} \ a_{21} & a_{22} & \cdots & a_{2n} \ \vdots & \vdots & \ddots & \vdots \ a_{m1} & a_{m2} & \cdots & a_{mn} \end{bmatrix} ) where ( a_{ij} ) is the element at row ( i ) and column ( j ).

Types of Matrices

• A row matrix has only one row: ( 1 \times n ).
• A column matrix has only one column: ( m \times 1 ).
• A square matrix has an equal number of rows and columns: ( n \times n ).
  • The diagonal principal consists of elements ( a_{ii} ).
  • The diagonal secondary consists of elements ( a_{ij} ) where ( i + j = n + 1 ).
• A null/Zero matrix has all elements equal to zero.
• An identity matrix (( I_n )) is a square matrix where ( a_{ii} = 1 ) and ( a_{ij} = 0 ) for ( i \neq j ).
• The transpose matrix (( A^T )) is obtained by swapping rows and columns: if ( A = [a_{ij}] ), then ( A^T = [a_{ji}] ).
• A symmetric matrix satisfies ( A = A^T ).
• An anti-symmetric matrix satisfies ( A = -A^T ).

Matrix Operations

• Addition and subtraction are only possible between matrices of the same size. Elements are added (or subtracted) at corresponding positions: ( A + B = [a_{ij} + b_{ij}] ).
• Scalar multiplication involves multiplying each element of the matrix by the scalar: ( \alpha \cdot A = [\alpha \cdot a_{ij}] ).
• To multiply matrices ( A_{m \times n} \cdot B_{n \times p} = C_{m \times p} ), element ( c_{ij} ) is the sum of the products of row ( i ) from ( A ) with column ( j ) from ( B ): ( c_{ij} = a_{i1}b_{1j} + a_{i2}b_{2j} + \cdots + a_{in}b_{nj} ).

Determinants

• Determinants are functions that associate a real number to a square matrix.
• For a 2x2 matrix: ( det(A) = a_{11}a_{22} - a_{12}a_{21} ).
• For a 3x3 matrix, the Rule of Sarrus can be used
• If a row or column is all zeros, the determinant is zero.
• Swapping two rows or columns changes the sign of the determinant.
• Multiplying a row or column by a scalar multiplies the determinant by that scalar.
• ( det(A^T) = det(A) ).
• ( det(A \cdot B) = det(A) \cdot det(B) ).

Matrix Inverse

• The inverse (( A^{-1} )) satisfies ( A \cdot A^{-1} = A^{-1} \cdot A = I ).
• The inverse can be calculated as ( A^{-1} = \frac{1}{det(A)} \cdot Adj(A) ), where ( Adj(A) ) is the adjugate of ( A ).
• A matrix ( A ) is invertible if and only if ( det(A) \neq 0 ).

Linear Systems

• Linear systems can be represented in matrix form as ( A \cdot X = B ).
  • ( A ) is the matrix of coefficients.
  • ( X ) is the matrix of unknowns.
  • ( B ) is the matrix of constant terms.

Resolution

• Cramer's Rule uses determinants to solve for the unknowns.
• Escalation transforms a system into an equivalent, simpler system.
• With the Inverse method, if ( A ) is invertible, then ( X = A^{-1} \cdot B ).

Alternative matrix definition

• A matrix (A) of size (m \times n) is a rectangular table of (m) rows and (n) columns, where each entry (a_{ij}) is a real or complex number.
• Matrices can be used to represent and manipulate images and graphics.

Bernoulli's Principle

• Discovered by Daniel Bernoulli, states that for an inviscid flow, an increase in fluid speed occurs simultaneously with a decrease in pressure or a decrease in the fluid's potential energy.

Types of Fluid Flow

Laminar Flow

• Laminar flow has the fluid flowing in parallel layers without disruption.

Turbulent Flow

• Turbulent flow has the fluid undergoing irregular fluctuations or mixing.

The Equation

$$ P + \frac{1}{2} \rho v^2 + \rho gh = \text{constant} $$

Variables

• ( P ) is the static pressure of the fluid
• ( \rho ) is the density of the fluid
• ( v ) is the speed of the fluid
• ( g ) is the acceleration due to gravity
• ( h ) is the height

Applications

Airplanes

• Aircraft wings are shaped to make air flow faster over the top than underneath.
• Pressure above the wing is lower than pressure below, producing lift.

Race Cars

• Race cars use Bernoulli's principle to generate downforce and increase traction.

Chimneys

• Tall chimneys use wind for smoke draft.
• Wind moving across the top causes a pressure drop inside, drawing smoke upwards.

Atomizers

• Perfume bottles and paint sprayers use fast-moving air over an open tube to disperse a fine spray.

Algorithmic Trading

What is it?

• Algorithmic trading uses computer programs to execute trades based on pre-defined instructions.
• It considers various factors like price, timing, and volume.

Benefits

• Reduced transaction costs, improved speed, increased efficiency, minimized errors, and back-testing.

Order Execution Strategies

Market Order

• Executed immediately at the best available price.
• Provides certainty of execution but no price guarantee.

Limit Order

• Executed only at a specified price or better.
• Offers price control but execution is not guaranteed.

Stop Order

• Triggered when the price reaches a specified level.
• Can be used to limit losses or protect profits.
• Becomes a market order when triggered.

Volume-Weighted Average Price (VWAP)

• Aims to execute orders close to the VWAP.
• Divides the order into smaller portions and executes them over a period.
• Suited for large orders to minimize market impact.
• Formula: $$VWAP = \frac{\sum_{i}(Price_{i} \times Volume_{i})}{\sum_{i}Volume_{i}}$$

Time-Weighted Average Price (TWAP)

• Executes orders evenly over a specified time frame.
• Reduces the risk of adverse selection.
• Useful when minimal market impact is desired.
• Formula: $$TWAP = \frac{\sum_{i}Price_{i}}{n}$$ where ( n ) is the number of prices observed over the time period.

Implementation Shortfall

• Seeks to minimize the difference between the actual execution price and the decision price.
• Considers market impact, opportunity cost, and transaction costs. Aims for best execution.
• Formula: $$Implementation \ Shortfall = (End \ Portfolio \ Value - Benchmark \ Portfolio \ Value) - Transaction \ Costs$$.

Market Impact

• Algorithmic trading can impact market prices and liquidity.
• Large orders may lead to temporary price fluctuations.
• High-frequency trading (HFT) can amplify volatility.

High-Frequency Trading (HFT)

• A subset of algorithmic trading characterized by high speed, high turnover, and short-term positions.
• It uses sophisticated technology and co-location to gain speed advantages.

Regulatory and Ethics

• Regulatory bodies like the SEC and FINRA monitor algorithmic trading.
• Regulations aim to prevent market abuse and ensure fair trading practices.
• Ethical considerations include transparency, fairness, and investor protection.