Linear Block Codes

Linear Block Codes

• A linear block code is characterized by the condition that linear combinations of codewords also produce a codeword.
• An (n, k) linear block code converts k information bits into n code bits.
• Key property: if x and y are codewords, x + y is also a codeword.
• Key property: if x is a codeword and a is a scalar, then ax is also a codeword.
• Repetition and single parity check codes are examples of linear block codes.

Generator Matrix

• The generator matrix G for an (n, k) linear block code is a k × n matrix.
• Codewords are generated by x = mG, where m is a k-bit message vector.
• Rows of G must be linearly independent to ensure distinct codewords for different messages.
• Repetition code generator matrix example: G = (1 1 1).

Systematic Codes

• Systematic codes include the message bits directly in the codeword.
• The generator matrix for a systematic code can be represented as G = (Ik P).
  • Ik is a k × k identity matrix
  • P is a k × (n-k) matrix
• The first k bits of the codeword represent the message, while the remaining n-k bits are parity check bits.
• (7, 4) Hamming code example: Rows of G are (1 0 0 0 1 1 0), (0 1 0 0 1 0 1), (0 0 1 0 0 1 1), (0 0 0 1 1 1 1)
• In the (7,4) Hamming code example, if m = (m1 m2 m3 m4), then x = (m1 m2 m3 m4 m1+m2+m4 m1+m3+m4 m2+m3+m4).

Parity Check Matrix

• The parity check matrix H for an (n, k) linear block code is an (n-k) × n matrix.
• It satisfies the condition HxT = 0 for all codewords x.
• If the generator matrix G = (Ik P), then the parity check matrix is H = (-PT In-k).
• (7, 4) Hamming code example: Rows of H are (1 1 0 1 1 0 0), (1 0 1 1 0 1 0), (0 1 1 1 0 0 1)

Syndrome Decoding

• The syndrome of a received vector y is defined as s = HyT.
• If s = 0, y is a codeword; if s ≠ 0, y contains errors.
• The syndrome is only dependent on the error vector e, according to s = HeT.
• Syndrome decoding steps:
  • Compute the syndrome s = HyT.
  • Find the error vector e corresponding to syndrome s.
  • Decode as 𝑥̂ = y - e.
• The syndrome s uniquely identifies an error pattern e only if the columns of H are all distinct and nonzero.
• With distinct and nonzero columns in H, the code can correct up to one error.
• To prove uniqueness, suppose two error patterns e1 and e2 produce the same syndrome, resulting in H(e1 - e2)T = 0, so (e1 - e2) is a codeword.
• For a code that corrects one error, e1 - e2 must have at least three nonzero components.
• Continuing the (7, 4) Hamming code example- H = (1 1 0 1 1 0 0), (1 0 1 1 0 1 0), (0 1 1 1 0 0 1)-- and y=(0 1 0 1 1 1 1), s = (1 0 1).
• error is therefore in the 3rd position, so e = (0 0 1 0 0 0 0), and 𝑥̂ = y - e = (0 1 1 1 1 1 1).

Algorithmic Trading Basics

• Involves using computer programs to automate trade execution based on instructions.
• Subset: High-Frequency Trading (HFT).
• Advantages: optimized pricing, immediate execution, reduced costs, accurate timing, manual error reduction, backtesting, accessibility of difficult assets.
• Disadvantages: specialized knowledge needed, requires high-speed data and platforms, capital intensive, vulnerable to technical glitches, potential for market manipulation.

Common Strategies: Trend Following Strategies

• Identifying and capitalizing on market trends using indicators like moving averages and trend lines.
• Example: Moving Average Crossover (MAC) system.

Common Strategies: Arbitrage Opportunities

• Taking advantage of price discrepancies of an asset across different markets.
• Relies on real-time data and complex algorithms.
• Example: Buy low on one exchange, sell high on another.

Common Strategies: Mean Reversion

• Trading on the expectation that an asset's price will revert its average.
• Uses statistical indicators to identify overbought/oversold conditions.
• Example: Relative Strength Index (RSI).

Common Strategies: Mathematical Model

• Developing trading strategies based on statistical analysis and probability theory.
• Optimizing trading parameters like trade size, stop-loss, and timing.
• Example: Kalman filter for price forecasting.

Common Strategies: Time Weighted Average Price (TWAP)

• Breaking large orders into smaller ones, releasing with a mathematically schedule.
• Aims to minimize impact on market price.
• Formula: TWAP = (∑ Pi)/n, where Pi is the asset's price at time i and n is the number of observations.

Common Strategies: Volume Weighted Average Price (VWAP)

• VWAP executes large orders without impacting the market price, with a weight on trade volume.
• Orders broken and released based on historical volume profiles.
• Formula: VWAP = (∑ Pi × Vi) / (∑ Vi), where Pi is the price at time i and Vi is the volume at time i.

Introduction to Calculus

• Variational methods use derivatives.
• The adjoint method is an efficient way to compute derivatives of scalar functions that depend on vector fields.
• Scalar Functionals: J = J[u], where x ∈ Ω, describe a scalar functional J of a vector field u(x)
• Examples: lift or drag of an air and fuel consumed by a vehicle.

The Adjoint Method: Continuous Adjoint

• Scalar functional J[u(x), v(x)]; constraint E[u(x), v(x)] = 0; E is scalar functional
• Lagrangian ℒ = J + λE, where adjoint variable= Lagrange multiplier λ(x)
• The first variation of Lagrangian = 𝛿ℒ = 𝛿J + λ𝛿E + E𝛿λ.
• If E = 0, then 𝛿ℒ = 𝛿J + λ𝛿E. Therefore, 𝛿𝐽= ∫Ω(𝜕𝐽/𝜕𝑢 𝛿𝑢 + 𝜕𝐽/𝜕𝑣 𝛿𝑣) dx.
• Variation of E is: 𝛿E = ∫Ω(𝜕E/𝜕𝑢 𝛿𝑢 + 𝜕E/𝜕𝑣 𝛿𝑣) dx.
• First Variation of Lagrangian= 𝛿ℒ = ∫Ω [(𝜕𝐽/𝜕𝑢 + 𝜆𝜕E/𝜕𝑢)𝛿𝑢 + (𝜕𝐽/𝜕𝑣 + 𝜆𝜕E/𝜕𝑣)𝛿𝑣]dx.
• Adjoint equation: 𝜕𝐽/𝜕𝑣 + 𝜆𝜕E/𝜕𝑣 = 0
• If the adjoint equation is satisfied, the first variation of the Lagrangian reduces to 𝛿ℒ= ∫Ω(𝜕𝐽/𝜕𝑢 + 𝜆𝜕E/𝜕𝑢) 𝛿𝑢 dx.
• Hence, dJ/du = 𝜕𝐽/𝜕𝑢 + 𝜆𝜕E/𝜕𝑢
• Continuous Adjoint summary: solve primal equation E[u, v ]=0; solve adjoint equation 𝜕𝐽/𝜕𝑣 + 𝜆𝜕E/𝜕𝑣 = 0; subsequently gradient is computed from dJ/du = 𝜕𝐽/𝜕𝑢 + 𝜆𝜕E/𝜕𝑢.

The Adjoint Method: Discrete Adjoint

• Scalar System: J(u, v); constraint system: E(u, v )=0
• Convert the equation into discretization to obtain vectors: u,v ∈ 𝑅𝑛
• E(u, v )=0 then becomes: e(u,v)=0
• Langrangian equation: ℒ = 𝐽(u,v) + 𝜆𝑇e(u,v), where 𝜆 ∈ 𝑅𝑛
• First variation: 𝛿ℒ = 𝛿𝐽+ 𝜆𝑇 𝛿e + e𝑇 𝛿𝜆
• Since e(u,v) = 0, : 𝛿ℒ = 𝛿𝐽+ 𝜆𝑇 𝛿e
• Variation of j equation: 𝛿𝐽=𝜕𝐽/𝜕𝑢𝑇𝛿𝑢 + 𝜕𝐽/𝜕𝑣𝑇𝛿𝑣
• Variation of e, equation being: 𝛿e = 𝜕e/𝜕𝑢𝛿𝑢 + 𝜕e/𝜕𝑣 𝛿𝑣
• First Variation therefore: 𝛿ℒ = (𝜕 𝐽/𝜕𝑢𝑇+ 𝜆𝑇 𝜕e/𝜕𝑢) 𝛿𝑢 + (𝜕𝐽/𝜕𝑣𝑇+ 𝜆𝑇𝜕e/𝜕𝑣) 𝛿𝑣
• Discrete Adjoint equation: 𝜕𝐽/𝜕𝑣𝑇+ 𝜆𝑇𝜕e/𝜕𝑣 = 0 -> : (𝜕e/𝜕𝑣)𝑇𝜆 = - 𝜕𝐽/𝜕𝑣
• Gradient, final equation; 𝛿ℒ=(𝜕 𝐽/𝜕𝑢𝑇 + 𝜆𝑇 𝜕e/𝜕𝑢)delta(u) or dJ/du = 𝜕 J/𝜕 u + (𝜕 e/𝜕 u)𝑇 𝜆.
• Discrete Adjoint Summary: solve the primal equation e(u,v)=0. Subsequently, solve(𝜕e/𝜕𝑣)𝑇 𝜆 =-𝜕𝐽/𝜕𝑣., therefore, gradient =dJ/du=𝜕 J/𝜕 u + (𝜕 e/𝜕 u)𝑇 𝜆.

Adjoint Method Example

• Let objective function J =.5 * (u𝑁 − 𝑢𝑡𝑎𝑟𝑔𝑒𝑡)^2,
• The constraint equation is e = Au − b=0
• ∂J/ ∂u = 0, ∂J/ ∂v = (0,0, . . . ,(𝑢𝑁 − 𝑢𝑡𝑎𝑟𝑔𝑒𝑡))𝑇
• (∂e) / (∂u )=A and (∂e) / (∂v )=I
• Solve Au=b, then equationITλ=−∂J/∂v, thus λ =−∂J/∂v ==(0,0,…,−(𝑢𝑁 − 𝑢𝑡𝑎𝑟𝑔𝑒𝑡))𝑇.
• Solution::dJ/du=0+𝐴𝑇 𝜆 = 𝐴𝑇 𝜆

The Gradient Introduction

$$ \nabla f=\left\langle f_{x}, f_{y}\right\rangle=\left\langle\frac{\partial f}{\partial x}, \frac{\partial f}{\partial y}\right\rangle $$ Example: Find ∇𝑓 for 𝑓(𝑥,𝑦) = 𝑥2y + 𝑒𝑥𝑦 : $$ \begin{aligned} \nabla f&=\left\langle f_{x}, f_{y}\right\rangle=\left\langle 2 x y+y e^{x y}, x^{2}+x e^{x y}\right\rangle \end{aligned} $$

The Gradient Meaning

• Gradient Meaning ∇𝑓(𝑥,𝑦)vector in the steep gradient direction
• Direction of steepest ascent at (a, b) =>∇𝑓(𝑎,𝑏); Slope=>|∇𝑓(𝑎,𝑏)|
• Example Problem: Consider the a terrain which can be defined as a function whose height by 𝑓(𝑥,𝑦)=1000−0.01𝑥^2 −0.02𝑦^2
• Solution: you want to maximize so use maxgradient: ∇𝑓(𝑥,𝑦)=⟨−0.02𝑥,−0.04𝑦⟩ at pt, ∇𝑓(60,80)=⟨−1.2,−3.2⟩ so u need to move in that steep axis of ascent to climb fastest.
• gradient ascent magnitude is |∇𝑓(60,80)|= √11.68≈3.42

Tangent Planes Review:

• equation to the elliptic paraboloid =>z=fx(a,b)(x−a)+fy(a,b)(y−b)+f(a,b)
• Equation to plane can also be defined as:fx(a,b)(x−a)+fy(a,b)(y−b)−(z−f(a,b))=0
• If we use vector notation ro(a, b, f(a, b)) and r(x, y, z), and vector normal defined asfn(fx(a, b), fy(a, b),−1). Using these 3 definitions with plane eq, n⋅ (r− ro)=0
• Example problem:equation of the tangent plane to z==2x^2+y^2 at the point (1,1,3) ?
• Solution: fx==4x fy==2y, thus at pt, fx==4 fy==2 : z==4(x−1)+2(y−1)+3 -> :z=4x+2y−3

Tangent Planes to Level Surfaces

• Surface defined implicitly in terms of functions: 𝐹(𝑥,𝑦,𝑧)=k
• If a point (𝑧0,𝑦0,𝑥0), where 𝐹(𝑥0,𝑦0,𝑧0), then the tangent can can represented as: ∇𝐹(𝑥0,𝑦0,𝑧0)⋅ (⟨𝑥,𝑦,𝑧⟩– 𝑥o,yo,zo⟩)=0
• Final normal vector defined as ∇F(xyz).
• Example:: Equation of tangent plane on the surface 𝑥2+4𝑦2+𝑧2=18 at the point (1,2,1)?
• Solution: Find the gradient to surf. gradient -> ∇𝐹(𝑥,𝑦,𝑧)= (2𝑥,8y, 2z); find pt
• point is ∇F=(2(1),8(2),2(1)) therefore:: 2(x−1)+16(y−2)+2(z−1)=0-> x+8y+z=18

Algorithmic Trading

• Short-term Price Reversals introduction :mean reversion
• A mean equation, where x defined as the average value over it's existence. Ex) stock is temporarily high, it can stay that way as result of outside market forces.
• However, market forces and short term analysis: stocks have "weak/short term" equations.
• For stock, strong mean-reversion is unlikely since new information arrives constantly

Intra-day Mean Reversion Theory

• We find that on 95% confidence ratio we expect any return lies within: (x>-2)& (x<2). If std return >2, next equation is likely negative. If std return<<-2 then positive
• Based on short term prices reversals caused by investors, "over-reacting" is 1 key contributor for stock instability. Rational actors must correct it and balance price
• To model more complex situations, we can use Ornstein-Uhlenbeck process.
• Ornstein-Uhlenbeck Equation: dXt=𝜽(𝝁− Xt)dt+ σdwt; Xt: level, μ:mean, σ: volatility, Wt: equation of Weiner.

Ornstein-Uhlenbeck Process

• 1)Estimate 𝜇 value is the easy, it can be defined as: sample/historical mean that it already has.
• 1. We can also estimate the derivatives using μ,𝜽 ,and σ
• 3. Ornstein 1 time drawback may occur, that it assumes its constantly mean reverting

Pairs Trading Method: Def & Example

• Pairs trading looks for deviation from the historical mean of 2 assets
• Take 𝑙𝑜𝑛𝑔on underperforming and 𝑠ℎ𝑜𝑡ℎigh-performing for profit
• Hypothethically. if the price has been restored, the positions will be closed for a long term gain.
• EXAMPLE : 2 stocks, 𝑥,𝑦 that have good 𝑟=0.87 correlation but the correlation is unstable;
• if 𝛥𝑥>Δ𝑦 ,SHORTthe more outperforming x value AND longterm 𝑦->

Pairs Trading: Identifying, & Drawback

Methods to identify those pairs:

• 1:cor-> can perform regression/find the 𝜌 (pearson)
• 2:Cointegration-> statistical indicator using the cointegrated property, test from it.
• The Drawback of 1 time equation breakage, you can potentially loss on trades you take