Neural Networks: Representation

Neural Networks: Representation

• Neural networks are inspired by the brain's structure
• They are useful in scenarios where the number of features is very large, such as in computer vision tasks
• The complexity of non-linear hypotheses grows rapidly with the number of features

Non-linear Hypotheses Using Neural Networks

• Logistic regression can recognize cars
• Images are very large, so the number of features will also be very large
• For a 50x50 pixel image in RGB, there are 7500 features

Neurons and the Brain

• Neural networks are algorithms that try to mimic the brain
• A neuron is a logistic unit with a sigmoid activation function: $h_\theta(x) = g(\theta^Tx) = g(\sum_{i=0}^n\theta_ix_i)$
• The neuron takes inputs $x_1, x_2, x_3$ and outputs $h_\theta(x)$
• $x_0$ is the bias unit (+1), which is similar to the intercept term
• $\theta$ represents the parameters or weights of the model

Model Representation

• Neural networks consist of layers
• The activation of unit i in layer j is denoted as $a_i^{(j)}$
• $\Theta^{(j)}$ is the matrix of weights that controls the mapping from layer j to layer j+1
• With $s_j$ units in layer j and $s_{j+1}$ units in layer j+1, the dimension of $\Theta^{(j)}$ is $s_{j+1}$ x ($s_j$ + 1)
• $a_i^{(j)} = g(\Theta_{i,0}^{(j-1)}x_0 + \Theta_{i,1}^{(j-1)}x_1+...+\Theta_{i,n}^{(j-1)}x_n)$
• $h_\Theta(x) = a_1^{(3)} = g(\Theta_{1,0}^{(2)}a_0^{(2)} + \Theta_{1,1}^{(2)}a_1^{(2)}+...+\Theta_{1,n}^{(2)}a_n^{(2)})$
• Neural networks can be seen as feature learners

Examples and Intuitions

• Neural networks can learn complex functions
• Can implement the AND function using a simple neural network
• For the AND function, one parameter configuration is $\Theta = [-30, 20, 20]$

Algorithmic Trading Basics

• Algorithmic trading uses pre-programmed instructions to execute orders based on time, price, and volume
• It is also known as automated trading, black-box trading, algo-trading, and systematic trading
• Traders use or create algorithms to generate buy/sell signals, with trades executed automatically

Algorithmic Trading Strategies

• Common strategies includes trend following, mean reversion, arbitrage, and market making
• Statistical arbitrage uses statistical models to trade on pricing discrepancies
• Execution algorithms optimize trade execution by breaking large orders into smaller ones

Benefits of Algorithmic Trading

• Algorithms offer speed, efficiency, and reduced emotional influence
• Backtesting allows optimization using historical data
• Algorithms facilitate portfolio diversification through trading in multiple assets

Risks of Algorithmic Trading

• Technical issues, over-optimization, and market changes can cause malfunction
• Algorithmic trading is subject to regulatory scrutiny
• Can be complex and require specialized knowledge

Tools and Technologies in Algorithmic Trading

• Common tools are programming languages (Python, Java, C++) and trading platforms (MetaTrader, TradingView, Interactive Brokers)
• Data feeds (Bloomberg, Reuters), cloud computing (Amazon Web Services, Google Cloud Platform) and machine learning (TensorFlow, scikit-learn) are beneficial

Linear Regression in Algorithmic Trading

• Equation is $Y = a + bX$
• Used for trend identification and price prediction

Time Series Analysis in Algorithmic Trading

• Used for predicting future price movements
• Involves techniques like moving averages and exponential smoothing

K-Nearest Neighbors (KNN) in Algorithmic Trading

• Classifies a data point based on the majority class of its k-nearest neighbors
• Used for price movement prediction and anomaly detection

Random Forest in Algorithmic Trading

• An ensemble learning method constructing multiple decision trees
• Used for feature selection and trading signal classification

Support Vector Machine (SVM) in Algorithmic Trading

• Finds the optimal hyperplane to separate data into different classes
• Used for pattern recognition and high-dimensional data analysis

Neural Networks in Algorithmic Trading

• Uses interconnected nodes to process information
• Used for complex pattern recognition and high-frequency trading

Naive Bayes in Algorithmic Trading

• Based on Bayes' theorem with feature independence assumptions
• The formula is $P(A|B) = \frac{P(B|A)P(A)}{P(B)}$
• Used for sentiment analysis and categorical data classification
• *Algorithmic trading involves risk and you can lose money

Complex Numbers Defined

• Numbers can be expressed in the form $a + bi$
• 'a' is the real part
• 'b' is the imaginary part
• 'i' is the imaginary unit, where $i = \sqrt{-1}$

Complex Number Examples

• $3 + 2i$
• $-1 - i$
• $4i$ (purely imaginary)
• $5$ (real number, as $5 + 0i$)

Addition of Complex Numbers

• Add the real parts and the imaginary parts separately: $(a + bi) + (c + di) = (a + c) + (b + d)i$
• Example: $(3 + 2i) + (1 - i) = (3 + 1) + (2 - 1)i = 4 + i$

Subtraction of Complex Numbers

• Subtract the real parts and the imaginary parts separately: $(a + bi) - (c + di) = (a - c) + (b - d)i$
• Example: $(3 + 2i) - (1 - i) = (3 - 1) + (2 - (-1))i = 2 + 3i$

Multiplication of Complex Numbers

• Use the distributive property: $(a + bi)(c + di) = ac + adi + bci + bdi^2$
• Since $i^2 = -1$, $(a + bi)(c + di) = (ac - bd) + (ad + bc)i$
• Example: $(3 + 2i)(1 - i) = 3 - 3i + 2i - 2i^2 = 3 - i + 2 = 5 - i$

Division of Complex Numbers

• Multiply the numerator and denominator by the conjugate of the denominator
• $\frac{a + bi}{c + di} = \frac{(a + bi)(c - di)}{(c + di)(c - di)} = \frac{(ac + bd) + (bc - ad)i}{c^2 + d^2}$
• Example: $\frac{3 + 2i}{1 - i} = \frac{(3 + 2i)(1 + i)}{(1 - i)(1 + i)} = \frac{3 + 3i + 2i + 2i^2}{1 - i^2} = \frac{1 + 5i}{2} = \frac{1}{2} + \frac{5}{2}i$

Complex Conjugate Concepts

• The complex conjugate of $a + bi$ is $a - bi$
• $z + \bar{z} = 2Re(z)$, where $Re(z)$ is the real part of $z$
• $z - \bar{z} = 2iIm(z)$, where $Im(z)$ is the imaginary part of $z$
• $z \cdot \bar{z} = |z|^2 = a^2 + b^2$, where $|z|$ is the magnitude of $z$

Magnitude of a Complex Number

• The magnitude of $z = a + bi$ is the distance from the origin to the point $(a, b)$
• $|z| = \sqrt{a^2 + b^2}$
• Example: $|3 + 4i| = \sqrt{3^2 + 4^2} = 5$

Argument of a Complex Number

• The argument is the angle $\theta$ between the positive real axis and the line connecting the origin to $(a, b)$
• $\theta = \arctan\left(\frac{b}{a}\right)$
• Adjust the angle based on the quadrant

Polar Form of Complex Numbers

• $z = r(\cos\theta + i\sin\theta)$
• Where $r = |z|$ and $\theta$ is the argument of $z$

Euler's Formula Foundation

• $e^{i\theta} = \cos\theta + i\sin\theta$
• Can be written as: $z = re^{i\theta}$
• $z = 1 + i$, $r = \sqrt{2}$, $\theta = \frac{\pi}{4}$, thus $z = \sqrt{2}e^{i\frac{\pi}{4}}$

De Moivre's Theorem Definition

• For $z = r(\cos\theta + i\sin\theta)$ and any integer $n$: $z^n = r^n(\cos(n\theta) + i\sin(n\theta))$
• $(re^{i\theta})^n = r^ne^{in\theta}$
• Solving $(1 + i)^4: (1 + i)^4 = (\sqrt{2})^4\left(\cos\left(4 \cdot \frac{\pi}{4}\right) + i\sin\left(4 \cdot \frac{\pi}{4}\right)\right) = -4$

Complex Numbers Usage

• Electrical engineering
• Quantum mechanics
• Fluid dynamics
• Signal processing
• Control systems