Regresión en Aprendizaje Supervisado

Supervised Learning

• The goal is to learn a function that maps an input to its output based on example input-output pairs.

Regression

• Regression is to predict a continuous output given an input.

Regression Algorithms

• Linear Regression: Models the relationship as a linear function, regularized by L1 (Lasso), L2 (Ridge), or Elastic Net.

• Polynomial Regression: Models the relationship as a polynomial function, regularized by L1 (Lasso), L2 (Ridge), or Elastic Net.

• Support Vector Regression: Uses support vectors to find the optimal hyperplane, regularized by L1 or L2.

• Decision Tree Regression: Partitions the input space and models the relationship as a constant within each region with regularization through pruning and limiting tree depth.

• Random Forest Regression: An ensemble of decision trees improving accuracy and reducing overfitting, regulated via pruning and limiting tree depth.

• Gradient Boosting Regression: Builds trees sequentially, each correcting errors of its predecessor, regularized via pruning, limiting tree depth, and learning rate.

• Neural Network Regression: Uses one or more hidden layers for a non-linear mapping with regularization by L1, L2, or Dropout.

Classification

• Classification predicts a discrete output (class label) given an input.

Classification Algorithms

• Logistic Regression: Models the probability of class membership using a sigmoid function, regularized by L1 (Lasso) or L2 (Ridge).

• Support Vector Machine: Finds the optimal hyperplane separating data into classes, regularized by L1 or L2.

• Decision Tree Classification: Partitions input space assigning a class label based on the majority class of training data, regularized by pruning and limiting tree depth.

• Random Forest Classification: An ensemble of decision trees that predicts the class label by prediction aggregation with pruning and limiting tree depth

• Gradient Boosting Classification: An ensemble of decision trees predicting the class label by aggregating predictions, is built sequentially, with each tree correcting the errors of the previous. It is regularized by pruning, limiting tree depth, and learning rate.

• Naive Bayes: Applies Bayes' theorem with strong independence assumptions and involves no regularization.

• K-Nearest Neighbors: Classifies based on the majority class of its k-nearest neighbors without regularization.

• Neural Network Classification: With hidden layers, maps inputs to probability distributions over class labels, predicting the one with the highest probability, and regularized by L1, L2, or Dropout.

Unsupervised Learning

• Finds patterns and structure in data without explicit labels.

Clustering

• Groups similar data points together into clusters.

Clustering Algorithms

• K-Means: Partitions data into k clusters, each point belonging to the nearest mean (centroid), using Euclidean, Manhattan, or Cosine distance metrics.

• Hierarchical Clustering: Builds a hierarchy of clusters by iteratively merging or splitting based on similarity, using Euclidean, Manhattan, or Cosine distance metrics.

• DBSCAN: Groups closely packed data points and marks those in low-density regions as outliers, using Euclidean, Manhattan, or Cosine distance metrics.

• Gaussian Mixture Models: Assumes data is generated from a mixture of Gaussian distributions, estimating parameters to identify clusters via Mahalanobis distance.

Dimensionality Reduction

• Reduces the number of features while preserving essential structure.

Dimensionality Reduction Algorithms

• Principal Component Analysis (PCA): Identifies principal components and projects data onto a lower-dimensional subspace.

• t-distributed Stochastic Neighbor Embedding (t-SNE): Reduces dimensionality while preserving local structure, suited for visualizing high-dimensional data in low dimensions.

Regulation of Gene Expression

• Gene expression is regulated to conserve energy, respond to the environment, and for development processes.

Gene Expression

• Constitutive Genes are always expressed and are often "Housekeeping" genes.
• Regulated Genes have expression turned on or off in response to signals.

Prokaryotic Gene Regulation

• Transcription is controlled at the Promoter, Operator, and Regulatory Genes.

Transcriptional Control

• The Promoter is the RNA polymerase binding site.
• The Operator is the repressor protein binding site.
• The Regulatory Gene codes for the repressor protein.

The lac Operon

• The lac Operon is an inducible operon that codes for lactose metabolism enzymes.

lac Operon: Absence of Lactose

• A repressor protein binds to the operator.
• RNA polymerase cannot bind to the promoter.
• Transcription is blocked.

lac Operon: Presence of Lactose

• Lactose binds to the repressor protein, changing its shape and detaching it from the operator.
• RNA polymerase can bind to the promoter.
• Transcription occurs.

The trp Operon

• The trp Operon is a repressible operonCoding for tryptophan synthesis enzymes.

trp Operon: Absence of Tryptophan

• Repressor protein does not bind to operator.
• RNA polymerase can bind to promoter.
• Transcription occurs.

trp Operon: Presence of Tryptophan

• Tryptophan binds to the repressor protein.
• Repressor protein binds to operator.
• RNA polymerase cannot bind to promoter.
• Transcription is blocked.

Other Transcriptional Controls

• Attenuation: Transcription is initiated but prematurely terminated.
• Riboswitches: mRNA molecules bind to specific metabolites, changing mRNA structure and affecting transcription or translation.

Eukaryotic Gene Regulation

• Eukaryotic gene regulation has Transcriptional and Post-Transcriptional Control.

Transcriptional Control

• Chromatin Modification: including DNA methylation, which represses, and Histone acetylation, which promotes transcription.
• Transcription Factors: Proteins that bind to DNA and regulate transcription.
• Enhancers: DNA sequences that bind activators and increase transcription.
• Silencers: DNA sequences that bind repressors and decrease transcription.

Post-Transcriptional Control

• RNA Processing: Including splicing and mRNA degradation, which impacts translation.
• Translation Control: Including initiation factors and regulatory proteins.

Post-Translational Control

• Protein Modification: Including phosphorylation, glycosylation, and ubiquitination.
• Protein Degradation: Via proteasomes.

Non-Coding RNAs (ncRNAs)

• Gene regulation includes micro RNAs (miRNAs) and Small Interfering RNAs(siRNAs).

Micro RNAs (miRNAs)

• Small RNA molecules block translation or degrade mRNA

Small Interfering RNAs (siRNAs)

• Used in RNA interference (RNAi) to block gene expression.

Cancer and Gene Expression

• Mutations in genes controlling cell growth/division as well as changes in gene expression can lead to cancer.

Development and Gene Expression

• Gene expression patterns control development.
• Homeotic genes control body plans.

Environmental Effects on Gene Expression

• Environmental factors can affect gene expression.
• Epigenetics represents changes not due to changes in DNA sequence, but can be inherited

The Fourier Transform

• The Fourier Transform is used to switch between the time domain and the frequency domain representations of a signal

Motivation

• A signal can be represented in the time domain $f(t)$ or the frequency domain $F(\omega)$.
• The frequency domain can be more useful for tasks like Compression, Denoising, and Feature extraction.

The Fourier Transform Formula

$$ F(\omega) = \int_{-\infty}^{\infty} f(t)e^{-j\omega t} dt $$

Where:

• $f(t)$ is the signal in the time domain.
• $F(\omega)$ is the signal in the frequency domain.
• $\omega$ is the frequency.
• $t$ is the time.
• $j = \sqrt{-1}$ is the imaginary unit.

Euler's Formula

$$ e^{jx} = cos(x) + j sin(x) $$

The Inverse Fourier Transform Formula

$$ f(t) = \frac{1}{2\pi} \int_{-\infty}^{\infty} F(\omega)e^{j\omega t} d\omega $$

Properties of the Fourier Transform

• Linearity holds for any two signals and constants.
• Time Shifting corresponds to multiplying its Fourier Transform by a complex exponential.
• Time Scaling corresponds to scaling its Fourier Transform and multiplying by a constant.
• Differentiation corresponds to multiplication by $j\omega$ in the frequency domain.
• Convolution corresponds to multiplication in the frequency domain.

Discrete Fourier Transform (DFT)

• The Discrete Fourier Transform (DFT) is a version of the Fourier Transform for discrete signals.

Discrete Fourier Transform (DFT) Definition

• Given a discrete signal $f[n]$ of length $N$, the DFT is defined as: $$ F[k] = \sum_{n=0}^{N-1} f[n]e^{-j2\pi kn/N} $$
• For $k = 0, 1,..., N-1$ Where:
  • $f[n]$ is the signal in the time domain.
  • $F[k]$ is the signal in the frequency domain.
  • $n$ is the sample index in the time domain.
  • $K$ is the sample index in the frequency domain.
  • $N$ is the length of the signal.
  • $j = \sqrt{-1}$ is the imaginary unit.

Inverse Discrete Fourier Transform (IDFT) Formula

$$ f[n] = \frac{1}{N} \sum_{k=0}^{N-1} F[k]e^{j2\pi kn/N} $$

• For $n = 0, 1,..., N-1$

Fast Fourier Transform (FFT)

• The Fast Fourier Transform (FFT) is an algorithm for computing the DFT, with a time complexity of $O(N log N)$.

Application of Fourier Transform

• The Fourier Transform can be used in image compression
• The Fourier Transform can be used in Denoising.
• The Fourier Transform can be used in Feature Extraction.

Resumen

• The Fourier Transform, the Inverse Fourier Transform, the Discrete Fourier Transform and Inverse Discrete Fourier Transform allow for computations in different domains to better solve different problems

Funciones vectoriales de variable real

• Es una función que asigna a cada número real $t$ un vector en el espacio $n$-dimensional denotado por $\mathbb{R}^n$
• $\vec{r}(t) = (f_1(t), f_2(t),..., f_n(t))$
• Cada $f_i(t)$ es una componente de la función vectorial $\vec{r}(t)$

Dominio

• El dominio de una función vectorial $\vec{r}(t)$ es el conjunto de todos los valores de $t$ para los cuales todas las funciones componentes $f_i(t)$ están definidas
• $\qquad \text{Dominio}(\vec{r}(t)) = \bigcap_{i=1}^{n} \text{Dominio}(f_i(t))$

Límite

• El límite de una función vectorial $\vec{r}(t)$ cuando $t$ se acerca a $a$ se define componente a componente
• $\qquad \lim_{t \to a} \vec{r}(t) = (\lim_{t \to a} f_1(t), \lim_{t \to a} f_2(t),..., \lim_{t \to a} f_n(t))$
• siempre que todos los límites de las componentes existan.

Continuidad

• Una función vectorial $\vec{r}(t)$ es continua en $t = a$ si se cumplen las siguientes condiciones
• 1. $\vec{r}(a)$ está definida.
• 2. $\lim_{t \to a} \vec{r}(t)$ existe.
• 3. $\lim_{t \to a} \vec{r}(t) = \vec{r}(a)$

Derivada

• La derivada de una función vectorial $\vec{r}(t)$ se define componente a componente:
• $\qquad \vec{r}'(t) = (f_1'(t), f_2'(t),..., f_n'(t))$
• siempre que existan las derivadas de todas las componentes. La derivada $\vec{r}'(t)$ representa el vector tangente a la curva descrita por $\vec{r}(t)$ en el punto correspondiente a $t$.

Reglas de derivación

• Sean $\vec{r}(t)$ y $\vec{u}(t)$ funciones vectoriales derivables, y $f(t)$ una función escalar derivable. Entonces
• $\frac{d}{dt} [\vec{r}(t) + \vec{u}(t)] = \vec{r}'(t) + \vec{u}'(t)$
• $\frac{d}{dt} [c\vec{r}(t)] = c\vec{r}'(t)$, donde $c$ es una constante.
• $\frac{d}{dt} [f(t)\vec{r}(t)] = f'(t)\vec{r}(t) + f(t)\vec{r}'(t)$
• $\frac{d}{dt} [\vec{r}(t) \cdot \vec{u}(t)] = \vec{r}'(t) \cdot \vec{u}(t) + \vec{r}(t) \cdot \vec{u}'(t)$
• $\frac{d}{dt} [\vec{r}(t) \times \vec{u}(t)] = \vec{r}'(t) \times \vec{u}(t) + \vec{r}(t) \times \vec{u}'(t)$
• $\frac{d}{dt} [\vec{r}(f(t))] = \vec{r}'(f(t))f'(t)$ (Regla de la cadena)

Integral

• La integral de una función vectorial $\vec{r}(t)$ se define componente a componente

Integral Definida

• $\qquad \int \vec{r}(t) dt = (\int f_1(t) dt, \int f_2(t) dt,..., \int f_n(t) dt) + \vec{C}$
• donde $\vec{C}$ es un vector constante de integración.

Integral Definida

:

• $\qquad \int_{a}^{b} \vec{r}(t) dt = (\int_{a}^{b} f_1(t) dt, \int_{a}^{b} f_2(t) dt,..., \int_{a}^{b} f_n(t) dt)$

Aplicaciones

• Las funciones vectoriales son esenciales en física e ingeniería para describir el movimiento de partículas en el espacio, campos vectoriales, y muchas otras aplicaciones

The Laws of Thermodynamics

Zeroth Law

• If two thermodynamic systems are each in thermal equilibrium with a third, then they are in thermal equilibrium with each other.

First Law

• Energy can neither be created nor destroyed, it can only change forms.
• In any process, the total energy of the universe remains the same.
• For a thermodynamic cycle the net heat supplied to the system is equal to the net work done by the system.
• $\Delta U = Q - W$

Second Law

• The entropy of an isolated system not in equilibrium will tend to increase over time, approaching a maximum value at equilibrium.
• $\Delta S \ge 0$

Third Law

• As temperature approaches absolute zero, the entropy of a system approaches a minimum or zero value.
• $T \to 0, \Delta S \to 0$

Entropy

• Entropy Changes For Closed Systems

• $\Delta S = \int_{1}^{2} \frac{\delta Q}{T} + S_{gen}$
  • Where:
    • $\Delta S$ is the change in entropy
    • $\delta Q$ is the heat transfer
    • $T$ is the absolute temperature
    • $S_{gen}$ is the entropy generated
• For a reversible process:
  • $\Delta S = \int_{1}^{2} \frac{\delta Q}{T}$
• For an irreversible process:
  • $\Delta S > \int_{1}^{2} \frac{\delta Q}{T}$

Entropy Change of an Ideal Gas

• Constant Specific Heats:
  • $S_2 - S_1 = C_v \ln{\frac{T_2}{T_1}} + R\ln{\frac{V_2}{V_1}}$
  • $S_2 - S_1 = C_p \ln{\frac{T_2}{T_1}} - R\ln{\frac{P_2}{P_1}}$
• Variable Specific Heats:
  • $S_2 - S_1 = \int_{1}^{2} C_v(T) \frac{dT}{T} + R\ln{\frac{V_2}{V_1}}$
  • $S_2 - S_1 = \int_{1}^{2} C_p(T) \frac{dT}{T} - R\ln{\frac{P_2}{P_1}}$

4.1 The Eigenvalue Problem

Definition 4.1.1

• Let $A$ be an $n \times n$ matrix. A scalar $\lambda$ is called an eigenvalue of $A$ if there is a non-zero vector $x$ such that $Ax = \lambda x$
• The vector $x$ is called an eigenvector of $A$ corresponding to the eigenvalue $\lambda$.

Theorem 4.1.1

• Let $A$ be an $n \times n$ matrix. A scalar $\lambda$ is an eigenvalue of $A$ if and only if $\lambda$ is a solution of the characteristic equation $det(A - \lambda I_n) = 0$

Definition 4.1.3

• Let $A$ be an $n \times n$ matrix and let $\lambda$ be an eigenvalue of $A$. The set of all eigenvectors of $A$ corresponding to $\lambda$, together with the zero vector, is called the eigenspace of $A$ corresponding to $\lambda$.

Theorem 4.1.2

• If $A$ is an $n \times n$ matrix, then the following are true:
  • a) The characteristic polynomial of $A$ has degree $n$.
  • b) $A$ has at most $n$ distinct eigenvalues.

Theorem 4.1.3

• If $A$ is an $n \times n$ matrix, then the following are true:
  • a) For each eigenvalue $\lambda$, $1 \leq$ (dimension of the eigenspace corresponding to $\lambda$) $\leq$ (algebraic multiplicity of $\lambda$).
  • b) If $A$ has $k$ distinct eigenvalues, then $k \leq n$.
  • c) The sum of the dimensions of the eigenspaces of $A$ is less than or equal to $n$.

Funciones Vectoriales de Variable Real

• Una función vectorial de variable real es una función $\overrightarrow{r}: I \subseteq \mathbb{R} \longrightarrow \mathbb{R}^{2}$ tal que a cada $t \in I$ le asigna un vector $\overrightarrow{r}(t)=(f(t), g(t))$, donde $f, g: I \longrightarrow \mathbb{R}$ son funciones reales de variable real.

• La gráfica de la función vectorial $\overrightarrow{r}$ es el conjunto de puntos $(x, y) \in \mathbb{R}^{2}$ tales que $(x, y)=(f(t), g(t))$ para algún $t \in I$. A este conjunto de puntos se le llama curva plana.

Derivada de una función vectorial

• Sea $\overrightarrow{r}: I \subseteq \mathbb{R} \longrightarrow \mathbb{R}^{2}$ una función vectorial y $t_{0} \in I$. Se dice que $\overrightarrow{r}$ es derivable en $t_{0}$ si existe el siguiente límite: $\lim {h \rightarrow 0} \frac{\overrightarrow{r}\left(t{0}+h\right)-\overrightarrow{r}\left(t_{0}\right)}{h}$
• A este límite se le llama derivada de $\overrightarrow{r}$ en $t_{0}$ y se denota por $\overrightarrow{r}^{\prime}\left(t_{0}\right)$.
• Si $\overrightarrow{r}(t)=(f(t), g(t))$, entonces $\overrightarrow{r}^{\prime}(t)=\left(f^{\prime}(t), g^{\prime}(t)\right)$

Interpretación geométrica de la derivada

• Si $\overrightarrow{r}(t)$ es una función vectorial que describe una curva en el plano, entonces $\overrightarrow{r}^{\prime}(t)$ es un vector tangente a la curva en el punto $\overrightarrow{r}(t)$.

Propiedades de la derivada

• Sean $\overrightarrow{r}, \overrightarrow{s}: I \subseteq \mathbb{R} \longrightarrow \mathbb{R}^{2}$ funciones vectoriales derivables y $c \in \mathbb{R}$. Entonces:
  • $(\overrightarrow{r}+\overrightarrow{s})^{\prime}(t)=\overrightarrow{r}^{\prime}(t)+\overrightarrow{s}^{\prime}(t)$
  • $(c \overrightarrow{r})^{\prime}(t)=c \overrightarrow{r}^{\prime}(t)$
  • $(\overrightarrow{r} \cdot \overrightarrow{s})^{\prime}(t)=\overrightarrow{r}^{\prime}(t) \cdot \overrightarrow{s}(t)+\overrightarrow{r}(t) \cdot \overrightarrow{s}^{\prime}(t)$

Integral de una función vectorial

• Sea $\overrightarrow{r}(t)=(f(t), g(t))$ una función vectorial, donde $f$ y $g$ son funciones continuas en el intervalo $[a, b]$. Entonces, la integral de $\overrightarrow{r}$ desde $a$ hasta $b$ se define como: $\int_{a}^{b} \overrightarrow{r}(t) d t=\left(\int_{a}^{b} f(t) d t, \int_{a}^{b} g(t) d t\right)$

Algèbre linéaire Cours et exercices corrigés

• Claude Deschamps - André Warusfel
  • Avec la collaboration de
  • J.P. Ramis - Odile Paulo - J. Chevalet
• controllable linear system
  • $\begin{cases} x' = Ax + Bu \ y = Cx + Du \end{cases}$
    • where
      • $x \in \mathbb{R}^n$ is the state of the system
      • $u \in \mathbb{R}^m$ is the control input
      • $y \in \mathbb{R}^p$ is the output
      • $A \in \mathbb{R}^{n \times n}$, $B \in \mathbb{R}^{n \times m}$, $C \in \mathbb{R}^{p \times n}$, $D \in \mathbb{R}^{p \times m}$

Algorithmic Trading

• Algorithmic trading uses computer programs to execute trades based on predefined instructions.

Types of Algorithmic Trading Strategies

• Trend Following
  • Identify and capitalize on the direction of market trends.
  • Technical indicators and moving averages are commonly used.
• Arbitrage
  • Exploit price differences of the same asset across different markets
  • Requires real-time data and rapid execution.
• Mean Reversion
  • Identify assets whose prices have deviated from their average.
  • Bet on prices reverting to their mean.
• Statistical Arbitrage
  • Use statistical models to identify pricing discrepancies.
  • Involves complex algorithms and large datasets.
• Market Making
  • Place buy and sell orders to provide liquidity to the market.
  • Profit from the spread between bid and ask prices.

Key Considerations

• Backtesting
  • Test the strategy on historical data to evaluate performance.
  • Optimize parameters and refine the algorithm.
• Risk Management
  • Implement stop-loss orders and position sizing techniques.
  • Monitor portfolio risk and adjust strategy accordingly.
• Execution Speed
  • Minimize latency and ensure timely order execution.
  • Co-location services and direct market access are beneficial.
• Data Analysis
  • Utilize real-time market data and news feeds.
  • Analyze data to identify trading opportunities and patterns.

Pros and Cons

• Pros
  • Increased efficiency: Automated execution reduces manual errors.
  • Faster execution: Algorithms can react quickly to market changes.
  • Reduced emotional bias: Decisions are based on predefined rules.
  • Backtesting capabilities: Evaluate strategy performance on historical data.
• Cons
  • Technical expertise required: Programming and data analysis skills are necessary.
  • Over-optimization: Risk of creating a strategy that performs well on historical data but poorly in live trading.
  • Unexpected events: Algorithms may not handle unforeseen market conditions effectively.
  • Maintenance and monitoring: Requires ongoing maintenance and monitoring to ensure proper functioning.

Calculus

• Calculus is the mathematics of continuously changing quantities.

What is Calculus?

• Calculus allows us to study the instantaneous speed at any moment.

The Big Ideas of Calculus

• Differential Calculus: Tangent lines and the rate of change of a function.
• Integral Calculus: Area under the curve and the accumulation of a function.

Why is Calculus Important?

• Calculus is used in:
  • Physics (motion, energy)
  • Engineering (design, optimization)
  • Economics (modeling markets)
  • Computer Science (machine learning, graphics)
• Essentially any field that uses mathematical modeling.

Key Concepts in Calculus

• The foundation of calculus
• $\lim_{x \to a} f(x) = L$
• What value does $f(x)$ approach as $x$ gets closer and closer to $a$?

Derivatives

• • The slope of the tangent line.
• $f'(x) = \frac{dy}{dx} = \lim_{h \to 0} \frac{f(x+h) - f(x)}{h}$

Integrals

• The area under the curve
  • $\int_{a}^{b} f(x) , dx$

Fundamental Theorem of Calculus

• Connects derivatives and integrals. $\int_{a}^{b} f'(x) , dx = f(b) - f(a)$

Summary

• Calculus is a powerful tool for understanding change and accumulation. It has two main branches: differential calculus and integral calculus, which are connected by the Fundamental Theorem of Calculus.