Bayesian vs. Frequentist Methods

Bayesian vs. Frequentist Approaches

• Frequentist methods aim to find the single best parameter value θ, treating it as a fixed unknown constant. Uncertainty is measured using confidence intervals and p-values. Prediction involves plugging the 'best' θ into the model. Can overfit predictions; usually computationally cheaper. Most useful with no prior, abundant data, and only caring about the best answer
• Bayesian methods determine the distribution $p(\theta)$ - treating theta as a random variable - encoded directly by the posterior $p(\theta|D)$. Prediction involves integrating over all possible values of $\theta$, weighted by their posterior probability: $p(x|D) = \int p(x|\theta) p(\theta|D) d\theta$. Bayesian methods never overfit (if you can do the integrals), explicit encoding in a prior distribution $p(\theta)$. The approach can be very expensive computationally
• Bayesian approach is useful when there is a good prior; useful when data is scarce; useful when needing to quantify uncertainty

Bayesian Linear Regression

• Model: $y = \theta^T x + \epsilon$, where $\epsilon \sim \mathcal{N}(0, \sigma^2)$.

Frequentist Approach to Linear Regression

• Seeks the "best" $\theta$ by minimizing the sum of squared errors:
• $\theta^* = \arg \min_\theta \sum_{i=1}^N (y_i - \theta^T x_i)^2 = (X^T X)^{-1} X^T y$
• $X = \begin{bmatrix}
  • & x_1^T & - \
  • & x_2^T & - \ & \vdots & \
  • & x_N^T & - \end{bmatrix}$, $y = \begin{bmatrix} y_1 \ y_2 \ \vdots \ y_N \end{bmatrix}$
• It provides a single point estimate $\theta^*$.

Bayesian Approach to Linear Regression

• Compute a distribution over $\theta$ instead of single "best" $\theta$
• Prior: Specify a prior distribution $p(\theta)$. It encodes beliefs about $\theta$ before seeing any data.
• Likelihood: Specify the likelihood $p(y|X, \theta)$. It specifies the probability of seeing the data $y$ given the input $X$ and parameters $\theta$.
• Posterior: Use Bayes' rule to compute the posterior distribution $p(\theta|X, y)$:
• $p(\theta|X, y) = \frac{p(y|X, \theta) p(\theta)}{p(y|X)} = \frac{p(y|X, \theta) p(\theta)}{\int p(y|X, \theta) p(\theta) d\theta}$.
• The posterior distribution encodes beliefs about $\theta$ after seeing the data.

Choosing a Prior

• A Gaussian prior is common for Bayesian Linear Regression:
• $p(\theta) = \mathcal{N}(\mu_0, \Sigma_0)$,
• $\mu_0$ is the prior mean and $\Sigma_0$ is the prior covariance. The prior encodes beliefs about the plausible values of $\theta$ before seeing any data.

Computing the Posterior

• With a Gaussian prior and a Gaussian likelihood, the posterior is also Gaussian:
• $p(\theta|X, y) = \mathcal{N}(\mu_N, \Sigma_N)$,
• $\Sigma_N = (\Sigma_0^{-1} + \frac{1}{\sigma^2} X^T X)^{-1}$
• $\mu_N = \Sigma_N (\Sigma_0^{-1} \mu_0 + \frac{1}{\sigma^2} X^T y)$

Luku 1: Vektorit

1.1 Vektorit tasossa ja avaruudessa

• Reaalilukukentän $\mathbb{R}$ vektoriavaruus $\mathbb{R}^n = {(x_1, x_2,..., x_n) | x_i \in \mathbb{R} }$
• Vektorien yhteenlasku: $\bar{x} + \bar{y} = (x_1 + y_1, x_2 + y_2,..., x_n + y_n)$
• Skalaarikertolasku: $\lambda\bar{x} = (\lambda x_1, \lambda x_2,..., \lambda x_n)$
• Euklidinen normi: $||\bar{x}|| = \sqrt{x_1^2 + x_2^2 +... + x_n^2}$
• Huomautus: $||\bar{x}|| \geq 0$ kaikilla $\bar{x} \in \mathbb{R}^n$ ja $||\bar{x}|| = 0$ ainoastaan kun $\bar{x} = \bar{0}$. Lisäksi $||\lambda\bar{x}|| = |\lambda|||\bar{x}||$

1.2 Pistetulo

• $\bar{x}, \bar{y} \in \mathbb{R}^n$. Tällöin $\bar{x} \cdot \bar{y} = x_1y_1 + x_2y_2 +... + x_ny_n = \sum_{i=1}^{n} x_iy_i$
• Pistetulo on skalaari
• $\bar{x} \cdot \bar{x} = ||\bar{x}||^2$
• Lause: $\bar{x}, \bar{y} \in \mathbb{R}^n$. Tällöin $\bar{x} \cdot \bar{y} = ||\bar{x}|| \cdot ||\bar{y}|| cos(\theta)$, missä $\theta$ on $\bar{x}:n$ ja $\bar{y}:n$ välinen kulma.
• Seuraus: (Cauchy-Schwarzin epäyhtälö): $|\bar{x} \cdot \bar{y}| \leq ||\bar{x}|| \cdot ||\bar{y}||$
• Määritelmä: $\bar{x}$ ja $\bar{y}$ ovat kohtisuorassa, jos $\bar{x} \cdot \bar{y} = 0$

1.3 Vektorien lineaarinen riippuvuus ja riippumattomuus

• Vektorit $\bar{x_1}, \bar{x_2},..., \bar{x_k} \in \mathbb{R}^n$ ovat lineaarisesti riippuvia, jos on olemassa skalaarit $\lambda_1, \lambda_2,..., \lambda_k \in \mathbb{R}$, jotka eivät kaikki ole nollia, siten, että $\lambda_1\bar{x_1} + \lambda_2\bar{x_2} +... + \lambda_k\bar{x_k} = \bar{0}$. Jos vektorit eivät ole lineaarisesti riippuvia, ne ovat lineaarisesti riippumattomia.
• Vektorit $\bar{x_1}, \bar{x_2},..., \bar{x_k} \in \mathbb{R}^n$ ovat lineaarisesti riippumattomia, jos $\lambda_1\bar{x_1} + \lambda_2\bar{x_2} +... + \lambda_k\bar{x_k} = \bar{0}$ ainoastaan kun $\lambda_1 = \lambda_2 =... = \lambda_k = 0$.
• Lause: Jos vektorit $\bar{x_1}, \bar{x_2},..., \bar{x_k} \in \mathbb{R}^n$ ovat lineaarisesti riippumattomia, niin $k \leq n$.

1.4 Vektorijoukkojen virittäminen

• Vektoriavaruuden $V$ vektorijoukko ${\bar{x_1}, \bar{x_2},..., \bar{x_k}}$ virittää $V:n$, jos jokainen vektori $\bar{x} \in V$ voidaan kirjoittaa lineaarikombinaationa $\bar{x} = \lambda_1\bar{x_1} + \lambda_2\bar{x_2} +... + \lambda_k\bar{x_k}$, missä $\lambda_1, \lambda_2,..., \lambda_k \in \mathbb{R}$.
• Vektoriavaruuden $V$ kanta on joukko lineaarisesti riippumattomia vektoreita, jotka virittävät $V:n$.
• Vektoriavaruudella $V$ on useita kantoja, mutta kaikissa kannoissa on sama määrä vektoreita. Tämä lukumäärä on $V:n$ dimensio, $dim(V)$.
• $\mathbb{R}^n:n$ standardikanta on ${\bar{e_1}, \bar{e_2},..., \bar{e_n}}$, missä $\bar{e_i} = (0, 0,..., 1,..., 0)$. Tällöin $dim(\mathbb{R}^n) = n$.

3. Projections

3.1 Motivation

• Our world is 3D, but displays are 2D

3.2 Taxonomy

3.2.1 Orthographic Projection

• Parallel lines remain parallel
• Good for technical drawings
• Preserves both lengths and angles
• Not realistic

3.2.2 Perspective Projection

• Parallel lines converge to a single point
• Mimics how our eyes and cameras work
• More realistic

3.3 Orthographic Projection

3.3.1 Definition

• Also called "parallel projection"
• Project along a direction (DOP: direction of projection) onto the projection plane
• Assume DOP is z-axis, projection plane is z=0
• Then $(x, y, z) \rightarrow (x, y)$

3.3.2 Variants

• Multi-view orthographic projection
• Projection plane is parallel to one of the principal planes (xy, yz, zx)
• Also called "axonometric projection"
• Oblique projection
• Projection plane is not parallel to any of the principal planes

3.4 Perspective Projection

3.4.1 Definition

• Project points to the center of projection (COP)
• The projection plane is where the image is formed
• Also called a "central projection"

3.4.2 Math

• Assume COP is at the origin, projection plane is z=d
• A point (x,y,z) is projected to (x',y',d)
• These three points (x,y,z), (0,0,0), and (x',y',d) are collinear
• Therefore, $\begin{bmatrix} x'\ y'\ d \end{bmatrix} = \lambda \begin{bmatrix} x\ y\ z \end{bmatrix}$
• $\frac{x'}{x} = \frac{y'}{y} = \frac{d}{z}$
• $(x', y') = (\frac{dx}{z}, \frac{dy}{z})$
• Note that z cannot be zero

3.4.3 Homogeneous Coordinates

• Perspective projection becomes a linear operation in homogeneous coordinates
• $(x, y, z, 1) \rightarrow (\frac{dx}{z}, \frac{dy}{z}, d, 1) \sim (dx, dy, dz, z)$
• $\begin{bmatrix} x'\ y'\ z'\ w' \end{bmatrix} = \begin{bmatrix} d & 0 & 0 & 0\ 0 & d & 0 & 0\ 0 & 0 & d & 0\ 0 & 0 & 1 & 0 \end{bmatrix} \begin{bmatrix} x\ y\ z\ 1 \end{bmatrix}$
• $(x', y', z') = (\frac{dx}{z}, \frac{dy}{z}, d)$

3.5 Field of View

3.5.1 Definition

• The amount of the scene that is visible on the image plane
• Determined by the camera's lens and the size of the image sensor

3.5.2 Vertical Field of View

$\theta = 2arctan(\frac{h}{2d})$

• h is the height of the image sensor
• d is the distance from the lens to the image sensor

3.6 Vanishing Points

3.6.1 Definition

• The point where parallel lines appear to converge in a perspective projection
• Each set of parallel lines has its own vanishing point
• The vanishing point is the projection of the point at infinity in the direction of the lines

3.6.2 Number of Vanishing Points

• 1-point perspective: one vanishing point
• 2-point perspective: two vanishing points
• 3-point perspective: three vanishing points

3.6.3 Example

• The picture shows an example of 3-point perspective. There are three vanishing points, each corresponding to one of the three principal axes.

Algorithmic Trading and Order Execution

Definitions

• Algorithmic Trading: Utilizing computer programs to execute orders based on pre-defined instructions.
• Direct Market Access (DMA): Accessing the order book directly, bypassing traditional brokers.
• Order Book: Electronic list of buy and sell orders for a specific security, organized by price level.
• VWAP (Volume Weighted Average Price): Average price of a security weighted by its trading volume.

Advantages

• Reduced transaction costs
• Improved order execution speed
• Increased market access
• Enhanced ability to implement complex trading strategies
• Better record keeping
• Reduced probability of error

Execution Algorithms

• Volume Weighted Average Price (VWAP)
  • Objective: Execute a large order at or better than the VWAP.
  • Mechanics: Breaks large order into smaller pieces released over a period of time, based on historical or real-time volume patterns.
  • Suitable for: Minimizing market impact of large orders.
• Implementation Shortfall
  • Objective: Minimize the difference between the actual execution price and the decision price.
  • Mechanics: Balances the trade-off between immediate execution and waiting for a better price.
  • Suitable for: Traders with a strong view on the direction of the market.
• Target Close
  • Objective: Execute an order as close as possible to the closing price.
  • Mechanics: Algorithm increases order size as the close approaches.
  • Suitable for: Traders who need to be executed by the end of the day.
• Percentage of Volume (POV)
  • Objective: Participate in a specified percentage of the market volume.
  • Mechanics: Algorithm adjusts order size to maintain target participation rate.
  • Suitable for: Maintaining stealth while executing a large order.

Strategic Algorithms

• Pairs Trading
  • Objective: Exploit temporary mispricing between two historically correlated assets.
  • Mechanics: Simultaneously buy the undervalued asset and sell the overvalued asset, profiting when the relationship reverts to its mean.
  • Suitable for: Market-neutral strategies.
• Index Arbitrage
  • Objective: Exploit price differences between an index and its constituent stocks.
  • Mechanics: Simultaneously buy the index and sell the constituent stocks, or vice versa, to profit from the arbitrage.
  • Suitable for: Taking advantage of inefficiencies between index and component prices.
• Basket Trading
  • Objective: Execute a trade in a group of assets simultaneously.
  • Mechanics: Programmed to trade a list of stocks or other assets based on specific parameters.
  • Suitable for: Portfolio rebalancing or implementing investment strategies across a group of assets.

High-Frequency Trading (HFT)

• Characteristics
  • Extremely short-term investment horizon (milliseconds/seconds)
  • High portfolio turnover
  • Located very close to exchanges
  • Focus on small profits from each trade
• Strategies
  • Market Making: Providing liquidity by placing bid and offer orders
  • Latency Arbitrage: Exploiting tiny price differences due to information delays
  • Event Arbitrage: Trading based on news or events as soon as they are released

Order Placement

Order Types

• Market Order: Executed immediately at the best available price.
• Limit Order: Executed only at the specified price or better.
• Stop Order: Becomes a market order when the stop price is reached.
• Stop-Limit Order: Becomes a limit order when the stop price is reached.

Order Instructions

• Fill or Kill (FOK): Order must be executed in its entirety immediately, or it is canceled.
• Immediate or Cancel (IOC): Any portion of the order that cannot be immediately executed is canceled.
• All or None (AON): Order must be executed in its entirety, but not necessarily immediately.

Dark Pools

• Definition: Private exchanges or forums for trading securities.
• Characteristics:
  • Offer anonymity
  • Less market impact (for large trades)
  • Potential for price improvement

Regulation

• Purpose: To ensure fair and orderly markets, prevent manipulation, and protect investors.
• Key Areas of Focus:
  • Transparency of algorithmic trading strategies
  • Prevention of abusive trading practices (e.g., spoofing, layering)
  • System safeguards and risk controls for algorithmic trading firms

Lecture 19: The Simplex Method

1. Basic Idea

• The simplex method is an algorithm to solve linear programs.
• Assuming that we have a linear program in canonical form, the basic idea is as follows:
  1. Start with a basic feasible solution (BFS)
  2. Repeatedly improve the BFS by moving to a neighboring BFS with a better objective value.
• Since there are finitely many BFSs, we will eventually reach the optimal BFS.
• Two questions arise
  • How do we find an initial BFS?
  • How do we move to a neighboring BFS with a better objective value?

2. Moving to a Neighboring BFS

• Consider a linear program in canonical form: maximize $\quad c^T x$ subject to $Ax = b$ $x \geq 0$
• Where $A$ is an $m \times n$ matrix.
• Suppose we have a BFS $x = \begin{bmatrix} x_B \ x_N \end{bmatrix} = \begin{bmatrix} B^{-1}b \ 0 \end{bmatrix}$
• Where $B$ is a basis and $N$ is the set of nonbasic variables.
• We want to find a neighboring BFS with a better objective value.
• The idea is to increase the value of one of the nonbasic variables from 0 to a positive value.
• Let $x_j$ be a nonbasic variable that we want to increase.
• Then, we need to update the values of the basic variables to maintain $Ax = b$.
• That is, we need to find $\Delta x_B$ such that $B(x_B + \Delta x_B) + A_j x_j = b$
• Since $Bx_B = b$, we have $B \Delta x_B + A_j x_j = 0$
• Thus, $\Delta x_B = -B^{-1} A_j x_j$
• So, the new solution is $\begin{bmatrix} x_B \ x_N \end{bmatrix} = \begin{bmatrix} B^{-1}b - B^{-1}A_j x_j\ x_j \ 0 \end{bmatrix}$

3. Optimality Condition

• When do we know that we have reached the optimal BFS?
• Consider the objective function $c^T x = c_B^T x_B + c_N^T x_N$
• Since $x_B = B^{-1}b - B^{-1}A_j x_j$, we have $c^T x = c_B^T (B^{-1}b - B^{-1}A_j x_j) + c_j x_j$ $= c_B^T B^{-1}b + (c_j - c_B^T B^{-1}A_j) x_j$
• Let $\bar{c_j} = c_j - c_B^T B^{-1}A_j$ be the reduced cost of $x_j$
• If $\bar{c_j} \leq 0$ for all nonbasic variables $x_j$, then the current BFS is optimal.
• Because increasing the value of any nonbasic variable will not improve the objective value.
• If $\bar{c_j} > 0$ for some nonbasic variable $x_j$, then increasing the value of $x_j$ will improve the objective value.

4. The Simplex Algorithm

1. Start with a BFS
2. Compute the reduced costs $\bar{c_j} = c_j - c_B^T B^{-1}A_j$ for all nonbasic variables $x_j$.
3. If $\bar{c_j} \leq 0$ for all nonbasic variables $x_j$, then the current BFS is optimal. Stop.
4. Otherwise, choose a nonbasic variable $x_j$ with $\bar{c_j} > 0$ to increase.
5. Compute $\Delta x_B = -B^{-1} A_j x_j$.
6. Find the basic variable $x_i$ that first becomes 0 as $x_j$ increases.
7. Update the basis by replacing $x_i$ with $x_j$.
8. Go to step 2.

Estadística Descriptiva

¿Qué es la estadística descriptiva?

• Rama de la estadística que describe y resume conjuntos de datos. Proporciona métodos para organizar, presentar y analizar datos de manera que se puedan comprender y comunicar de manera efectiva.

Métodos

• Tablas de frecuencia: Resumen la frecuencia con la que aparecen diferentes valores en un conjunto de datos.
• Gráficos: Representan visualmente los datos, lo que facilita la identificación de patrones y tendencias.
• Medidas de tendencia central: Indican el valor típico de un conjunto de datos (media, mediana, moda).
• Medidas de dispersión: Miden la variabilidad de los datos (rango, desviación estándar, varianza).

Tipos de datos

• Datos cualitativos: Describen cualidades o características no numéricas (color, género, opinión).
• Datos cuantitativos: Representan cantidades numéricas (edad, altura, peso).
  • Discretos: Toman valores enteros (número de hijos, número de coches).
  • Continuos: Pueden tomar cualquier valor dentro de un rango (temperatura, altura).

Tablas de frecuencia

• Muestra la distribución de los datos al listar los valores y su frecuencia.

Gráficos

• Representaciones visuales de los datos que facilitan la comprensión y el análisis.
  • Histograma: Muestra la distribución de datos continuos.
  • Gráfico de barras: Compara las frecuencias de diferentes categorías.
  • Gráfico circular: Muestra la proporción de cada categoría en relación con el total.
  • Diagrama de dispersión: Muestra la relación entre dos variables.

Medidas de tendencia central

• Indican el valor típico de un conjunto de datos.
  • Media ($\mu$): Promedio de los valores: $\mu = \frac{\sum_{i=1}^{n} x_i}{n}$
  • Mediana: Valor central cuando los datos están ordenados.
  • Moda: Valor que aparece con mayor frecuencia.

Medidas de dispersión

• Indican la variabilidad de los datos.
  • Rango: Diferencia entre el valor máximo y el valor mínimo.
  • Varianza ($\sigma^2$): Dispersión promedio de los datos alrededor de la media: $\sigma^2 = \frac{\sum_{i=1}^{n} (x_i - \mu)^2}{n}$
  • Desviación estándar ($\sigma$): Raíz cuadrada de la varianza: $\sigma = \sqrt{\sigma^2}$

Conclusión

• Herramienta fundamental para resumir y comprender conjuntos de datos, facilitando la toma de decisiones y la comunicación de resultados.

STATISTICAL METHODS

LEARNING OBJECTIVES

• Define and explain the terms Statistics and Biostatistics.
• Describe the uses of Statistics in Biology and Medicine.
• Define data and describe different types of data.
• Explain the methods of data collection.
• Explain data processing, presentation, and interpretation.
• Calculate and interpret common measures of central tendency and dispersion.
• Calculate and interpret probabilities and use them to make predictions.
• Describe different types of statistical tests and when to use them.
• Use computer software to perform basic statistical analyses
• Interpret the results of statistical analyses and draw conclusions.

TOPICS

1. Introduction
2. Types of Data
3. Data Collection Methods
4. Data Processing and Presentation
5. Descriptive Statistics
6. Probability
7. Statistical Inference
8. Statistical Tests
9. Statistical Software

Reguläre Ausdrücke

Definition

• Reguläre Ausdrücke sind eine formale Sprache zur Beschreibung von Zeichenkettenmustern.

Bestandteile

• Literalen Zeichen: z.B. a, b, 1, 2
• Meta-Zeichen: Zeichen mit spezieller Bedeutung, z.B. ., *, +, ?, [], ()
• Zeichenklassen: z.B. [a-z], [0-9], \w, \d
• Quantoren: z.B. *, +, ?, {n}, {n,}, {n,m}
• Ankern: z.B. ^, $
• Gruppierungen: z.B. ()
• Alternativen: z.B. |

Meta-Zeichen im Detail

Zeichenklassen im Detail

Quantoren im Detail

Verwendung in Python

• In Python verwendet man das Modul re für reguläre Ausdrücke.

Wichtige Hinweise

• Reguläre Ausdrücke (regex) können komplex und schwer lesbar sein.
• Es ist wichtig, reguläre Ausdrücke zu testen, um sicherzustellen, dass sie das gewünschte Ergebnis liefern.
• Für komplexe Aufgaben kann es sinnvoll sein, auf spezialisierte Bibliotheken zurückzugreifen.
• Achte auf die Performance, da komplexe reguläre Ausdrücke sehr langsam sein können.

Advanced Calculus I (MAT 361)

Functions of Several Variables

• Definition: A real-valued function of n variables is a function whose domain is a subset of ℝ^n and whose range is a subset of R.

Examples

1. Area of a rectangle: $A(l, w) = lw$
2. Volume of a cylinder: $V(r, h) = \pi r^2 h$
3. Ideal Gas Law: $P(n, V, T) = \frac{nRT}{V}$, where $R$ is the ideal gas constant.

Level Sets (Contour Plots)

• The level sets of a function $f(x, y)$ are the curves defined by $f(x, y) = c$, where $c$ is a constant in the range of $f$. These curves are also called contours or level curves.
  • Example: The level sets of $f(x, y) = x^2 + y^2$ are circles centered at the origin.

Graph of a Function of Two Variables

• The graph of a function $f(x, y)$ is the set of all points $(x, y, z)$ in $\mathbb{R}^3$ such that $z = f(x, y)$. This forms a surface in three-dimensional space.

Linear Functions

• A linear function of two variables has the form $f(x, y) = ax + by + c$, where $a$, $b$, and $c$ are constants. The graph of a linear function is a plane.

Limits and Continuity

Definition of a Limit

• Let $f(x, y)$ be a function defined on a disk around $(x_0, y_0)$, except possibly at $(x_0, y_0)$ itself. We say that the limit of $f(x, y)$ as $(x, y)$ approaches $(x_0, y_0)$ is $L$, written as $$ \lim_{(x,y) \to (x_0, y_0)} f(x, y) = L $$
• if for every $\epsilon > 0$, there exists a $\delta > 0$ such that if $0 < \sqrt{(x - x_0)^2 + (y - y_0)^2} < \delta$, then $|f(x, y) - L| < \epsilon$.

Definition of Continuity

• A function $f(x, y)$ is continuous at $(x_0, y_0)$ if

1. $f(x_0, y_0)$ is defined.
2. $\lim_{(x,y) \to (x_0, y_0)} f(x, y)$ exists.
3. $\lim_{(x,y) \to (x_0, y_0)} f(x, y) = f(x_0, y_0)$.

• $f(x, y)$ is continuous on a set $S$ if it is continuous at every point in $S$.

Partial Derivatives

Definition

• The partial derivative of $f$ with respect to $x$ at $(x_0, y_0)$ is defined as $$ \frac{\partial f}{\partial x}(x_0, y_0) = \lim_{h \to 0} \frac{f(x_0 + h, y_0) - f(x_0, y_0)}{h} $$ provided the limit exists. Similarly, the partial derivative of $f$ with respect to $y$ at $(x_0, y_0)$ is defined as $$ \frac{\partial f}{\partial y}(x_0, y_0) = \lim_{h \to 0} \frac{f(x_0, y_0 + h) - f(x_0, y_0)}{h} $$ provided the limit exists.

Notation

• $\frac{\partial f}{\partial x} = f_x$
• $\frac{\partial f}{\partial y} = f_y$

Higher-Order Partial Derivatives

• $f_{xx} = \frac{\partial^2 f}{\partial x^2} = \frac{\partial}{\partial x} \left( \frac{\partial f}{\partial x} \right)$
• $f_{yy} = \frac{\partial^2 f}{\partial y^2} = \frac{\partial}{\partial y} \left( \frac{\partial f}{\partial y} \right)$
• $f_{xy} = \frac{\partial^2 f}{\partial y \partial x} = \frac{\partial}{\partial y} \left( \frac{\partial f}{\partial x} \right)$
• $f_{yx} = \frac{\partial^2 f}{\partial x \partial y} = \frac{\partial}{\partial x} \left( \frac{\partial f}{\partial y} \right)$

Clairaut's Theorem

• If $f_{xy}$ and $f_{yx}$ are continuous on an open set containing $(x_0, y_0)$, then $f_{xy}(x_0, y_0) = f_{yx}(x_0, y_0)$.

Introduction to Probability

Basic Definitions

Random Experiment

• An experiment whose outcome cannot be predicted with certainty.

Sample Space