Expectation-Maximization (EM) Algorithm

Expectation-Maximization (EM) Algorithm

• EM algorithm is an iterative technique for finding maximum likelihood estimates in models with latent variables.
• Latent variables are unobserved and complicate direct likelihood maximization.
• The EM algorithm alternates between:
  • E-step: computes the expected value of latent variables given observed data and current parameter estimates.
  • M-step: updates parameters to maximize the expected complete-data log-likelihood.

Latent Variable Models

• EM algorithm is useful for mixture models, Hidden Markov Models and Factor analysis

Advantages of EM Algorithm

• No closed form solution required

Applications of EM Algorithm

• EM Algorithm is useful for clustering, bioinformatics and natural language processing

Gaussian Mixture Model (GMM)

• GMM is a probabilistic model where data points are generated from a mixture of Gaussian distributions with unknown parameters.

GMM Algorithm

E-step

• Evaluate the "responsibilities" using current parameter values: $\gamma(z_{nk}) = \frac{\pi_k N(\mathbf{x}n | \mu_k, \Sigma_k)}{\sum{j=1}^{K} \pi_j N(\mathbf{x}_n | \mu_j, \Sigma_j)}$
  • $N(\mathbf{x}_n | \mu_k, \Sigma_k)$ is the Gaussian density.

M-step

• Re-estimate parameters using current responsibilities:
  • $N_k = \sum_{n=1}^{N} \gamma(z_{nk})$
  • $\mu_k = \frac{1}{N_k} \sum_{n=1}^{N} \gamma(z_{nk}) \mathbf{x}_n$
  • $\Sigma_k = \frac{1}{N_k} \sum_{n=1}^{N} \gamma(z_{nk}) (\mathbf{x}_n - \mu_k)(\mathbf{x}_n - \mu_k)^T$
  • $\pi_k = \frac{N_k}{N}$

Evaluate Log Likelihood

• $log p(\mathbf{X} | \mu, \Sigma, \pi) = \sum_{n=1}^{N} log {\sum_{k=1}^{K} \pi_k N(\mathbf{x}_n | \mu_k, \Sigma_k)}$

K-Means Clustering

Algorithm Steps

• Initialize cluster centroids $\mu_1, \mu_2, \dots, \mu_K$ randomly
• Assign each data point to the nearest centroid
• Update centroids based on the new assignments
• Repeat the point assignment and centroid updates until convergence

Differences Between K-Means and GMM

Cluster Shape

• K-Means clusters are spherical
• GMM clusters are elliptical

Cluster Size

• K-Means clusters have similar sizes
• GMM clusters can have varying sizes

Soft Assignment

• K-Means uses "hard" assignments
• GMM uses "soft" probabilistic assignments

Model

• K-Means is distance-based
• GMM is probabilistic

Convergence

• K-Means convergence is guaranteed toward a solution
• GMM may get to a local optimum point

Initialization

• K-Means results are sensitive to initial location
• GMM results are sensitive to initial conditions

Number of clusters

• K-Means requires the specification of of K
• GMM requires the specification of number of clusters

Unix Environment

Objective

• To familiarize with the Unix environment.

Background

• Unix is a powerful, stable, secure, and flexible command-line based operating system used in computer science.

Connecting to ECF Computers

• Using a terminal emulator:
  • Windows: PuTTY.
  • macOS/Linux: Built-in terminal application.
• Using ECF computers in the labs:
  • Log in using your ECF username and password.
  • Connect to ECF computers via ssh [email protected] and password.

Navigating the File System

• The Unix file system has a hierarchical tree structure with root directory /.

Commands

pwd

• Prints the absolute path of the current directory

cd

• Change the current directory
  • cd..: move to parent directory
  • cd: move to home directory

ls

• List files/directories in the current directory
  • ls -l: list all files/directories with detailed information
  • ls -a: list all files/directories, including hidden ones

Manipulating Files and Directories

mkdir

• Creates a new directory

rmdir

• Removes a directory
  • Directory must be empty

touch

• Creates a new empty file

cp

• Copies a file to a new location

mv

• Moves a file to a new location/renames a file

rm

• Removes a file
  • Use with extreme caution

Common Unix Utilities

cat

• Prints the contents of a file

less

• Views a file's contents one page at a time

head

• Prints the first 10 lines of a file
  • head -n k file_name: prints the first k lines of a file.

tail

• Prints the last 10 lines of a file
  • tail -n k file_name Print the last k lines of a file

grep

• Searches for a pattern in a file and prints matching lines

wc

• Counts the number of lines, words, and characters in a file

sort

• Sorts the lines in a file alphabetically

uniq

• Removes duplicate lines from a file

">"

• Redirects output to a file

"|"

• Pipes the output of one command to the input of another

Autonomic Pharmacology

Introduction

• The autonomic nervous system (ANS) regulates cardiac muscle, smooth muscle, and glands.

Basic Organization of the ANS

Efferent (Motor) Pathways

• It contrasts with somatic nervous system by using two neurons in its efferent pathway.
• Preganglionic neurons synapse with postganglionic neurons in autonomic ganglia. Postganglionic neurons then innervate the effector organ.

Divisions of the ANS

Sympathetic (Thoracolumbar) Division

• Preganglionic neurons in thoracic and lumbar regions of the spinal cord.
• Short preganglionic fibers; ganglia close to spinal cord.
• Long postganglionic fibers to effector organs.

Parasympathetic (Craniosacral) Division

• Preganglionic neurons in brainstem and sacral spinal cord.
• Long preganglionic fibers; ganglia near or within the effector organs.
• Short postganglionic fibers synapsing directly with target cells.

Neurotransmitters of the ANS

Acetylcholine (ACh)

• Primary neurotransmitter in all autonomic ganglia.
• At parasympathetic postganglionic nerve endings and sympathetic postganglionic nerve endings innervating sweat glands.
• Neurons releasing ACh are cholinergic.

Norepinephrine (NE)

• Primary neurotransmitter at most sympathetic postganglionic nerve endings.
• Neurons releasing NE are adrenergic.

Receptors of the ANS

Cholinergic Receptors

• Nicotinic (N) : In autonomic ganglia and the neuromuscular junction; activated by ACh and nicotine.
• **Muscarinic (M) **: On effector organs innervated by parasympathetic postganglionic fibers; activated by ACh and muscarine. Subtypes include M1, M2, M3, M4, and M5.

Adrenergic Receptors

• Divided into alpha ($\alpha$) and beta ($\beta$) receptors, with subtypes $\alpha_1$, $\alpha_2$, $\beta_1$, $\beta_2$, and $\beta_3$.
• Each has different affinities for adrenergic agonists/antagonists and mediates different effects.

Synthesis, Release, and Breakdown of Neurotransmitters

Acetylcholine

• Choline + Acetyl CoA $\rightarrow$ ACh via Choline acetyltransferase
• ACh degraded by acetylcholinesterase (AChE) in the synaptic cleft.

Norepinephrine

• Tyrosine $\rightarrow$ DOPA $\rightarrow$ Dopamine $\rightarrow$ NE
• NE removed by reuptake or degraded by monoamine oxidase (MAO) or catechol-O-methyltransferase (COMT).

Functions of the Sympathetic and Parasympathetic Nervous Systems

Heart

• Sympathetic Effects: Increased heart rate, increased contractility
• Parasympathetic Effects: Decreased heart rate, decreased contractility

Blood Vessels

• Sympathetic Effects: Constriction ( $\alpha_1$ receptors), dilation ( $\beta_2$ receptors)
• Parasympathetic Effects: Dilation

Bronchioles

• Sympathetic Effects: Dilation ( $\beta_2$ receptors)
• Parasympathetic Effects: Constriction

GI Tract

• Sympathetic Effects: Decreased motility and secretion
• Parasympathetic Effects: Increased motility and secretion

Urinary Bladder

• Sympathetic Effects: Relaxation of detrusor muscle, contraction of sphincter
• Parasympathetic Effects: Contraction of detrusor muscle, relaxation of sphincter

Eye

• Sympathetic Effects: Mydriasis (pupil dilation)
• Parasympathetic Effects: Miosis (pupil constriction)

Glands

• Sympathetic Effects: Increased sweating (sympathetic cholinergic), increased secretion of thick saliva
• Parasympathetic Effects: Increased secretion of watery saliva, increased lacrimation

Metabolic Effects

• Sympathetic Effects: Increased glycogenolysis, lipolysis.
• Parasympathetic Effects: None.

Adrenal Medulla

• Specialized part of the sympathetic nervous system.
• Releases epinephrine and norepinephrine directly into the bloodstream.

Clinical Applications

• Drugs that mimic or block the actions of the ANS are used to treat a wide range of conditions, including hypertension, asthma, glaucoma, and overactive bladder.

Important Considerations

Receptor Selectivity

• Selectively targeting receptor subtypes for minimal side effects.

Drug Interactions

• Considering potential drug interactions related to the ANS.

Disease States

• Disease states can alter ANS function and drug response.

Receptor Locations and Effects Overview

$\alpha_1$

• Location: Smooth muscle (blood vessels, sphincters), eye
• Effects: Vasoconstriction, increased peripheral resistance, mydriasis, increased closure of internal sphincter of bladder

$\alpha_2$

• Location: Presynaptic nerve terminals
• Effects: Inhibition of NE release

$\beta_1$

• Location: Heart, kidney
• Effects: Increase heart rate, increase contractility, increase renin secretion

$\beta_2$

• Location: Smooth muscle (bronchioles, blood vessels), uterus, liver
• Effects: Bronchodilation, vasodilation, uterine relaxation, glycogenolysis

$\beta_3$

• Location: Adipose tissue
• Effects: Lipolysis

$M_1$

• Location: CNS, gastric parietal cells
• Effects: Increase cognition, gastric acid secretion

$M_2$

• Location: Heart, smooth muscle
• Effects: Decrease heart rate, decrease contractility, smooth muscle contraction

$M_3$

• Location: Smooth muscle, glands, endothelium
• Effects: Smooth muscle contraction, increase secretion, vasodilation (via NO release)

$N_N$

• Location: Autonomic ganglia
• Effects: Ganglionic transmission

$N_M$

• Location: Neuromuscular junction
• Effects: Skeletal muscle contraction

GeoGebra Quick Start Guide

Interface

Menu Bar

• Offers options for File, Edit, View, Options, Tools, Window, and Help.

Toolbar

• Provides tools for constructing geometric objects.

Algebraic View

• Displays the algebraic representation of constructed objects.

Graphic View

• Presents the geometric representation of constructed objects.

Input Bar

• Allows the introduction of commands and algebraic expressions.

Moving Objects

• Select the "Move" tool and drag the desired object.

Creating Points

• Select the "Point" tool & click on the Graphic View
• Enter the x,y coordinates in the Input Bar

Creating Lines

• Select the "Line through Two Points" tool
• Select two existing points
• Enter an equation in the Input Bar like y = mx + c

Creating Circles

• Select the "Circle with Center and Radius" tool
• Select the center, then select again to set the radius
• Enter an algebraic equation in the Input Bar

Toolbar Tools

Mueve

• Move tool

Punto

• Tool for creating points.

Recta - Line Tool

Circunferencia - circle tool

Recta

• Line tool

Polígono -

• Polygon Tool

Ángulo

• Measure angles

Distancia o longitud

• Measuring distances.

Reflexión axial

• Axis reflection tool

Deslizador - Slider Tool

Deslizador

• Slider tool

Texto

• Insert text

Views

Vista Gráfica

• Geometric

Vista Algebraica

• Algebra

Hoja de Cálculo

• Spreadsheet

CAS (Cálculo Simbólico)

• Symbolic calculations like solve, derivate, integrate

Gráficos 3D - 3D Graphics View

Probabilidad

• Probability calculations

CAS Window (Symbolic Calculation)

• Used for symbolic algebraic computations.

Cas Eamples

• Resolver(x^2 + 2x + 1 = 0, x) - solve the equation in x
• Derivada(x^3 + 2x^2 + x) - derivative
• Integral(x^2 + 1, x) - integrate function
• Simplifica((x+1)^2 - (x^2 + 1)) - simplify expression

Basic Commands

Punto()

• Create point on object

Recta( , )

• Create a line

Circunferencia() - Create Circle

Circunferencia( , )

• tool to Create circle

Polígono( , ,... )

• Polygon Tool

Ángulo( , , )

• Angle Tool

Distancia - distance calculator

Distancia( , )

• Distance calculator

Refleja( , )

• Reflect object

Examples

Equilateral triangle construction

• Create two points A and B.
• Create the circle with center at A and radius AB.
• Create the circle with center at B and radius BA.
• Create intersection points C and D of the two circles.
• Create the polygon ABC (or ABD) to create the equilateral triangle.

Calculate Circle Area

• Create the A point as circle center
• Create the r slider for radiuis value
• Create the circle with center in A and radius r
• Insert in input bar this value Area = pi * r^2

Quadratic equation solve

• Open the CAS window.
• Insert this command: Resolver(a*x^2 + b*x + c = 0, x)
• Insert the value of coefficients a, b and c
• Get the equation solutions

Additional Resources

GeoGebra Website

• www.geogebra.org

Calculus Cheat Sheet

Derivatives

Basic Formulas

• Let $f$ and $g$ be differentiable functions
  • $(c f)^{\prime}=c f^{\prime}$
  • $(f \pm g)^{\prime}=f^{\prime} \pm g^{\prime}$
  • $(f g)^{\prime}=f^{\prime} g+f g^{\prime}$
  • $\left(\frac{f}{g}\right)^{\prime}=\frac{f^{\prime} g-f g^{\prime}}{g^{2}}$
  • $\frac{d}{d x}(c)=0$
  • $\frac{d}{d x}\left(x^{n}\right)=n x^{n-1}$
  • $\frac{d}{d x}(f(c x))=c f^{\prime}(c x)$
  • $\frac{d}{d x}\left(e^{x}\right)=e^{x}$
  • $\frac{d}{d x}(\ln (x))=\frac{1}{x}, x>0$
  • $\frac{d}{d x}(\sin (x))=\cos (x)$
  • $\frac{d}{d x}(\cos (x))=-\sin (x)$
  • $\frac{d}{d x}(\tan (x))=\sec ^{2}(x)$
  • $\frac{d}{d x}(\sec (x))=\sec (x) \tan (x)$
  • $\frac{d}{d x}(\csc (x))=-\csc (x) \cot (x)$
  • $\frac{d}{d x}(\cot (x))=-\csc ^{2}(x)$
  • $\frac{d}{d x}\left(\sin ^{-1}(x)\right)=\frac{1}{\sqrt{1-x^{2}}}$
  • $\frac{d}{d x}\left(\cos ^{-1}(x)\right)=-\frac{1}{\sqrt{1-x^{2}}}$
  • $\frac{d}{d x}\left(\tan ^{-1}(x)\right)=\frac{1}{1+x^{2}}$

Chain Rule

• $\frac{d}{d x}(f(g(x)))=f^{\prime}(g(x)) g^{\prime}(x)$
  • Example: $\frac{d}{d x}\left(\cos \left(x^{2}\right)\right)=-\sin \left(x^{2}\right) \cdot 2 x$

Integrals

Basic Formulas

• Let $f$ and $g$ be continuous functions:
  • $\int c f(x) d x=c \int f(x) d x$
  • $\int f(x) \pm g(x) d x=\int f(x) d x \pm \int g(x) d x$
  • $\int_{a}^{b} f(x) d x=\left.F(x)\right|_{a} ^{b}=F(b)-F(a)$ where $F^{\prime}(x)=f(x)$
  • $\int_{a}^{a} f(x) d x=0$
  • $\int_{a}^{b} f(x) d x=-\int_{b}^{a} f(x) d x$
  • $\int_{a}^{b} f(x) d x=\int_{a}^{c} f(x) d x+\int_{c}^{b} f(x) d x$

Integration Rules

- $\int d x=x+c$
- $\int x^{n} d x=\frac{x^{n+1}}{n+1}+c, n \neq-1$
- $\int e^{x} d x=e^{x}+c$
- $\int \frac{1}{x} d x=\ln |x|+c$
- $\int \cos (x) d x=\sin (x)+c$
- $\int \sin (x) d x=-\cos (x)+c$
- $\int \sec ^{2}(x) d x=\tan (x)+c$
- $\int \sec (x) \tan (x) d x=\sec (x)+c$
- $\int \csc ^{2}(x) d x=-\cot (x)+c$
- $\int \csc (x) \cot (x) d x=-\csc (x)+c$
- $\int \frac{1}{x^{2}+a^{2}} d x=\frac{1}{a} \tan ^{-1}\left(\frac{x}{a}\right)+c$
- $\int \frac{1}{\sqrt{a^{2}-x^{2}}} d x=\sin ^{-1}\left(\frac{x}{a}\right)+c$

Integration by Parts

• $\int u d v=u v-\int v d u$

Products of Trig Functions

• $\int \sin ^{n}(x) d x=-\frac{1}{n} \sin ^{n-1}(x) \cos (x)+\frac{n-1}{n} \int \sin ^{n-2}(x) d x$
• $\int \cos ^{n}(x) d x=\frac{1}{n} \cos ^{n-1}(x) \sin (x)+\frac{n-1}{n} \int \cos ^{n-2}(x) d x$
• $\int \tan ^{n}(x) d x=\frac{1}{n-1} \tan ^{n-1}(x)-\int \tan ^{n-2}(x) d x$