Intro to Machine Learning

Machine Learning

• Machine learning enables computers to learn without explicit programming.
• Key types are supervised learning and unsupervised learning.

Supervised Learning

• This involves learning from data that has already been labeled.
• Regression predicts continuous values, like house prices.
• Classification predicts discrete values, such as classifying emails as spam or not.

Linear Regression

• Model equation: ( h_\theta(x) = \theta_0 + \theta_1x )
• Cost function quantifies the error: ( J(\theta_0, \theta_1) = \frac{1}{2m}\sum_{i=1}^{m}(h_\theta(x^{(i)}) - y^{(i)})^2 )
• The goal is to minimize the cost function.

Gradient Descent

• An algorithm used to minimize the cost function.
• Update rule: ( \theta_j := \theta_j - \alpha \frac{\partial}{\partial \theta_j}J(\theta_0, \theta_1) ) for j = 0 and j = 1
• ( \alpha ) represents the learning rate.

Unsupervised Learning

• Learning from data that has not been labeled.
• Clustering groups similar data points together which is useful for customer segmentation.
• Dimensionality reduction reduces the number of variables, also called features, in a dataset.

K-Means Clustering

• Choose K, the number of clusters, and then randomly initialize K cluster centroids.
• The algorthim iterates to assign each data point to the closest centroid.
• Cluster centroids are recomputed based on the means of data points in each cluster.
• K-Means is sensitive to initial centroid positions.

Hypothesis Testing Overview

• Focuses on making statements about a population from sample data.
• Includes parameter estimation and assessing claims about the population.

Statistical Hypothesis Definition

• A statistical hypothesis is an assertion pertaining to one or more populations.

General Procedure for Hypothesis Testing

• State null and alternative hypotheses.
• Identify a test statistic.
• Determine test statistic distribution under the null hypothesis.
• Calculate the observed test statistic value.
• Calculate the p-value.
• Evaluate the p-value to conclude.

Definitions of Null and Alternative Hypotheses

• The null hypothesis, ( H_0 ), is assumed true until proven otherwise.
• The alternative hypothesis, ( H_1 ) or ( H_a ), contradicts the null hypothesis.
• The null hypothesis includes an equality sign: ( =, \le, \ge ).
• The alternative hypothesis never includes an equality sign: ( \ne, >,

Types of Hypothesis Tests

• A one-sided test checks directionality; for example:
  • ( H_0: \mu = \mu_0 )
  • ( H_1: \mu > \mu_0 ) (right-tailed test)
  • or
  • ( H_0: \mu = \mu_0 )
  • ( H_1: \mu < \mu_0 ) (left-tailed test)
• A two-sided test checks if a parameter differs from a specific value. For example:
  • ( H_0: \mu = \mu_0 )
  • ( H_1: \mu \ne \mu_0 )

Remarks on P-Value

• The p-value for one-sided tests is the probability of observing a test statistic as extreme as or more extreme than the observed value in the direction specified.
• The p-value for two-sided tests is double the probability of observing a test statistic as extreme as or more extreme than the observed value.
• One-sided tests are more powerful if the true value aligns with the alternative hypothesis.
• Two-sided tests are appropriate when there's no prior knowledge about the effect's direction.

Tensile Strength

• This represents the maximum stress a material can withstand before breaking.
• Formula: Tension Strength = ( \frac{Force}{Area} )
• Units: pounds per square inch (psi) or Mega Pascals (MPa).

Tensile Stress-Strain Curve

• Ductile Material Characteristics:
  • A-B: Linear (Elastic) Region abiding to Hooke's Law ( (Stress = E \times Strain) ), where E is Young's Modulus.
  • B: Yield Point, which indicates the start of permanent deformation.
  • B-C: Plastic Region, where permanent deformation occurs.
  • C: Ultimate Tensile Strength which is the maximum stress the material can endure.
  • C-D: Necking, reduces cross-sectional area.
  • D: Fracture, the point at which the material breaks.
• Brittle Material Characteristics:
  • A-B: Linear (Elastic) Region that follows Hooke's Law ( (Stress = E \times Strain) ).
  • B: Fracture point at which the material breaks.

Mutual Information Definition

• Quantifies the reduction in uncertainty about one random variable given knowledge of another.
• Symmetric as ( I(X; Y) = I(Y; X) ).
• Non-negative: ( I(X; Y) \geq 0 ).
• Alternative formula: ( I(X; Y) = H(X) + H(Y) - H(X, Y) )

Properties of Mutual Information

• ( I(X; Y) = H(X) - H(X|Y) = H(Y) - H(Y|X) ) shows equality from asymmetry.
• ( I(X; Y) \geq 0 ) holds; it is equal if and only if X and Y are independent.
• ( I(X; X) = H(X) ) meaning the mutual information of a variable with itself is its entropy.
• Conditional Mutual Information: ( I(X; Y|Z) = H(X|Z) - H(X|Y, Z) )

Venn Diagram Representation

• Represents relationships between entropy and mutual information, visually highlighting shared information and conditional dependencies.
• ( I(X; Y) = H(X) - H(X|Y) = H(Y) - H(Y|X) )
• ( H(X, Y) = H(X) + H(Y) - I(X; Y) )

Channel Capacity

• Defined as the maximum of ( I(X; Y) ) over all possible input distributions ( p(x) ).
• ( C = \max_{p(x)} I(X; Y) )
• Represents the maximum reliable information transmission rate over a channel.

Algorithmic Game Theory - Overview

• It's the study of conflicts and cooperation through mathematical models, from the perspective of intelligent and rational decision-makers.
• It integrates classical game theory with algorithm design to solve challenges that traditional game theory overlooks.
• It is needed because normal Game theory disregards computational complexity.
• Computer science focuses on designing and assessing effective algorithms.

The Value of Algorithmic Game Theory

• Traditional game theory does not consider computational complexity.
• Computer science is concerned with the design and analysis of efficient algorithms.
• Algorithmic game theory helps design good and workable auctions.

Selfish Routing

• Model: A network of ( n ) nodes and ( m ) edges.
• Each edge ( e ) has a cost function ( l_e(x) ), where ( x ) is traffic fraction.
• Numerous users minimize their costs traveling from source ( s ) to destination ( t ).
• Each user chooses the path that minimizes their cost.

Wardrop Equilibrium State

• Flow state where no user benefits from unilateral path change.
• All used paths between origin and destination have equal cost.

Price of Anarchy

• Quantifies system efficiency loss due to selfish behavior.
• ( PoA = \frac{\text{Cost of worst-case Nash equilibrium}}{\text{Cost of optimal solution}} )

Braess's Paradox

• Counterintuitively, adding a road can increase travel time for all users.

Linear Cost Functions

• When cost functions are linear ( l_e(x) = a_e x + b_e ), the price of anarchy is at most ( \frac{4}{3} ).

Review of Basic Algebra

• The Cartesian coordinate system uses two perpendicular number lines, is a horizontal axis (x-axis) and a vertical axis (y-axis).
• Each point in the plane corresponds to an ordered pair (x, y) of real numbers, x is the x-coordinate (abscissa) and y is the y-coordinate (ordinate) of the point.

The Distance Formula

• It can be used to determine the distance between the points ( (x_1, y_1) ) and ( (x_2, y_2) ):
• ( d = \sqrt{(x_2 - x_1)^2 + (y_2 - y_1)^2} )

General Forms of Equations of Lines

• Slope: ( m = \frac{y_2 - y_1}{x_2 - x_1} )
• Point-Slope Form: ( y - y_1 = m(x - x_1) )
• Slope-Intercept Form: ( y = mx + b )
• General Form: ( Ax + By + C = 0 )
• Vertical Line: ( x = a )
• Horizontal Line: ( y = b )

Chemical Kinetics - Reaction Rate

• This is defined as the speed at which reactants turn into products.
• ( Rate = -\frac{\Delta[Reactant]}{\Delta t} = \frac{\Delta [Product]}{\Delta t} )
• For reaction ( aA + bB \rightarrow cC + dD ):
• ( Rate = -\frac{1}{a}\frac{\Delta [A]}{\Delta t} = -\frac{1}{b}\frac{\Delta [B]}{\Delta t} = \frac{1}{c}\frac{\Delta [C]}{\Delta t} = \frac{1}{d}\frac{\Delta [D]}{\Delta t} )

Rate Law

• Expresses rate in relation to reactant concentrations: ( Rate = k[A]^m[B]^n )
• It includes:
• ( k ), which is the rate constant
• ( m ) and ( n ), which denote reaction orders

Understanding Integrated Rate Laws

• These relate reactant concentration to time.
• For first-order reactions: ( \ln[A]_t - \ln[A]0 = -kt ), with ( t{1/2} = \frac{0.693}{k} ).
• For second-order reactions: ( \frac{1}{[A]_t} - \frac{1}{[A]0} = kt ), with ( t{1/2} = \frac{1}{k[A]_0} ).
• In zero-order reactions, the rate is independent of concentration: ( [A]_t - [A]0 = -kt ), with ( t{1/2} = \frac{[A]_0}{2k} ).

Key Points of Collision Theory

• Molecules must collide with enough energy called activation energy ( E_a ).
• Molecules must collide with the correct orientation.

Arrhenius Equation

• This relates the rate constant to temperature: ( k = Ae^{-E_a/RT} )
• Also shows: ( \ln \frac{k_2}{k_1} = -\frac{E_a}{R}(\frac{1}{T_2} - \frac{1}{T_1}) )

Reaction Mechanisms

• These Mechanisms describe reactions through elementary steps and intermediates.
• Rate-determining step is the slowest step of a Reaction Mechanisms.

Effects of Catalysis

• Catalysis increases reaction rates without being consumed.
• Catalysts work by providing a pathway with a lower activation energy.
• Catalysis can either be homogeneous (same phase) or heterogeneous (different phase).

Causal Inference - Core Concept

• Identifying causes of effects is essential across multiple sectors for decision-making.

The Fundamental Problem

• It involves never being able to simultaneously observe a causal effect in both treated and untreated states.

Why Causal?

• Needed to understand effects rather than just predictions, especially in experimental settings.

Common Confusion

• There is a difference between causal inference and correlation.
• Correlation shows associations, while causal inference identifies causes.
• Highlighting the error of assuming causation from correlation.

Key Summary

• Causal inference determines cause and effect, crucial in decision-making.
• The core challenge is the inability to observe causal effects in treated and untreated conditions simultaneously.
• It requires justifiable assumptions and differs fundamentally from predictive modeling and correlational analysis.

Analisi Matematica 1

• Instructions: Duration of the test: 2 hours and 30 minutes. During the test, the use of books, notebooks, calculators and telephones is not allowed.
• Exercise 1 [6 points] Calculate the following limit: ( \lim_{n\rightarrow +\infty} \frac{e^{\frac{1}{n}}-1}{\sin(\frac{1}{n})\cos(n)+\frac{1}{n^2}} )
• Exercise 2 [6 points] Study the following function: ( f(x) = \frac{\ln(x)}{2\ln(x)-1} )
• Exercise 3 [5 points] Calculate the following integral: ( \int_{0}^{1} \frac{x}{\sqrt{4-x^4}}dx )
• Exercise 4 [6 points] Solve the following equation in the complex field: ( z^3 = 8i )
• Exercise 5 [6 points] Study the convergence of the following series: ( \sum_{n=1}^{+\infty} \frac{n^2+n}{n!+2^n} )
• Exercise 6 [4 points] Let ( f \in C^1(\mathbb{R}) ) such that ( f(0)=0 ) and ( 2 \leq f'(x) \leq 3 ) for every ( x \in \mathbb{R} ). Prove that ( 2x \leq f(x) \leq 3x ) for every ( x \geq 0 ).

Memory Management Objectives

• Aim to understand and execute memory allocation methods.
• Learn the design and implementation process for managing memory.

Understanding Dynamic Allocation

• Memory allocation takes place during program execution.
• Done via functions (e.g., malloc() and free() in C.)

Dealing with Memory Fragmentation

• Fragmentation occurs when memory is allocated or deallocated over time
• It leads to small blocks of free memory that can scattered throughout the heap.

Key Algorithms for Memory Allocation

• First-Fit: Selects the first suitable free block.
• Best-Fit: Chooses the smallest available block that fits the request.
• Worst-Fit: Allocates memory from the largest free block.

Implementation

• Key data structures necessary:
  • Structure for free memory blocks, tracking size and pointers to previous and next blocks.
  • Free list to monitor available memory chunks, organized by address.

Memory Management Functions

• They must be implemented as follows:
  • void *my_malloc(size_t size): Allocates a memory block.
  • void my_free(void *ptr): Deallocates a memory block.
  • void print_free_list(): Outputs the contents of the free list.

Focus Area

• The first-fit algorithm is the approach for allocating memory when implementing the memory allocation algorithm.