Linear Regression Modeling

Lab 2: Modeling with Linear Regression

• Linear regression can be performed on a dataset using the scikit-learn library.
• The performance of a linear regression model can be evaluated using appropriate metrics.
• The results of a linear regression model can be visualized.
• Coefficients of a linear regression model can be interpreted.

Overview of Linear Regression

• Linear regression models the relationship between a dependent variable and one or more independent variables.
• scikit-learn can be used to build and evaluate linear regression models.
• The "Boston Housing" dataset contains information about housing prices in the Boston area, with 506 instances and 13 attributes.
• The target variable is the median value of owner-occupied homes in $1000s.
• Packages required: numpy, pandas, scikit-learn, and matplotlib.

Simple Linear Regression

• Simple linear regression can be performed using only one independent variable.

Steps for Simple Linear Regression

• Load the Boston Housing dataset using pandas.
• Select one independent variable, such as the number of rooms (RM).
• Split the data into training and testing sets using train_test_split from sklearn.model_selection, using a test size of 0.2 and a random_state of 42.
• Train a linear regression model using the training data, using LinearRegression from sklearn.linear_model.
• Make predictions on the testing data.
• Evaluate the model using metrics such as Mean Squared Error (MSE) and R-squared.
  • MSE and R-squared can be calculated using mean_squared_error and r2_score from sklearn.metrics.
• Visualize the predicted values versus the actual values using matplotlib.pyplot.

Multiple Linear Regression

• Multiple linear regression can be performed using all available independent variables.

Steps for Multiple Linear Regression

• Prepare the data by using all independent variables from the dataset.
• Split the data into training and testing sets, using a test size of 0.2 and a random_state of 42.
• Train a linear regression model using the training data.
• Make predictions on the testing data.
• Evaluate the model using MSE and R-squared.
• Interpret the coefficients of the linear regression model.

Model Refinement

• Techniques to improve the performance of the linear regression model can be explored.

Steps for Model Refinement

• Feature scaling can be applied to the data using StandardScaler from sklearn.preprocessing.
• Train the model with scaled data.
• Make predictions.
• Evaluate the model using MSE and R-squared.
• Apply L1 (Lasso) or L2 (Ridge) regularization to the linear regression model, using Ridge from sklearn.linear_model with an appropriate alpha value.

Questions on Linear Regression

• The difference between simple and multiple linear regression
• How to interpret coefficients
• Why feature scaling is important
• What regularization is and how it can improve model performance

Algorithmic Game Theory

• Studies interactions between rational, self-interested players.
• Serves as a mathematical framework for decision-making.
• Has applications in economics, political science, biology, and computer science.

The Prisoner's Dilemma

• Two suspects are arrested and held separately.
• If one confesses and the other doesn't, the confessor goes free, and the other gets 10 years.
• If both confess, they each get 5 years.
• If neither confesses, they each get 1 year.

Payoff Matrix Example

• Suspect A confesses, and suspect B confesses: (5, 5)
• Suspect A confesses, and suspect B doesn't confess: (0, 10)
• Suspect A doesn't confess, and suspect B confesses: (10, 0)
• Suspect A doesn't confess, and suspect B doesn't confess: (1, 1)

Selfish Routing Model

• Involves a network of $n$ players, each controlling a small amount of traffic between a source and destination.
• Every player aims to minimize their travel time.
• The travel time on each edge depends on the total traffic on that edge.

Braess's Paradox

• Adding an edge to a network can increase travel time for all players.
• Initially, 1 unit of traffic goes from A to B with a travel time of 1.5 on each path.
• After adding a fast edge from C to D, all traffic shifts to A-C-D-B, increasing the travel time to 2.

Algorithmic Game Theory vs Traditional Game Theory

• Traditional game theory assumes perfect rationality and unlimited computational resources and focuses on predicting the outcome of games.
• Algorithmic game theory considers computational limitations and focuses on designing computationally efficient games with desirable outcomes.

Key Questions in Algorithmic Game Theory

• How hard is it to compute a Nash equilibrium?
• Can games be designed that are easy to play but still have good outcomes?
• How do players learn to play in a game?
• What is the price of anarchy?

Statics - Equilibrium

• Statics deals with bodies at rest.
• Dynamics deals with bodies in motion.
• A particle's size is irrelevant to its behavior.
• A particle is at Equilibrium when the resultant of all forces acting on it is zero.
• Newton's First Law of Motion: If the resultant force on a particle is zero, the particle will remain at rest or continue to move in a straight line at constant velocity.

Free-Body Diagram

• A sketch showing the particle freed from its surroundings with all the forces acting on it.
• Show all the forces that act on the body, whether known or unknown, and represent the effect of other bodies on the isolated body.
• Cables and cords are assumed to be inextensible and weightless.

Coplanar Force Systems

• Each force can be resolved into x and y components.
• For equilibrium, the resultant force must be zero.
  • $\vec{R} = \sum \vec{F} = 0$
  • $\sum F_x \hat{i} + \sum F_y \hat{j} = 0$
  • $\sum F_x = 0$ and $\sum F_y = 0$

Procedure for Analysis

Free-Body Diagram

• Establish the xy plane.
• Sketch the particle.
• Show all forces acting on the particle.
• Label forces with magnitudes and directions.
• Assume sense of a force with unknown magnitudes but known lines of action.

Equations of Equilibrium

• Represent unknown force components as unknowns.
• Apply equilibrium equations: $\sum F_x = 0$ and $\sum F_y = 0$.
• A negative result indicates the force's sense is opposite to what is shown on the free-body diagram.

Chemical Kinetics - Rates of Chemical Reactions

• A way of defining the rate
  • $aA + bB \rightarrow cC + dD$
  • Rate $= -\frac{1}{a}\frac{d[A]}{dt} = -\frac{1}{b}\frac{d[B]}{dt} = \frac{1}{c}\frac{d[C]}{dt} = \frac{1}{d}\frac{d[D]}{dt}$
    • $[A]$ = concentration of reactant A.
    • $t$ = time.
    • $a, b, c, d$ = stoichiometric coefficients.

Rate Law

• A formula for Rate
  • Rate = $k[A]^m[B]^n$
    • $k$ = rate constant.
    • $m$ = order with respect to A.
    • $n$ = order with respect to B.
    • $m + n$ = overall order of the reaction.

Integrated Rate Laws

Arrhenius Equation

• How to find the rate constant
  • $k = Ae^{-\frac{E_a}{RT}}$
    • $k$ = rate constant
    • $A$ = frequency factor
    • $E_a$ = activation energy
    • $R$ = gas constant (8.314 J/mol·K)
    • $T$ = temperature in Kelvin
  • A two-point version is:
    • $ln(\frac{k_2}{k_1}) = \frac{E_a}{R}(\frac{1}{T_1} - \frac{1}{T_2})$

Reaction Mechanisms

• Reactions may occur in a series of elementary steps that make up a reaction mechanism.
• The slowest step in the mechanism determines the overall rate of the reaction, and is called the rate-determining step.
• Catalysts speed up a reaction without being consumed.
  • They provides an alternate reaction pathway.
  • They lower the activation energy ($E_a$).

Equilibrium

Equilibrium Constant

• How to calculate it
  • $aA + bB \rightleftharpoons cC + dD$
  • $K = \frac{[C]^c[D]^d}{[A]^a[B]^b}$
    • $K$ is the equilibrium constant
    • $[A], [B], [C], [D]$ are equilibrium concentrations

Relationship between K, $\Delta$G, and E

• The relationships are:
  • $\Delta G = -RTlnK$
  • $\Delta G = -nFE$
    • $\Delta G$ = Gibbs free energy change
    • $R$ = gas constant
    • $T$ = temperature in Kelvin
    • $K$ = the equilibrium constant
    • $n$ = number of moles of electrons transferred
    • $F$ = Faraday's constant (96,485 C/mol)
    • $E$ = cell potential

Le Chatelier's Principle

• If a stress is applied to a system in equilibrium, the system will shift to relieve the stress.
• Changes in concentration, pressure, or temperature are different stresses.

Analisi Matematica 1 - Ing. Edile-Architettura

• An exam from February 16th, 2023

Question 1

• Given the function $f(x) = \sqrt{x^2 - 4} - \arcsin(\frac{x}{3})$:
  • Determine the domain of $f$.
  • Calculate the limits at the boundaries of the domain.
  • Determine the intervals of monotonicity of $f$.
  • Sketch a qualitative graph of $f$.

Question 2

• Calculate the following integral: $\int_{0}^{1} \frac{e^x}{e^{2x} + 3e^x + 2} dx$.

Question 3

• Study the convergence of the following series: $\sum_{n=1}^{\infty} \frac{n^2 + 2}{n^3 + n + 1}$.

Question 4

• Solve the following Cauchy Problem:
  • $\begin{cases} y' = \frac{2x}{1 + x^2}y + \sqrt{1 + x^2} \ y(0) = 1 \end{cases}$

Lecture 9: Taylor Series Applications

Recap of Last Lecture

• Last time, classic error bounds for the Taylor approximation were discussed.
  • If $f$ has $n+1$ continuous derivatives on an interval containing $a$ and $x$, then
    • $R_n(x) = \frac{f^{(n+1)}(c)}{(n+1)!}(x-a)^{n+1}$
    • for some $c$ between $a$ and $x$.
  • Taylor's Inequality states that If $|f^{(n+1)}(x)| \le M$ for $|x-a| \le d$, then $|R_n(x)| \le \frac{M}{(n+1)!}|x-a|^{n+1}$ for $|x-a| \le d$.

Applications of Taylor Series

• Taylor series can be used to evaluate the limits of indeterminate forms.
• Taylor series allow approximations to solutions of integrals.

Finding the limit of a Function

• Evaluate $\lim_{x \to 0} \frac{e^x - 1 - x}{x^2}$ from the Taylor expansion of $e^x = 1 + x + \frac{x^2}{2!} + \frac{x^3}{3!} +...$
  • Applying the limit yields the simplified expression for $\lim_{x \to 0} \frac{e^x - 1 - x}{x^2}$ which is equal to $\frac{1}{2}$

Finding the Integral of a Function

• Evaluate $\int e^{-x^2} dx$ from the Taylor expansion of $e^x = 1 + x + \frac{x^2}{2!} + \frac{x^3}{3!} +...$
  • Replacing $x$ with $-x^2$ can simplify the integral:
    • $\int e^{-x^2} dx = \int (1 - x^2 + \frac{x^4}{2!} - \frac{x^6}{3!} +...) dx$
    • $= x - \frac{x^3}{3} + \frac{x^5}{5 \cdot 2!} - \frac{x^7}{7 \cdot 3!} +... + C$