Inductively Coupled Circuits and Mutual Inductance

• The study notes below discuss concepts from Inductively Coupled Circuits and Machine Learning System Design

Inductively Coupled Circuits

• Mututal inductance occurs in closely wound coils when the current in one coil induces a voltage in the other
• Consider two coils wound near each other
• Current $i_1$ in coil 1 produces flux $\phi_1 = \phi_{11} + \phi_{12}$, where $\phi_{11}$ links only coil 1 and $\phi_{12}$ links both coils.
• Similarly, current $i_2$ in coil 2 produces flux $\phi_2 = \phi_{22} + \phi_{21}$, where $\phi_{22}$ links only coil 2 and $\phi_{21}$ links both coils.

Induced Voltage

• Voltage induced in coil 1 due to current $i_2$ in coil 2: $v_1 = M_{12} \frac{di_2}{dt}$
• Voltage induced in coil 2 due to current $i_1$ in coil 1: $v_2 = M_{21} \frac{di_1}{dt}$
• Usually, $M_{12} = M_{21} = M$, where $M$ is the mutual inductance in henries (H). Thus, $v_1 = M \frac{di_2}{dt}$ and $v_2 = M \frac{di_1}{dt}$.

Mutual Inductance Definition

• Mutual inductance $M$ can be defined via two different perspectives
• It is the ratio of induced voltage in one coil to the rate of current change in the other, measured in henries (H).
• Alternatively, it is the ratio of flux linkage in one coil to the current in the other.
• Mutual inductance can be calculated $M = N_2 \frac{d \phi_{21}}{di_1} = N_1 \frac{d \phi_{12}}{di_2}$

Coupling Coefficient

• Coupling coefficient, $k$, measures the amount of flux linking both coils: $k = \frac{\phi_{12}}{\phi_1} = \frac{\phi_{21}}{\phi_2}$, where $0 \leq k \leq 1$.
• When $k = 1$, it is a scenario of perfect or unit coupling. But, if $0 < k < 1$, there is loose coupling
• Mutual inductance $M$ ties to self-inductances $L_1$ and $L_2$ by $M = k \sqrt{L_1 L_2}$, with $L_1$ and $L_2$ being coil inductances.
• Figure 1 graphically depicts two coils with their currents ($i_1$, $i_2$) and turn numbers ($N_1$, $N_2$), along with fluxes ($\phi_1$, $\phi_2$).

Machine Learning System Design

• Splitting data is vital to machine learning
• Data Partitioning:
  • Training Set: Used to learn parameters.
  • Development (Dev) Set: Utilized for parameter tuning, feature selection, and other crucial decisions.
  • Test Set: Used to estimate model performance.

Data Partitioning Guidelines

• Traditional Approach:
  • Training set ≈ 60%
  • Dev set ≈ 20%
  • Test set ≈ 20%
• Modern Deep Learning (for very large datasets):
  • Training set ≈ 98%
  • Dev set ≈ 1%
  • Test set ≈ 1%
• Goal: Enable sufficient data for effective learning, performance assessment, and minimize tuning

Bias and Variance

• Critical to understanding an appropriate Machine Learning approach
• Figure 1 describes the balance between the High/Low variances, and the High/Low biases

Understanding Examples

• Consider an ideal Human-Level Error rate of ~0%, and the following cases
  • Case 1 yields Training Error of only 1% , and Dev Set Error of 11% exhibits High variance (Overfitting)
  • Model fits training data well but doesn't generalize to new data.
  • Case 2 Training Error results in 15% , and Dev Set Error yields 16% exhibits High bias
  • Model is underperforming on both training and dev sets.
  • Potentially due to simplistic model or insufficient features.
  • Case 3 Training Error is at 15% , and Dev Set Error results in 30% exhibits High bias and high variance
  • Model has both underfitting and overfitting issues.
• Key insight: Human-Level Error provides a benchmark for comparison.

Recipe for Machine Learning

• Addressing High Bias:
  • Increase Network Size: Use a larger neural network with more parameters
  • Extended Training: Lengthen the time spent in Training
  • Architectural Modifications: Use a different Neural Network architecture
• Alleviating High Variance:
  • More Extensive Data: Increase the amount of training data
  • Regularization Approaches: Implement regularization techniques
  • Change network structure: Use a different Neural Network structure

Regularization

• A powerful technique for generalization
• L2 Regularization
  • For logistic regression: $J(w,b) = \frac{1}{m}\sum_{i=1}^m L(\hat{y}^{(i)}, y^{(i)}) + \frac{\lambda}{2m}||w||_2^2$
  • $||w||2^2 = \sum{j=1}^{n_x}w_j^2 = w^Tw$
• L2 Regularization for Neural Network:
  • $J(W^{},b^{},...,W^{[L]},b^{[L]}) = \frac{1}{m}\sum_{i=1}^m L(\hat{y}^{(i)}, y^{(i)}) + \frac{\lambda}{2m}\sum_{l=1}^L||W^{[l]}||_F^2$
  • $||W^{[l]}||F^2 = \sum{i=1}^{n^{[l-1]}} \sum_{j=1}^{n^{[l]}}(W_{ij}^{[l]})^2$
  • Apply regularization during gradient descent:
  • $W^{[l]} := W^{[l]} - \alpha dW^{[l]}$
  • $dW^{[l]} = (from ; backprop) + \frac{\lambda}{m}W^{[l]}$
  • $W^{[l]} := (1 - \alpha\frac{\lambda}{m})W^{[l]} - \alpha (from ; backprop)$

Dropout Regularization

• Technique to randomly deactivate nodes during each training iteration, and prevents overfitting.
  • Each layer has a specific $keep_prob$(ability of retaining nodes during training).
• Inverted dropout:

• d3 = np.random.rand(a3.shape, a3.shape) < keep_prob
  a3 = np.multiply(a3, d3)
  a3 /= keep_prob
  

Other Regularization Methods

• Data Augmentation: Enlarging the training dataset, via slight transformations
• Early Stopping: Halt training when model, performing poorly , and prevents overfitting.

Setting Up Your Optimization

• Crucial step to boost machine learning performance
• Normalizing Inputs:
  • Zero-center:
  • $\mu = \frac{1}{m}\sum_{i=1}^m x^{(i)}$ ; $x := x - \mu$
  • Normalize variances:
  • $\sigma^2 = \frac{1}{m}\sum_{i=1}^m (x^{(i)})^2$ ; $x /= \sigma^2$
• Goal: Scale inputs and enhance gradient descent performance

Other Considerations

• Your Derivatives could potentially be exponentially large or small: Solve by using a careful initialization
• Careful choice of initialization, using $Var(W_i) = \frac{1}{n}$ is important
• $W^{[l]} = np.random.randn(shape) * np.sqrt(\frac{1}{n^{[l-1]}})$ is useful
• ReLU approach: $\sqrt{\frac{2}{n^{[l-1]}}}$
• $\tanh$: initialization is via $\sqrt{\frac{1}{n^{[l-1]}}}$ (Xavier initialization)

Gradient Checking

• Take $\Theta \in \mathbb{R}^n$
• $J(\Theta) \in \mathbb{R}$
• Check for approximate $\frac{\partial J}{\partial \Theta_i}$, by doing $d\Theta[i] \approx \frac{J(\Theta_1, \Theta_2,..., \Theta_i + \epsilon,...) - J(\Theta_1, \Theta_2,..., \Theta_i - \epsilon,...)}{2\epsilon}$
• Final Check: $\frac{||d\Theta_{approx} - d\Theta||2}{||d\Theta{approx}||_2 + ||d\Theta||_2}$
• Guidelines:
  • If $\epsilon = 10^{-7}$
  • If result is $10^{-7}$, then Great!
  • If result is $10^{-5}$ then OK.
  • If result is $10^{-3}$ or larger, then suspect there might be a bug

gradient Checking Tips

• Gradient checking should only be used for debugging , never during training iterations during modeling
• Check all components.
• Take regularization into account
• Gradient checking works with any value of $\epsilon$

Optimization Algorithms

• Mini-Batch Gradient Descent

  • Crucial to process training set, rather than processing training set at same time

• for t = 1,..., number of minibatches:
   # Forward prop on X^{t}
   # Compute cost J^{t}
   # Backpropagation to compute gradients
   # Update parameters
  

Understanding Mini-Batch Gradient Descent

• Figure 2 is important
• With batch gradient descent, the cost function decreases consistently with each iteration.
• With mini-batch gradient descent, the cost function decreases but with more noise.

Understanding Mini-Batch selection

• Small mini-batch size: result in Faster learning vs Noiser
• Also small size can result in lost vectorization
• Large mini-batch size result in slower learning,
• Guideline:
  • If small training set ($ < 2000$), use batch gradient descent.
  • Mini-batch sizes: 64, 128, 256, 512
  • Be sure it fits in CPU/GPU memory

Other methods

• Exponentially Weighted Averages

  • $V_t = \beta V_{t-1} + (1 - \beta)\Theta_t$

  • $\beta$ is important

  • $\beta = 0.9$, for averaging over last 10 values

  • $\beta = 0.98$, for averaging over last 50 values 
- $\beta = 0.999$, for averaging over last 1000 values

Bias Correction

• This method is extremely useful 
- $V_t = \frac{V_t}{1 - \beta^t}$ 
- Corrected to account for initial values 


Gradient Descent with Momentum

Momentum can exponentially compute weighted averages of gradients.

Here is how:

   On iteration t:
       Compute dW, db on current mini-batch
V_tdW = beta * V_tdW + (1 - beta) * dW
V_tdb = beta * V_tdb + (1 - beta) * db
       W = W - alpha * Vt_dW
b = b - alpha * Vt_db

RMSprop

Here is how the RMSprop formula works

   On iteration t:
       Compute dW, db on current mini-batch
SdW = beta2 * SdW + (1 - beta2) * dW^2
Sdb = beta2 * Sdb + (1 - beta2) * db^2
       W = W - alpha * dW / (sqrt(SdW) + epsilon)
b = b - alpha * db / (sqrt(Sdb) + epsilon)

Adam Optimization

• Adam stands for Adaptive Moment Estimation

This is how it works:

Vt_dW = beta1 * Vt_dW + (1 - beta1) * dW   # Momentum
Vt_db = beta1 * Vt_db + (1 - beta1) * db
SdW = beta2 * SdW + (1 - beta2) * dW^2   # RMSprop
Sdb = beta2 * Sdb + (1 - beta2) * db^2
Vt_dW_corrected = Vt_dW / (1 - beta1^t)  # Bias Correction
Vt_db_corrected = Vt_db / (1 - beta1^t)
SdW_corrected = SdW / (1 - beta2^t)
Sdb_corrected = Sdb / (1 - beta2^t)
   W = W - alpha * Vt_dW_corrected / (sqrt(SdW_corrected) + epsilon)  # Update
b = b - alpha * Vt_db_corrected / (sqrt(Sdb_corrected) + epsilon)

Optimizer Hyperparameter Choice

• In order of importance, we should tune the $\alpha$, then $\beta_1$, then $\beta_2$, and lastly $\epsilon$.
• $\alpha$ tuning helps to find convergence
• $\beta_1$ can be 0.9 for many purposes
• $\beta_2$ can be 0.999 for many purposes
• $\epsilon$ is not important

Hyperparameter Tuning Process

Figure 3

A diagram that illustrates 2 common approaches to Hyperparameter tuning:

• Caring (babysitting) one model
• Training many models in parallel

Tips for hyperparameter tuning process:

• Re-evaluate occasionally:
  • In test set changes.
  • When error costs adjust.
• Try random values , do not use a grid

Using Appropriate Scale

• Number of layers or units in a layer, use a linear scale
• Hyperparameters like using learning rate with $\alpha$, sample log($\alpha$) uniformly

Hyperparameter Tuning Guidelines

• Pandas approach: Use the babysitting one model idea.
• Caviar approach: Train many models in parallel.

HyperparameterTuning Tips

• Have the computation.
• Staff it with people
• Re-evaluate occasionally
• Is the Test set changed
• Do cost changes exist.
• Resources
• Computation
• People
• Remember to occasionally Re-evaluate Test set can changed, thus, always re-evaluate cost

Introduction to Forces

• Forces, a measure of push or pull between interacting objects.
  • Objects react accordingly, and may start acelerating
  • Vector quantities (possesses magnitude and direction).
  • Newtons (N).

Types of Forces

• Forces are applied with or without contact
• Contact Forces:
  • Applied Force: Person or object applies the force to another object. such as by pushing.
  • Frictional Force: Surface exerts force an in opposite direction of the movement. For instance rubbing a book on a table.
  • Tension Force: Ropes, wires, and other objects are pulled tight on opposite ends.
  • Normal Force: Stable object when touching another.
  • Air Resistance Force: Objects force applied as it traverses through Air. Spring Force: Spring compresses or stretched by an object attached

Action at a Distance Forces

• Gravitational Force: Force applied, mass attracts another object towards it. 

• Electrical Force: Interaction between charged objects. Like interaction of electron and proton

• Magnetic Force: Force exerted by magnets or magnetic matter. Like a compass


Free-Body Diagrams

• Free-Body Diagrams used to show direction force relative to other objects Tips
• Represent object as center
• Draw vectors
• Label each force vector
• Figure describes Fg pointing down from the center, and Fn pointing up


Research And Development

• Discusses Basic and Applied Research types
• Basic Research

Quantum Computing

• Still in early stages, but has the potential to revolutionize many fields
• Qubits can exist, and use superposition
• Areas to apply these include and drug discovery, materials science, financial modeling, cryptography, etc.

Challenges

• Stabilizing Qubit
• Accounting for error correction
• Expand Scalability

Artificial Intelligence (AI)

• Simulation of human intelligence in machines, and being used for image recognition, natural language processing, and robotics
• Important to focus on Explainable AI, Robust AI, Ethical AI
• Biotechnology is the use of living organisms or their products to develop new technologies
• Used for to develop new drugs or diagnostics

Applied research

Examples here include Gene therapy, Personalized medicine , Synthetic biology

Advanced Materials

• Are materials that are enhanced
• Strength
• Leightness Conductivity
• Examples include: Graphene, Carbon nanotubes, and Composites

Applications

• These can span Aerospace, Automotive, Construction etc
• Renewable energy
• The energy comes from replenished sources such as wind, hydropower

Renewable goals

• Reduce the reliance on combustion based fuels
• Mitigate climate change
• Solar panels and wind power are the main examples. Batteries are an example

Robotics

• The construction and application of Robots!
• Robots can be used in manufacturing, healthcare and logisics
• Human-robot interaction is an important concept

Development

Development is about product/process developement

• Product Development: It has to meet cutomer needs, and has prototyping involved

Technology Transfer

• Technology transfer is the process of transfer, there are various licensing and joint ventures
• Goals are to commercialize new technologies, start new businesses, etc


Algorithmic Trading

• Algoritmic trading executes orders automatically to account for variables such as price, volume, and timing
• Examples - TrendFollowing, Mean-reversion, Arbitrage, etc

Advantages

• Can perform trades best at possible prices
• Can be automated
• Backtesting avaliable

Disadvantages

• Technical ssues occur
• Need to be continuously optimized with marker cycles

How to get started

• Learn with Python! Test and backtest with historical data Start with little capital, stay updated

PLANCK'S CONSTANT

• Represented by h, found by E=hv
• Equal to $6.62607015 x 10^{-34} Js, very small number
• Important calculation with quantum mechanics

Gravitation Universal introduction:

• Johannes Kepler derived the law of the planetary movement

Newton Gravity law Newton came up with the gravity law f= gmm/r^2 characteristics:attractive,at distance

• vectoral form- F12 = -gmm/r^2 r
• Supremposition Principle: sum all gravitional to find net

Gravity near surface

Acelleration near earth Gmm/r^2 =mg

keplar laws first: ellipse -Second: line connectin area in intervals ke = at / 4

potencial gravity -u =-gmm/r when equal 0 velocidy of escabe v= sqrt 2gm/r

Algorithmic Game Theory

Basic Game Theory

• Game theory models strategic interactions among rational entities applied to Computer Science.

Algorithmic Game Theory:

• Algorithmic Game Theory (AGT) combines algorithm design + game theory by looking to predict behavior. The Internet increased its usefulness since things become more complex

Selfish Routing

• Users want to choose a path that reduces cost given a network. How much of the road gets filled influences the cost.

Price of anarchy

• Measures inefficiency with agent behavior.
• Cost Ratio = : (social cost of the worst case equilibrium /Optimal social cost).

Braess Paradigm:

• The amount of travel a person is using can increase time to travel -Example: Network can be augmented with a zero-cost edge, increasing cost

Convex Optimization:

To minimize of f find X in B such an inequality is satisfied

• Strict means

that f( X bar) is true if equality does not hold consider an epsiolon variable (alpha) using taylor expansion around points on the set to find minimizer

Notes

Key facts

• Nucleus contains chlorophyll, absorbs (light), enables photosynthesis

Additional notes

• Mitochondria participates in respiration and release cell energy
• Ribosomes produce proteins
• Cell membrane controls substances
• Cell wall strengthens and supports plant cells
• Chloroplasts enable photosynthesis
• DNA helps bacteria adapt or envolve
• Plasmids encode/contain antibiotic resistence

Genetic material

• For Eukaryotics, it found in nuclues or chromosomes *
• For prokaryotics, it found in the nucleoid*