Understanding Algorithmic Complexity

Algorithmic Complexity

• Efficiency of an algorithm
• Measures time (time complexity) or space (space complexity) required to solve a problem.
• Expressed as a function of input size ($n$).
• Focuses on how quickly time/space grows as the input size increases (Big O notation).
• Helps in selecting algorithms, assessing problem-solving feasibility at scale, and comparing solutions.
• Determined by counting operations (time) or memory used (space).
• Simplified by ignoring constants and lower order terms
• Expressed using Big O notation.

Common Complexities

• Constant: $O(1)$
  • When $n = 10$, value is 1
  • When $n = 100$, value is 1
• Logarithmic: $O(log(n))$
  • When $n = 10$, value is 3
  • When $n = 100$, value is 7
• Linear: $O(n)$
  • When $n = 10$, value is 10
  • When $n = 100$, value is 100
• Log-Linear: $O(nlog(n))$
  • When $n = 10$, value is 30
  • When $n = 100$, value is 700
• Quadratic: $O(n^2)$
  • When $n = 10$, value is 100
  • When $n = 100$, value is 10,000
• Cubic: $O(n^3)$
  • When $n = 10$, value is 1,000
  • When $n = 100$, value is 1,000,000
• Exponential: $O(2^n)$
  • When $n = 10$, value is 1,024
  • When $n = 100$, value is Massive

Systems of Linear Equations

• A linear equation has the form $a_1x_1 + a_2x_2 +... + a_nx_n = b$, where $a_1, a_2,..., a_n$ and $b$ are constants, and $x_1, x_2,..., x_n$ are variables.
• It is a set of one or more linear equations involving the same variables.
• A solution is a set of values for the variables that satisfy all equations.
• The set of all solutions is called the solution set.
• A system is consistent if it has at least one solution; otherwise, it is inconsistent.
• Equivalent systems have the same solution set.

Forms to represent a system of linear equations

• Equation form
• Matrix form
  • $Ax = b$, where $A$ is the coefficient matrix, $x$ the variables vector, and $b$ the constants vector.
• Augmented matrix
  • $[A|b]$

Elementary Row Operations

• Interchange two rows.
• Multiply a row by a nonzero constant.
• Add a multiple of one row to another.

Row Echelon Form and Reduced Row Echelon Form

• Row Echelon Form is achieved if:
  • All nonzero rows are above any rows of all zeros.
  • Each leading entry is in a column to the right of the leading entry of the previous row.
  • All entries in a column below a leading entry are zeros.
• Reduced Row Echelon Form is achieved if:
  • The matrix is in row echelon form.
  • Each leading entry is 1.
  • Each leading entry is the only nonzero entry in its column.

Uniqueness of Reduced Row Echelon Form

• Each matrix is row equivalent to one and only one matrix in reduced row echelon form.

Reduction Algorithm

• Given an $m \times n$ matrix $A$, to transform into reduced row echelon form:
  1. Start with the leftmost column and work right.
  2. Select a nonzero entry as a pivot. Interchange rows if needed.
  3. Use row operations to make zeros below the pivot position.
  4. Cover the pivot row and repeat with remaining rows until no more rows to cover.
  5. Starting with the rightmost pivot column, create zeros above the pivot and make each pivot 1.

Basic and Free Variables

• Variables corresponding to pivot columns are basic variables.
• Variables corresponding to non-pivot columns are free variables.

Existence and Uniqueness Theorem

• A linear system is consistent if and only if the rightmost column of the augmented matrix is not a pivot column.
• If a linear system is consistent, the solution set contains:
  • A unique solution if there are no free variables.
  • Infinitely many solutions if there is at least one free variable.

Vector Functions of a Real Variable

Definition

• A vector function is defined as $\overrightarrow{r}: I \subseteq \mathbb{R} \longrightarrow \mathbb{R}^n$
• Each real number $t$ in the interval $I$ corresponds to a vector $\overrightarrow{r}(t) = (f_1(t), f_2(t),..., f_n(t))$.
• Component functions are defined as, $f_i: I \subseteq \mathbb{R} \longrightarrow \mathbb{R}$ with $i = 1, 2,..., n$
• Example: $\overrightarrow{r}(t) = (t^2 + 1, \cos(t), e^t)$ with $t \in \mathbb{R}$

Limit

• The limit of a vector function is found by evaluating the limit of each component function.
• If $r(t) = (f_1(t), f_2(t),..., f_n(t))$, then: $\lim_{t \to a} \overrightarrow{r}(t) = (\lim_{t \to a} f_1(t), \lim_{t \to a} f_2(t),..., \lim_{t \to a} f_n(t))$
• This is true, as long as the limits of all the component functions exist.

Continuity

• A vector function $\overrightarrow{r}$ is continuous at $a$ only if each of its component functions are continuous at $a$
• Defined as, $\lim_{t \to a} \overrightarrow{r}(t) = \overrightarrow{r}(a)$

Derivative

• The derivative of a vector function is found by derivating each of its component functions.
• If $\overrightarrow{r}(t) = (f_1(t), f_2(t),..., f_n(t))$, then: $\overrightarrow{r}'(t) = (f_1'(t), f_2'(t),..., f_n'(t))$
• The derivatives of all component functions must exist
• The notation is, $\overrightarrow{r}'(t) = \frac{d\overrightarrow{r}}{dt}$

Geometric Interpretation

• $\overrightarrow{r}(t)$ is the position vector of a particle at time $t$
• $\overrightarrow{r}'(t)$ is the velocity vector of the particle at time $t$, indicating the direction of movement.
• $\overrightarrow{r}'(t)$ is also a tangent vector to the curve described by $\overrightarrow{r}(t)$ at the point $\overrightarrow{r}(t)$.

Differentiation Rules

• Let $\overrightarrow{r}(t)$ and $\overrightarrow{s}(t)$ be differentiable vector functions, and let $f(t)$ be a differentiable scalar function. Then:
  • $[\overrightarrow{r}(t) + \overrightarrow{s}(t)]' = \overrightarrow{r}'(t) + \overrightarrow{s}'(t)$
  • $[c\overrightarrow{r}(t)]' = c\overrightarrow{r}'(t)$
  • $[f(t)\overrightarrow{r}(t)]' = f'(t)\overrightarrow{r}(t) + f(t)\overrightarrow{r}'(t)$
  • $[\overrightarrow{r}(t) \cdot \overrightarrow{s}(t)]' = \overrightarrow{r}'(t) \cdot \overrightarrow{s}(t) + \overrightarrow{r}(t) \cdot \overrightarrow{s}'(t)$
  • $[\overrightarrow{r}(t) \times \overrightarrow{s}(t)]' = \overrightarrow{r}'(t) \times \overrightarrow{s}(t) + \overrightarrow{r}(t) \times \overrightarrow{s}'(t)$
  • $[\overrightarrow{r}(f(t))]' = \overrightarrow{r}'(f(t))f'(t)$

Integrals

• The integral of a vector function $\overrightarrow{r}(t) = (f_1(t), f_2(t),..., f_n(t))$ is defined by integrating each of its component functions.
• $\int_a^b \overrightarrow{r}(t) dt = (\int_a^b f_1(t) dt, \int_a^b f_2(t) dt,..., \int_a^b f_n(t) dt)$

Fluids

• Defined as liquids or gases
• They conform to the boundaries of a container

12.1 Pressure of a Fluid

• Pressure = Force/Area
  • $P = \frac{F}{A}$
  • Pressure is measured in Pascals (Pa)
    • $N/m^2 = Pascal = Pa$
    • $1 atm = 1.013 \times 10^5 Pa = 760 mm-Hg$
• At a depth h below the surface of a liquid:
  • $P = P_0 + \rho g h$
  • Where:
    • $P_0$ is the pressure at the surface
    • $\rho$ is the density of the liquid

Pascal's Principle

• A change in pressure applied to an enclosed fluid is transmitted undiminished to every point in the fluid and to the walls of the container
• Formula for a Hydraulic Lift:
  • $P_1 = P_2$
  • $\frac{F_1}{A_1} = \frac{F_2}{A_2}$
    • Where a small force $F_1$ applied to a small area $A_1$ can create a large force $F_2$ on a large area $A_2$.
  • $F_2 = F_1 \frac{A_2}{A_1}$

Buoyant Force

• The buoyant force is the upward force exerted by a fluid on an object.
• Equation: $B = W_{displaced fluid} = m_{fluid}g = \rho_{fluid} V_{object} g$
• Archimedes' Principle: The buoyant force on an object is equal to the weight of the fluid displaced by the object

Floating

• An object floats when the buoyant force is equal to the weight of the object.
• $B = W$
• $\rho_{fluid} V_{submerged} g = \rho_{object} V_{object} g$
• $\frac{V_{submerged}}{V_{object}} = \frac{\rho_{object}}{\rho_{fluid}}$

Fluid Dynamics

• Fluid assumed to have these properties:
  1. Incompressible (density is constant).
  2. Non-viscous (no internal friction).
  3. Steady flow (velocity is constant in time).
  4. Irrotational flow (no turbulence).

Flow Rate

• Defined as the volume of fluid passing a point per unit time:
  • $Q = \frac{Volume}{time} = \frac{V}{t}$
  • Units: $m^3/s$
  • Also: $Q = vA$
    • Where:
      • $v$ is the average velocity of the fluid
      • $A$ is the cross-sectional area of the pipe

Equation of Continuity

• The flow rate is constant.
• Formula: $A_1v_1 = A_2v_2$

Bernoulli's Equation

• Conservation of energy.
• $P_1 + \frac{1}{2} \rho v_1^2 + \rho g y_1 = P_2 + \frac{1}{2} \rho v_2^2 + \rho g y_2$
  • $P$ is the pressure
  • $\rho$ is the density
  • $v$ is the velocity
  • $y$ is the height

Chapter 4: Results of Text Classification

Data Size

• Original data size is 10,000.

Experimental Setup

• Epochs: 10.
• Batch size: 32.
• Optimizer: Adam.
• LR: 1e-3.
• $\beta_1$: 0.9.
• $\beta_2$: 0.999.
• $\epsilon$: $10^{-8}$.
• Weight Decay: 0.
• Clip Norm: 1.0
• GPUs: NVIDIA RTX 3090

Text Classification Results

• Measured by Accuracy, Precision, Recall, and F1-score. -BERT - Accuracy: 0.930 - Precision: 0.931 - Recall: 0.930 - F1-score: 0.930
  • RoBERTa
    • Accuracy: 0.935
    • Precision: 0.936
    • Recall: 0.935
    • F1-score: 0.935