Vectors and Matrices in Machine Learning

Study Notes

Vectors and matrices are essential in machine learning for data representation, mathematical operations, feature transformation, model parameter storage, and linear algebra applications.
They enable efficient handling of complex and high-dimensional data structures.
Vectors and matrices facilitate mathematical operations like dot products and matrix multiplications crucial for algorithms like linear regression and neural networks.
They are used to transform features, enabling dimensionality reduction techniques like Principal Component Analysis (PCA) and feature scaling.
Vectors and matrices store model parameters and weights, facilitating efficient computation and optimization in training processes such as gradient descent and backpropagation.

Vectors are mathematical objects with length and direction.
They can be viewed as elements of a vector space, which is a collection of objects closed under addition and scalar multiplication.
They are represented as 1-dimensional arrays of numbers called components, displayed in column or row form.
Geometrically, vectors represent coordinates in an n-dimensional space, where n is the number of dimensions.
A vector can also be represented as an arrow with an origin, direction, and magnitude (length).

Vector Addition: Two vectors, u and v, of the same dimension can be added by adding their corresponding components:
- u = [u1, u2, ..., un], v = [v1, v2, ..., vn]
- u + v = [u1 + v1, u2 + v2, ..., un + vn]
Example: In R3, u = [1, 1, -1] and v = [2, 3, 1]
- u + v = [3, 4, 0]
Vector Subtraction: Two vectors, u and v, of the same dimension can be subtracted by subtracting their corresponding components:
- u = [u1, u2, ..., un], v = [v1, v2, ..., vn]
- u - v = [u1 - v1, u2 - v2, ..., un - vn]
Example: In R3, u = [1, 1, -1] and v = [2, 3, 1]
- u - v = [-1, -2, -2]

Dot product: The dot product of two vectors, u and v, of the same dimension is calculated by multiplying their corresponding components and summing the results:
- u = [u1, u2, ..., un], v = [v1, v2, ..., vn]
- u · v = u1v1 + u2v2 + ... + unvn
Example: In R3, u = [1, 1, -1] and v = [2, 3, 1]
- u · v = 2 + 3 - 1 = 4

Length/Magnitude: The length or magnitude of a vector v = [v1, v2, ..., vn] is calculated as the square root of the sum of the squares of its components:
- |v| = √(v · v) = √(v1² + v2² + ... + vn²)
Example: v = [2, 3, 1], |v| = √(2² + 3² + 1²) = √(4 + 9 + 1) = √14

A type of machine learning where the model is trained on labeled data.
The model learns to map inputs to the correct outputs.
Key Techniques:
- Regression (e.g., linear regression, polynomial regression) for predicting continuous values.
- Classification (e.g., logistic regression, support vector machines, decision trees) for categorizing data into discrete classes.
Applications:
- Spam detection in emails
- Fraud detection in finance
- Image and speech recognition

A machine learning approach using unlabeled data.
The model identifies patterns and structures within the data without prior labels.
Key Techniques:
- Clustering (e.g., k-means, hierarchical clustering) for grouping similar data points together.
- Dimensionality reduction (e.g., PCA, t-SNE) for simplifying complex data while preserving important information.
Applications:
- Customer segmentation
- Anomaly detection
- Market basket analysis

A type of machine learning where an agent learns to make decisions by taking actions in an environment to maximize some notion of cumulative reward.
The agent interacts with the environment, receives feedback (rewards or penalties), and adjusts its actions accordingly.
Key Concepts:
- Agent: The learner or decision maker
- Environment: The domain in which the agent operates
- Actions: Choices made by the agent
- Rewards: Feedback from the environment based on agent's actions
Applications:
- Robotics (e.g., autonomous navigation)
- Game AI (e.g., playing chess or Go)
- Personalized recommendations in dynamic environments