Complex Number Operations

Definition of Complex Numbers

• A complex number combines a real number and an imaginary number.
• It is in the form z = a + bi, where a is the real part, b is the imaginary part, and i = √-1.
• Example: For z = 3 + 2i, the real part is 3, and the imaginary part is 2.

Operations with Complex Numbers

Addition

• To add complex numbers, add real parts, and imaginary parts separately: (a + bi) + (c + di) = (a + c) + (b + d)i
• Example: (3 + 2i) + (1 - i) = 4 + i

Subtraction

• To subtract complex numbers, subtract real parts and imaginary parts separately: (a + bi) - (c + di) = (a - c) + (b - d)i
• Example: (3 + 2i) - (1 - i) = 2 + 3i

Multiplication

• To multiply complex numbers, use the distributive property and i² = -1: (a + bi)(c + di) = (ac - bd) + (ad + bc)i
• Example: (3 + 2i)(1 - i) = 5 - i

Division

• To divide, multiply the numerator and denominator by the conjugate of the denominator: (a + bi)/(c + di) = ((ac + bd) + (bc - ad)i) / (c² + d²)
• Example: (3 + 2i) / (1 - i) = (1/2) + (5/2)i

Complex Conjugate

• The complex conjugate of z = a + bi is denoted as z̄ and defined as z̄ = a - bi.
• z + z̄ = 2Re(z) = 2a, twice the real part of z.
• z - z̄ = 2iIm(z) = 2bi, twice the imaginary part of z.
• z * z̄ = a² + b², which is a real number.
• Example: If z = 3 + 2i, then z̄ = 3 - 2i.

Modulus of a Complex Number

• The modulus |z| of z = a + bi is the distance from the origin to the point (a, b) in the complex plane: |z| = √(a² + b²)
• |z| ≥ 0 for all complex numbers z.
• |z| = |z̄|.
• |z₁ * z₂| = |z₁| * |z₂|.
• |z₁ / z₂| = |z₁| / |z₂|.
• Example: If z = 3 + 4i, then |z| = 5.

Argument of a Complex Number

• The argument arg(z) of z = a + bi is the angle between the positive real axis and the line connecting the origin to (a, b).
• θ = arg(z) = arctan(b/a).
• The principal argument Arg(z) lies in the interval (-π, π].
• Example: For z = 1 + i, θ = arctan(1/1) = π/4.

Polar Form of Complex Numbers

• A complex number z = a + bi can be represented as z = r(cosθ + isinθ), where r = |z| and θ = arg(z).
• Using Euler's formula, z = re^(iθ).
• Example: For z = 1 + i, r = √2 and θ = π/4, so z = √2(cos(π/4) + isin(π/4)) = √2e^(iπ/4).

De Moivre's Theorem

• For z = r(cosθ + isinθ) and any integer n, z^n = r^n(cos(nθ) + isin(nθ)).
• Or, using the exponential form: (re^(iθ))^n = r^n e^(inθ).
• Example: To find (1 + i)^4:
  • 1 + i = √2e^(iπ/4).
  • (1 + i)^4 = (√2)^4 e^(i(π/4)*4) = 4e^(iπ) = -4.

Linear Classification lab

Introduction

• Implement and train a linear classifier for image classification.
• Implement a Softmax classifier.
• Train and validate implementation on a dataset.
• Understand the effect of hyperparameters.

Datasets

• Use the Cifar-10 dataset.
  • 60,000 32x32 color images in 10 classes.
  • 6,000 images per class.
  • 50,000 training images and 10,000 test images.

Softmax Classifier

Tab and Gradient Function

• Implement the vectorized version of the Softmax loss function and its gradient.
• Complete the implementation of the functions softmax_loss_naive and softmax_loss_vectorized in the file softmax.py.
• Avoid using for-loops.
• Check implementation with numerical gradient.

Test Implementation

• To test implementaion:

python test.py softmax

Training the Softmax Classifier

Training Function

• Implement the train function in the file linear_classifier.py.

Prediction Function

• Implement the predict function in the file linear_classifier.py.

Test Implementation

• To test implementaion:

python test.py linear_classifier

Experiments

Load Cifar-10 Dataset

import numpy as np
from utils import get_CIFAR10_data
from linear_classifier import Softmax

## Load CIFAR-10 dataset
cifar10_dir = 'datasets/cifar-10-batches-py'
X_train, y_train, X_test, y_test = get_CIFAR10_data(cifar10_dir)

## Print the dimensions of the training and test data
print('Training data form: ', X_train.shape)
print('Training labels form: ', y_train.shape)
print('Test data form: ', X_test.shape)
print('Test labels form: ', y_test.shape)

Train the Softmax Classifier

## Create a Softmax classifier
softmax = Softmax()

## Train the Softmax classifier on the training data
loss_history = softmax.train(X_train, y_train, learning_rate=1e-7, reg=2.5e4, num_iters=1500)

## Predict labels for training data and calculate training accuracy
y_train_pred = softmax.predict(X_train)
training_accuracy = np.mean(y_train == y_train_pred)
print('Training accuracy: ', training_accuracy)

## Predict labels for test data and calculate test accuracy
y_test_pred = softmax.predict(X_test)
test_accuracy = np.mean(y_test == y_test_pred)
print('Test accuracy: ', test_accuracy)

Hyperparameter Tuning

• Use cross-validation to tune the hyperparameters (learning rate and regularization strength).
• For each combination of hyperparameters, train the Softmax classifier on the training data and evaluate its performance on the validation data.
• Select the combination of hyperparameters that gives the best performance on the validation data.
  • Use np.random.choice to generate random subsets of the training data for validation.
  • Save the results of the cross-validation in a dictionary.
  • Plot the validation accuracy as a function of the hyperparameters.

Assignment

• File submission:
  • Completed softmax.py and linear_classifier.py files.
• Report:
  • A report, describing your experiments and results.
  • A description of your implementation of the Softmax classifier.
  • A description of your training procedure.
  • A discussion of the effect of hyperparameters.
  • A description of your cross-validation results.
  • An analysis of your results.