Neural Networks & Loss Functions Quiz

Layers: Composition of Functions

• Neural Networks are composed of layers performing transformations on data.
• Input layer: X = [1, 𝑥1 , 𝑥2 , … , 𝑥𝑝 ]
• Output of each layer becomes input for the next layer.
• Final output is calculated based on all layers.

Generalization – 3: Loss Function

• Loss function is also known as cost function or objective.
• We use loss function to train our model by optimizing (minimizing or maximizing) it.
• The goal is to find the optimal weights in the network.
• Common Loss Functions Include:
  • Logistic Regression: Likelihood
  • Linear Regression: Squared Error

Loss Function: Cross Entropy

• Used for binary classification
• Compares two discrete distributions
• Maximizing likelihood is equivalent to minimizing cross-entropy loss
• Can be generalized to multi-class classification.

Gradient Descent

• Algorithm to find the local minimum of a loss function (e.g., f(W)).
• Notation: 𝑊 (𝑡) represents weights at iteration t.
• Initialization: 𝑊 (𝑡) is initialized for iteration 𝑡 = 0.
• Repeat until convergence: 𝑊 (𝑡+1) = 𝑊 (𝑡) − 𝛼∇𝑓(𝑊 (𝑡) )
  • ∇𝑓(𝑤 (𝑡) ): Gradient pointing towards the direction of fastest increase of the function.
  • −∇𝑓 𝑤 𝑡 : Direction of the steepest descent
  • 𝛼: Step size or learning rate

Gradient Descent: Initialization

• Initialization of weights is important for gradient descent convergence.

Gradient Descent: Learning Rate

• Learning rate (𝛼) determines the size of the step in each iteration of gradient descent.
• Too small learning rate leads to slow convergence.
• Too large learning rate can cause overshooting the minimum.

Variants of Gradient Descent

• Basic Gradient Descent: Uses entire training data to compute gradients.
  • Accurate but slow.
• Stochastic Gradient Descent (SGD): Uses one training datapoint per iteration.
  • Faster but less accurate.
• Batch Gradient Descent: Computes gradients using a batch of training data.
  • Intermediate strategy between basic and stochastic.

Training

• Weights in neural networks are computed through gradient descent.
• Information propagates layer-wise in neural networks.
• Backpropagation algorithm updates weights efficiently using the chain rule.

Backpropagation: Intuition

• Example of backpropagation: 𝑓 = 𝑥 + 𝑦 𝑧
• Chain rule is used to update weights.
• Backpropagation involves forward and backward passes to update weights.

Customer Churn

• Customers may leave and not return without significant re-acquisition costs.
• Churn is the percentage of customer base leaving in a given period.
• At an individual level, churn refers to the probability of a customer leaving at a given point in time.
• Churn = 1 - Retention rate

Customer Churn and LTV

• Churn management focuses on retention in customer lifetime value (LTV).
• LTV equation: LTV = σ𝑡=0 ∞ 𝑚 𝑡 r𝑡 / (1+𝛿)𝑡 = σ𝑡=0 ∞ 𝑚𝑡 (1−𝑐)𝑡 / (1+𝛿)𝑡
• Churn significantly impacts business, especially in the digital world.

Types of Churn

• Involuntary churn: Company terminating the relationship, often due to poor payment history.
• Voluntary churn: Customer chooses to leave.
  • Deliberate: Dissatisfaction or better competitive offer.
  • Incidental: No longer need the product or moved to a location without service.

Major Factors Causing Churns

• Customer satisfaction: Satisfied customers are less likely to churn.
  • Fit-to-needs is crucial.
• Switching costs: Obstacles customers face while switching to a competitor.
• Network Effects: Benefits gained from more users.
• Competition: Competitive offers and opportunities are prime causes of churn.

Customer Satisfaction and Churn

• More satisfied customers are less likely to churn.
• Product customization can increase satisfaction.

Network Effects and Switching Costs

• Network effects influence consumer choice by increasing benefits with more users.
• Switching costs create consumer lock-in by making it harder for customers to leave.

Competition and Churn

• Competitive offers are major causes of churn.
• Competition can occur within or outside the industry or product category.
• Difficulty in identifying competition makes it challenging to study the impact of competition on churn.

Reducing or Managing Churn

• Untargeted approaches focus on increasing customer satisfaction or switching costs.
• Targeted approaches aim to identify and “rescue” customers most likely to churn.
  • Reactive: Corrective action taken after customer identifies as likely to churn.
  • Proactive: Identifying and addressing potential churn before customer expresses intent to leave.

Reactive Churn Management

• Reactive approaches require accurate prediction of churners.
• Company can incentivize customers to stay based on churn predictions.