Stock Market Analysis with LSTM

Dissertation Title

• Stock Market Analysis and Prediction using LSTM

Author

• Aradhya Mahanta Borah

Degree

• Bachelor of Technology in Computer Science and Engineering

Supervisor

• Mr. Ved Prakash Chaubey

Institution

• Lovely Professional University, Phagwara, Punjab (India)

Acknowledgement

• Expresses gratitude to teacher Ved Prakash Chaubey for support, guidance, and insights
• Acknowledges resources like Tiingo for datasets instrumental in research

Supervisor's Certificate

• Confirms the dissertation is original work
• States the work has not been submitted elsewhere

Table of Contents

• Includes sections like Abstract, Problem Statement, Dataset Description, Solution Approach, Required Libraries/Libs Used, Introduction, Literature Review, Methodology, Results, Analysis, Conclusion, References, and GitHub Repository

Abstract

• Explores Long Short-Term Memory (LSTM) networks for stock market prediction
• Aims to predict closing stock prices using historical data and LSTM models
• Captures complex temporal dependencies in stock market
• Evaluates LSTM's effectiveness in forecasting
• Shows potential of deep learning techniques in financial markets
• Employs metrics like MAE, MSE, and RMSE to assess accuracy

Problem Statement

• Stock market is dynamic and complex
• Inherent unpredictability and high volatility in stock prices make accurate forecasting challenging
• Traditional statistical methods often struggle with complex and non-linear relationships
• Aims to predict future stock prices using historical data

Dataset Description

• Dataset comprises historical stock price data for various stocks
• Includes daily records of features like Date, Open, High, Low, Close, Volume, Adj Close, Adj High/Low/Open, Div Cash, and Split Factor
• Data is preprocessed, cleaned, and normalized before training
• Focuses on Close price as target variable

Solution Approach

• Deep Learning is used
• Relies on Long Short-Term Memory (LSTM) model for prediction
• LSTM networks are used for handling sequential data
• Processes include data preprocessing, feature selection, normalization, data splitting, model architecture design, and training

Required Libraries

• Pandas (data manipulation and analysis)
• NumPy (numerical computation)
• Matplotlib (data visualization)
• Seaborn (statistical data visualization)
• Scikit-learn (machine learning utilities and scaling)
• Keras (deep learning framework, specifically LSTM)
• TensorFlow (deep learning framework)
• Joblib (model serialization)
• OS (file and directory management)
• DateTime (date and time manipulation)
• TensorFlow Hub (pre-trained models and components)

Model Training

• Uses backpropagation through time (BPTT)
• Employs Mean Squared Error (MSE) loss function for measuring difference between predicted and actual stock prices
• Uses Adam Optimizer
• Divides training data into epochs and batch sizes

Evaluation and Testing

• Tests on unseen data
• Analyzes residuals to check for patterns
• Visualization (predicted vs actual prices)

Future Price Prediction

• Uses last 100 days of stock prices to predict next day's price
• Iterative forecasting where predicted value used as input for next day prediction

Literature Review

• Discusses ARIMA, GARCH, SVM, Decision Trees, and Random Forests, exploring their applications in stock price prediction
• Highlights limitations of traditional approaches, emphasizing the need for machine learning models
• Examines deep learning models, focusing on LSTM's ability to handle long-term dependencies

Methodology

• Details the steps for data collection, preprocessing, feature selection, model development, and evaluation
• Includes methods for handling missing data, feature scaling, sequence creation, model architecture (LSTM), and model training using Keras and TensorFlow
• Includes details of LSTM model architecture and its layers (input, LSTM, and dense)

Results

• Presents evaluation metrics like RMSE, MAE, and R-squared
• Plots predicted vs actual stock prices
• Presents analysis of residuals

Comparison with Other Models

• Compares LSTM model performance with Linear Regression model
• Employs metrics like RMSE and R-Squared to evaluate models

Analysis

• Assesses model's generalization performance
• Explores potential enhancements to improve stock price prediction accuracy
• Details ways to improve on limitations of the model

Future Work

• Suggests incorporating external data sources like news, sentiment analysis, and macroeconomic factors
• Recommends exploring more complex models (e.g., attention-based networks) or hybrid approaches