Data Science Tools & Python

Popular Data Science Tools

• Python, R, Microsoft Excel, SAS, and SPSS are used for data pre-processing and analysis
• Tableau, Qlikview, and Microsoft Excel are used for data exploration and visualization
• Apache Spark and Apache Hadoop are used for parallel and distributed computing with big data

Evolution of Python

• Guido van Rossum developed Python in the late eighties at the 'National Research Institute for Mathematics and Computer Science' in the Netherlands
• Python's editions include Python 1.0, 2.0, and 3.0

Python as a Programming Language

• Supports multiple programming paradigms: Functional, Structural, and OOPs
• Uses dynamic typing, with runtime type safety checks
• Employs reference counts, deallocating unused objects
• Uses late binding, where methods are looked up by name during runtime
• Tim Peters' Zen of Python describes the 20 aphorisms that guide it's design
• The "Python Software Foundation" manages the Standard CPython interpreter
• Other interpreters include: JPython (Java), Iron Python (C#), Stackless Python (C, for parallelism), and PyPy (Python itself JIT compilation)
• Standard libraries are written in Python itself and maintains high standards of readability
• Python is cross-platform, meaning it's compatible across Windows, Linux, and Mac
• It has a large community group, providing high support and better error handling
• In comparison to Java, Java is statically typed, meaning type safety is checked during compilation (static compilation)
• Therefore, in Java the time required to develop the code is more
• Python, which is dynamically typed, compensates for huge compilation time compared to Java
• Dynamically typed codes tend to be less verbose, offering more readability

Advantages of Using Python

• Features make it well suited for data science
• It is open source and benefits from community development and it's Open Source Initiative license allows it to be used and distributed commercially for free
• Python has a simple syntax and is easy to understand
• Many libraries are designed for specific data science tasks
• It combines well with the majority of the cloud platform service providers

Coding Environment

• A software program can be written using a terminal, command prompt (cmd), a text editor or through an Integrated Development Environment (IDE)
• The program needs to be saved in a file with an appropriate extension, such as “.py” for Python or “.mat” for Matlab
• Then it can be executed in corresponding environment, i.e. Python, Matlab
• An Integrated Development Environment (IDE) is a software product solely developed to support software development in various or specific programming language(s)
• Python 2.x support was available until 2020
• Python 3.x is an enhanced version of 2.x maintained from 3.6.x post 2020
• It is recommended to install a basic python version or use the online python console
• Basic print statements, naming conventions, conditional operations, function declarations, and installing modules, should be executed to view the outputs

Integrated Development Environment (IDE)

• Software application made of a cohesive unit of tools required for development
• IDEs are designed to simplify software development
• Utilities include tools for managing, compiling, deploying, and debugging software

Coding Environment: IDE Features

• IDEs usually have a source code editor, compiler, and debugger
• Additional features include syntax and error highlighting and code completion
• Supports building and executing programs along with debugging

Coding Environment: IDE Examples

• Version control features are included in the best IDEs
• Popular general IDEs with Python support: Eclipse+PyDev, Sublime Text, Atom, GNU Emacs, Vi/Vim, Visual Studio, Visual Studio Code
• Python-specific editors: Pycharm, Jupyter, Spyder, Thonny

Spyder IDE

• Supported across Linux, Mac OS X, and Windows platforms
• Available as open-source
• Can be installed separately or through Anaconda distribution
• Specifically developed for Python and data science
• Features include a code editor with robust syntax and error highlighting, as well as code completion, navigation, a debugger, and integrated documents
• Interface is similar to MATLAB and RStudio

PyCharm IDE

• Compatible with Linux, Mac OS X, and Windows platforms
• Available as a community (free open source) and professional (paid) version
• Only supports Python
• Can be installed separately or through Anaconda distribution
• Features include a code editor with syntax and error highlighting, code completion, navigation, unit testing, debugger, and version control

Jupyter Notebook

• Web application for creation and manipulation of documents called 'notebooks'
• Supported across Linux, Mac OS X, and Windows platforms
• Available as open source
• Bundled with Anaconda distribution or can be installed separately
• Supports Julia, Python, R, and Scala
• Consists of an ordered collection of input and output cells that contain code, text, plots, etc
• Users can also share code and narrative text through output formats like PDF, HTML etc
• Jupyter Notebook is an education and presentation tool
• Jupyter Notebook lacks most of the features of a good IDE

How to Choose the Best IDE

• Requirements and working with different IDEs helps understand one's own requirements