Data Analysis, Data Engineering, and Data Science - PDF

Application of Communication and Information Technologies Fall 2024 Course Instructor: Mr. Abu Bakar Siddique - Lecturer Department of Creative Technologies Air University - Islamabad [email protected] About Instructor Mr. Abu B...

Application of Communication and Information Technologies Fall 2024 Course Instructor: Mr. Abu Bakar Siddique - Lecturer Department of Creative Technologies Air University - Islamabad [email protected] About Instructor Mr. Abu Bakar Siddique MS Computer Engineering GIK Institute BS Computer Software Engineering UET Peshawar Topics Covered Data Analysis Data Engineering Data Science Machine Learning Various Data based Job Roles Data Analyst Data Engineer Data Scientist ML Engineer Machine Learning Development Life Cycle To build a ML based software system: Plan Data Process EDA Optimize Deploy Evaluate Modelling Continued Due to this process, there are different jobs. No individuals can do all these steps alone especially if you are working in a big company. So, job roles of DE, DA, DS, ML engineer, are divided among these steps. Data Engineer A Data Engineer is responsible for gathering, organizing, and delivering data to ensure its availability for analysis and decision-making processes. Database to Datawarehouse. Data Engineering is the pure software field. Data Engineer Job roles: Scrape Data from the given sources Store the data in optimal servers/warehouse. Build data pipelines/APIs for easy access to the data. Handle databases/data warehouses. Data Engineer Skills required: Strong grasp of algorithms and data structures. Programming languages ( Java/R/Python) and script writing. Advanced DBMS’s. Big data tools (Apache Spark, Hadoop, etc) Cloud platforms (Amazon web services, Google cloud platform) Distributed systems Data pipelines Data Analyst Now as we have data, in data analysis process, data analyst summarize the past data. For example, we have a particular sale company data, why particular product failed? The core part is to run analysis on data. Data Analyst Cleaning and organizing raw data. Analyzing data to derive insights. Creating data visualizations. Producing and maintaining reports. Collaborating with teams based on the insight gained. Optimizing data collection procedures. Data Analyst Skills needed: Statistical programming Programming languages (R/Python) Creating and analytical thinking Business acumen – Medium to High preferred Strong communication skills Data mining, cleaning and munging Data visualization Data story telling SQL Advanced Microsoft Excel Data analyst vs Data Engineer https://www.youtube.com/shorts/ ktYs9Qg3ioI Data Scientist “A data scientist is someone who is better at statistics than any software engineer and better at software engineering than any statistician”. Data analyst – Past Data Scientist – Future Goal is to build best model. Machine Learning Engineer Responsibilities: Deploying machine learning models to production ready environment. Scaling and optimizing model for production. Monitoring and maintenance of deployed models. Machine Learning Engineer skills Mathematics Programming languages (R/Python) Distributed systems Data model and evaluation Machine learning models Software engineering and systems design Comparison Analytical skills Business Data Soft skills Software skills acumen storytelling Data analyst High Medium to high High Medium to high Medium Data Engineer Medium Low Low Medium High Data Scientist High High High High Medium ML Engineer Medium to high Medium Low High High Thank you!