Data Storytelling: Roles and Responsibilities

• Data storytelling involves data originating from various sources. Extracting data and moving it to a suitable location requires effort, particularly in a business context.
• Current applications and projects can be extensive and intricate, often necessitating the combined knowledge and skills of multiple individuals.
• Individuals contribute unique expertise, collaborate on tasks, and coordinate responsibilities.
• Roles like business analysts and business intelligence developers were common for data processing and comprehension.
• An excessive increase in data size and variety has led to the evolution of these roles into specialized skill sets that modernize and streamline data engineering and analysis processes.
• Various data roles and their specific responsibilities in data detection and understanding, are highlighted.

Business Analyst

• Business analysts and data analysts share similarities, but differ in their use of data.
• Business analysts are closely connected to the business, specializing in interpreting data from visualization.
• Data analyst and business analyst duties can be performed by the same person.

Data Analyst

• Data analysts empower companies to maximize the value of their data assets using visualization and reporting tools like Microsoft Power BI.
• Data analysts are responsible for data profiling, cleansing, and transformation.
• Responsibilities include designing and creating scalable and efficient semantic models, and enabling advanced analytics features in reports for analysis.
• Data analysts work with relevant stakeholders to determine data and reporting requirements, then converts raw data into meaningful insights.
• Data Analysts manage Power BI resources, including reports, dashboards, workspaces, and underlying semantic models.
• Data Analysts are responsible for implementing and configuring security procedures and stakeholder requirements to ensure the security of all Power BI resources and data.
• Data analysts collaborate with data engineers to identify and locate appropriate data sources that meet stakeholder needs.
• Data analysts work with data engineers and database administrators to ensure adequate access to data sources, while also collaborating with data engineers to identify new or improve existing data collection processes.

Data Engineer

• Data engineers provision and configure on-premises and cloud-based data platform technologies.
• Data engineers manage and secure the flow of structured and unstructured data from multiple sources, including relational and non-relational databases, data streams, and file stores.
• Data engineers also ensure that data services are integrated securely and seamlessly into data platforms.
• Primary responsibilities include using local and cloud-based data services, along with tools for ingesting, outputting, and transforming data from diverse sources.
• Data engineers work with business stakeholders to identify and meet data requirements, and design and implement solutions.
• While some similarities may exist between the tasks and responsibilities of a data engineer and a database administrator, a data engineer's scope goes beyond database and server maintenance, likely excluding general administration of operational data.
• Data engineers contribute significantly to data science and business intelligence projects.
• Data engineers combine data, often described as data cleansing and transformation, allowing data scientists to focus on their specialized areas, which accelerates projects.
• Data engineers help data analysts ensure access to various structured and unstructured data sources, and optimize semantic models typically served from a modern data warehouse or data lake.
• Both database administrators and business intelligence professionals can become data engineers by familiarizing themselves with the tools and technologies for processing large data volumes.

Data Scientist

• Data scientists perform advanced analysis to extract value from data.
• Data scientists' work ranges from descriptive to predictive analytics.
• Descriptive analysis assesses data through exploratory data analysis (EDA).
• Predictive analysis employs machine learning to apply modeling techniques that can detect anomalies or patterns, which are crucial for forecasting models.
• Descriptive and predictive analysis represent only partial aspects of data scientists' work. Some data scientists may delve into deep learning, conducting iterative experiments to solve complex data problems using complex custom algorithms.
• Anecdotal evidence suggests that a large portion of data science project efforts is spent on data cleansing and transformation, and feature engineering. Data scientists can expedite experimentation when data engineers use their expertise to cleanse and transform data properly.
• At first glance, the work of a data scientist and a data analyst may appear unrelated, but that is not the case.
• Data scientists examine data to determine questions that need answers and design hypotheses or experiments, often turning to data analysts for assistance with data visualization and reporting.

Database Administrator

• Database administrators implement and manage operational aspects of hybrid and cloud-native data platform solutions based on Microsoft Azure and Microsoft SQL Server data services.
• Database administrators are responsible for the overall availability, optimization, and consistent performance of database solutions.
• Database administrators work with stakeholders to identify and implement policies, tools, and processes for backup and recovery plans.
• The role of database administrators differs from that of data engineers.
• Database administrators monitor and manage the overall health of a database and its hardware, while data engineers are involved in the data cleansing and transformation process, specifically ingesting, transforming, validating, and cleansing data to meet business needs and requirements.
• The database administrator is responsible for managing the overall security of the data and granting/restricting data access and privileges to users based on business needs and requirements.