Introduction to Data Visualization & Visual Perception

Introduction to Data Visualization & Visual Perception Created By: Jeffrey A. Shaffer Vice President, Unifund Adjunct Faculty, University of Cincinnati (513) 615-0001 | [email protected] @HighVizAbility Goals By completing the cour...

Introduction to Data Visualization & Visual Perception Created By: Jeffrey A. Shaffer Vice President, Unifund Adjunct Faculty, University of Cincinnati (513) 615-0001 | [email protected] @HighVizAbility Goals By completing the course modules, students will: Define data visualization and express its value as a tool for understanding our world Describe the fundamental aspects visual perception and these relate to data visualization Learn the history of data visualization, important people in the field, and contemporary practitioners Understand the preattentive attributes, the embedded meaning of color and size, and the two thinking systems that we use to process the world around us DATA VISUALIZATION AND REPORTING What You’ll Learn – Create clear and engaging visuals to explore data. – Share findings effectively through visual storytelling. – Use design principles to make visuals look professional. – Build charts and graphs step-by-step. Why Data Visualization Matters – Simplifies complex data – Makes it easier to understand. – Reveals patterns and trends – Helps in decision-making. – Tells a story – Engages your audience with visuals. Communicating Why Data Visualization Matters Insights Through – Essential Skill – Communicating insights from data is crucial in any field. Data – Visual Impact – Visuals are often the most compelling way to share data stories. Visualization Key Components of Data Visualization 1. Design and Art – Creating visuals that are beautiful and engaging. 2. Science and Math – Delivering accurate insights through data. What You’ll Learn in This Lesson – Why visuals are more effective than summary statistics. – Common plots and charts used in data visualization. – An introduction to dashboards for organizing and displaying data. – A preview of design principles to improve your visualizations. What is Data Visualization? And Why is it Important? Analytics: A New Path to Value – MIT 2010 Source: Big Data, Analytics and the Path From Insights to Value MIT Sloan Management Review, December 2010 Visual analytics is a growing field… “…2014 will be a critical year in which the task of making ‘hard types of analysis easy’ for an expanded set of users…will continue to dominate BI market requirements.” – Gartner 2014 Magic Quadrant for Business Intelligence and Analytics Platforms Visual analytics is a growing field… Data visualization classes now taught at universities Has become part of Business Analytics, Statistics, Information Systems, Business Administration and Econometrics. Companies adding visual analyst positions Visually trained business analysts with a data orientation Growing ranks of data “storytellers” and “data visualization” positions Current Data Visualization Jobs (July 2017) Data Visualization Designer/Storyteller at Facebook Data Visualization Expert as a Leading Hedge Fund Data Visualization Manager at Deloitte Source: Indeed.com “In God we trust, all others bring data.” - W. Edwards Deming Business Intelligence Moving from data to action…. Data Information Knowledge Plan to guide actions or key This flow—from data to strategy—is the essence of business decisions intelligence. It’s about turning raw data into actionable insights that drive decision-making and success.  Facts and statistics collected Data together for reference or analysis. Facts provided or learned about Information something or someone. Information and skills acquired through experience or education; Knowledge the theoretical or practical understanding of a subject. A plan of action or policy Strategy designed to achieve a major or overall goal. How many times does the digit 7 appear? Preattentive attributes of visual perception and use of color  Precise Quantitative Comparisons Length or Width 2D Position # of times digit 7 appears: 17 # of times digit 7 appears: 17  Label problems Color problems Hard to make visual comparisons Do we get the same Information?  Data: Answers "How many?" and "How much?" Example: SQL Business Intelligence query can count widgets in Ohio. Information: Contextualizes data. We ask, "What are these?" Knowledge: Seeks reasons with "Why?" Not a simple Moving from data to action…. database query; requires analysis. Strategy: Tackles "How do we?" or "What can we do?" Requires human interpretation for action plans. Data Visualization: Helps with quantity and some "why" questions, but people must decide on "how" to act. Information How many…? How much…? Knowledge What are…? Plan to guide actions or key Why are…? decisions How do we…? What can we…? A Brief History of Data Analytics  The Early Years – 2nd Century CE A copy of the Almagest from the 9th century, in Greek, on parchment Claudius Ptolemy publishes the Almagest around 150 CE in Egypt, providing a thorough treatise on astronomy, solar, lunar, and planetary theory1 Earliest preserved use of a table; held detailed astronomical information2 Image of Ptolemy used under the Wikipedia Creative Commons license. http://www.wikipedia.com Image of the Almagest from the Library of Congress. http://www.loc.gov/exhibits/vatican/math.html  17th Century René Descartes (1596-1650) Invented a method of presenting number-based data using 2-D coordinate scales2 Originally designed to allow algebraic equations to be expressed visually, linking Euclidean geometry and algebra3,4 Later became known as the Cartesian coordinate system3 Images used under the Wikipedia Creative Commons license. http://www.wikipedia.com  Late 18th – Early 19th Century William Playfair (1759-1823) Scottish engineer and political economist5 Founder of graphical methods of statistics Developed new designs and improved existing methods to provide “systematic visual representations of his ‘linear arithmetic.’”6 Created the time-series line graph, bar chart, and pie chart.7 Images used under the Wikipedia Creative Commons license. http://www.wikipedia.com  Early 19th Century Charles Minard (1781-1870) French civil engineer; produced an array of graphics that combine a many data points into a compelling visual story Produced 70+ depictions including thematic maps and graphic tables between 1844- 1870.7 Map of Napoleon’s Russian campaign regarded by some as the “best statistical graphic ever drawn.”8 Images used under the Wikipedia Creative Commons license. http://www.wikipedia.com  20th Century Jacques Bertin (1918-2010) French cartographer and theorist Author of many scientific maps, articles, and other papers on semiology, the study of signs, and how we process visual information 1967 – Published Sémiologie Graphique (Semiology of Graphics), asserting that our visual perception follows rules that can be followed Images used under the Wikipedia Creative Commons license. http://www.wikipedia.com  20th Century John Tukey (1915-2000) American statistician Introduced the box plot in Exploratory Data Analysis, published in 1977 Exploratory Data Analysis (EDA) emphasized presentation of the main characteristics of a data set in a visual, easy to understand form, without using a statistical model or hypothesis10 Images used under the Wikipedia Creative Commons license. http://www.wikipedia.com  20th Century Visualization techniques used in EDA include: Box plot Histogram Sample histogram. Pareto chart Scatter plot Sample box plot. Tukey’s advocacy for EDA encouraged the development of software for statistical computing like S, which inspired S-Plus and R.10 Sample scatter plot. Images used under the Wikipedia Creative Commons license. http://www.wikipedia.com  Contemporary Practitioners Edward R. Tufte (b. 1942) American statistician, sculptor, and Professor Emeritus of Political Science, Statistics, and Computer Science, Yale University Widely recognized expert in the fields of information design and visual literacy11 Credited as a pioneer in teaching the fundamental skills required for visual communication3 Edward Tufte photo from http://www.edwardtufte.com/bboard/images/0003mW-10280.jpg  Contemporary Practitioners Edward R. Tufte The Visual Display of Quantitative Information (1983) Provides an essential reference for how effective design can positively influence understanding Tufte also invented sparklines  Contemporary Practitioners William S. Cleveland Professor of Statistics and Courtesy Professor of Computer Science at Purdue University; previously worked at Bell Labs Authored over 100 papers/publications including Visualizing Data (1993) and The Elements of Graphing Data (1994), to enhance awareness and provide examples of effective data presentation Initial developer of trellis charts, which make visualization possible in data sets William S. Cleveland photo from http://www.stat.purdue.edu/~wsc/ with multiple variables Trellis Chart Example Graphic from http://peltiertech.com/WordPress/trellis-plot-alternative-to-stacked-bar-chart/  Contemporary Practitioners Stephen Few Prolific writer and author with a focus on designing simple information displays that are effective and communicative. Books include: Show Me The Numbers (2004) Now You See It (2009) Information Dashboard Design, 2nd ed. (2013) Several of Few’s examples will be used in class. Few’s biography retrieved from http://www.perceptualedge.com/about.php  Other Critical Events 1984 - Apple Computer introduces the Macintosh Innovations, like the The Macintosh was the first popular graphical user interface and and affordable computer designed to mouse, are invented at Xerox display graphics in an interactive Palo Alto Research Center interface.3 (Xerox PARC). Images used under the Wikipedia Creative Commons license. http://www.wikipedia.com  “I See” Visual Perception “Show me” Used by Permission of Dr. Beau Lotto (www.LottoLab.org) Source: Wiki Commons (Lotus, Illinois Railroad Tracks) Source: Wikipedia (from the Lunar and Planetary Institute: http://www.lpi.usra.edu) The Moiré effect Gridlines spaced and muted The Hermann effect (the scintillating grid)  Unit Chart (notice Hermann effect) Gestalt Principle - Closure (Kanizsa triangle) c- Closure  What letter is this? Adapted from Lera Boroditsky as seen on Brain Games Putting it all together We live in a world that is rich with data Tools let us look at the past, present, and “try out” future scenarios Ultimately, we want to understand our world and make better choices, change behavior, grow wealth, improve quality of life, etc. We discover how to do these things by asking questions through data… Putting it all together The understanding and interpretation of data is an activity of human cognition – Asking questions, discovering patterns, drawing meaning from the data – Creative, visually-driven process; also requires empirical and mathematical skills – Requires subject matter knowledge – Built-in rules (or at least, guidelines) affect the way we process information Easy, right? Not so fast… Software promises to reveal the answers…but… – Computers can’t figure out what our data means, how it connects to the business or problem, and what to do – Data needs are changing (e.g., big data) – Users expect “iPhone easy” We don’t train for visual intelligence Data + tools + brains >>> business intelligence Most people agree that this is data visualization… Source: Information Dashboard Design (2 Ed.) by Stephen Few  Stretching our thinking about data… …what about this? Source: http://www.clevelandart.org/art/collection/search  Stretching our thinking about data… …what about this? Source: http://www.clevelandart.org/art/collection/search Concluding Thoughts Ultimately, we’re trying to understand our world and make better choices, change behavior, enhance wealth, or improve quality of life. Learning more about the way our brains work, how we perceive and process data, and improving how we practice visualization is essential to achieving these goals. The Building Blocks of Data Visualization  We are born with an enhanced visual path to cognition Approximately 70% of sense receptors are in our eyes 40% of the cerebral cortex is involved in processing visual information The visual connection to the brain has more bandwidth than other paths Visual perception is intimately connected to understanding This is reflected in language - “I see what you’re talking about…” - “Sketch out the idea…” - “Seeing is believing…”  Our brain is powerful… but working memory is limited Long-term memory is very important Working memory limited to a small number of “chunks” Visualization allows us to consolidate complex statistics so we can process more data simultaneously (seeing the forest along with the trees) The picture is not the end goal – It’s what we do with it that is important  Making sense of our visual world… According to Bertin, our perception of data on a typical printed page is associated with the following visual variables:9 –Shape –Texture –Orientation –Value –Color –Size …and the two planar dimensions (x and y) which are encoded in Position and Order. Based on the contemporary preponderance of digital technology, these factors have also become critical: –Motion – animated presentation of frames of data –Medium – the physical strata on which data is displayed –Context – the sensory and emotional environment Shape Shape Orientation Orientation Color (Hue) Color (Hue)  Changes to the HUE (color) Changes to the SATURATION (intensity) Changes to the Value (brightness) Relationships on a Traditional Color Wheel Image used under the Wikipedia Creative Commons license. http://www.wikipedia.com The primary colors are red, yellow, and blue. Classical painters would have used this arrangement to find compliments. Relationships on a RGB Color Wheel Image used under the Wikipedia Creative Commons license. http://www.wikipedia.com Computer displays use red, green, and blue elements. This results in a shifted arrangement of complimentary colors. Source: The Big Book of Dashboards (page 15) Does this color use enhance or detract? What does color even mean here? From http://online.wsj.com - The Wall Street Journal Online, originally published August 7, 2010 Color (Value) Color (Value) Color (Value) Texture Texture Size Size Position Position Order Order  We also have built in biases… Psychologists recognize two “thinking systems” that we use to make sense of the world. System 1 (bottom up) – operates automatically and quickly, with little or no effort and no sense of voluntary control System 2 (top down) – allocates attention to the effortful mental activities that demand it, including complex computations  We also have built in biases… System 1 generates impressions, intuitions, intentions, and feelings for system 2 System 2 can be engaged as needed to solve more complex problems or where System 1 runs into difficulty What’s the answer? What’s the answer? System 2 shifts into gear to figure out the answer. (it’s 437)  Impressions in system 1 affect the conclusions of system 2. Presentation impacts the way data is perceived Mood and emotions impact critical thinking Caution: over-simplification can result in distorted understanding We’ll explore this further in a future class. https://www.tableau.com/tft/activation TC62-A1E6-5920-7522-AA6F Citations 1 From Rome Reborn: The Vatican Library & Renaissance Culture, Library of Congress website. Accessed on 2/18/2012. http://www.loc.gov/exhibits/vatican/math.html 2 Few, Stephen (2007), Data Visualization: Past, Present, and Future, p.3 3 Wikipedia entry René Descartes, used under the Creative Commons-Share Alike 3.0 Unported license. Accessed on 2/18/2012. http://en.wikipedia.org/wiki/Rene_descartes 4 Wikipedia entry Cartesian Coordinate System, used under the Creative Commons- Share Alike 3.0 Unported license. Accessed on 2/18/2012 http://en.wikipedia.org/wiki/Cartesian_coordinate_system 5 Wikipedia entry William Playfair, used under the Creative Commons-Share Alike 3.0 Unported license. Accessed on 2/18/2012 http://en.wikipedia.org/wiki/William_Playfair 6 Tufte, Edward R. (2001), The Visual Display of Quantitative Information, Second Edition, p. 9 7 Friendly, Michael (2001), Gallery of Data Visualization, Electronic document, http://www.datavis.ca/gallery/, Accessed: 02/19/2012 22:20:39 8 Tufte, Edward R. (2001), The Visual Display of Quantitative Information, Second Edition, p. 40 9 Bertin, Jacques (1983), Semiology of Graphics, English translation by William J. Berg, pp. 42, 65 10 Wikipedia entry Exploratory Data Analysis, https://en.wikipedia.org/wiki/Exploratory_data_analysis 11 Wikipediaentry Edward Tufte, used under the Creative Commons-Share Alike 3.0 Unported license. Accessed on 2/19/2012. http://en.wikipedia.org/wiki/Edward_Tufte