chapter07_ID5e_2024.pdf

Chapter 7 INTERFACES  20 interface types covered 1. Command 2. Graphical 3. Multimedia 4. Virtual reality 5. Web...

Chapter 7 INTERFACES  20 interface types covered 1. Command 2. Graphical 3. Multimedia 4. Virtual reality 5. Web 6. Mobile 7. Appliance 8. Voice 9. Pen 10. Touch 11. Gesture 12. Haptic 13. Multimodal 14. Shareable 15. Tangible 16. Augmented Reality 17. Wearables 18. Robots and drones 19. Brain–computer interaction 20. Smart Introduction | Interface Types | Which Interface? | Summary 2  Command line interfaces Commands such as abbreviations (for instance, ls) typed in at the prompt to which the system responds (for example, by listing current files) Some are hard wired at keyboard (e.g. delete button), while others can be assigned to keys (e.g. F11 for print) + Efficient, precise, and fast - Large overhead to learning set of commands Introduction | Interface Types | Which Interface? | Summary 3  Second Life command line-based interface for visually-impaired users Introduction | Interface Types | Which Interface? | Summary 4  Research and design considerations Form, name types and structure are key research questions Consistency is most important design principle  For example, always use first letter of command Introduction | Interface Types | Which Interface? | Summary 5  Graphical user interfaces (GUIs) Original GUI was called WIMP and consisted of: Windows  Sections of the screen that can be scrolled, stretched, overlapped, opened, closed, and moved around the screen using the mouse Icons  Pictograms that represent applications, objects, commands, and tools that were opened when clicked on Menus  Lists of options that can be scrolled through and selected Pointing device  A mouse controlling the cursor as a point of entry to the windows, menus, and icons on the screen Introduction | Interface Types | Which Interface? | Summary 6  Example of first generation GUI Introduction | Interface Types | Which Interface? | Summary 7  Simple smartwatch menus with 1, 2, or 3 options Introduction | Interface Types | Which Interface? | Summary 8  Window design Windows were invented to overcome the physical constraints of a computer display  They enable more information to be viewed and tasks to be performed Scroll bars within windows enable more information to be viewed Multiple windows can make it difficult to find desired one  Listing, tabbing, and thumbnails are techniques that can help Introduction | Interface Types | Which Interface? | Summary 9  Window design: Thumbnails of top websites visited and suggested highlights Introduction | Interface Types | Which Interface? | Summary 10  Selecting a country from a scrolling window Introduction | Interface Types | Which Interface? | Summary 11  Is this method any better? Introduction | Interface Types | Which Interface? | Summary 12  Menu styles Flat list: Good for showing small number of options at the same time when display is small Expanding : Shows more options on same screen (for example, cascading) Contextual: Provides access to often-used commands associated with a particular item Collapsible: Toggles between + and − icons on a header to expand or contract its contents Mega: All options shown using 2D drop-down layout Introduction | Interface Types | Which Interface? | Summary 13  Template for a collapsible menu Introduction | Interface Types | Which Interface? | Summary 14  A mega menu Introduction | Interface Types | Which Interface? | Summary 15  Research and design considerations Window management  Enables users to move fluidly between different windows (and monitors) How to switch attention between windows without getting distracted Design principles of spacing, grouping, and simplicity should be used Which terms to use for menu options (for example, “front” versus “bring to front” Mega menus easier to navigate than drop-down ones Introduction | Interface Types | Which Interface? | Summary 16  Icon design Icons are assumed to be easier to learn and remember than commands Icons can be designed to be compact and variably positioned on a screen Now pervasive in every interface  For example, they represent desktop objects, tools (for example, a paintbrush), applications (for instance, a web browser), and operations (such as cut, paste, next, accept, and change) Introduction | Interface Types | Which Interface? | Summary 17  Icon forms The mapping between the representation and underlying referent can be:  Similar (for example, a picture of a file to represent the object file)  Analogical (for instance, a picture of a pair of scissors to represent ‘cut’)  Arbitrary (such as the use of an X to represent ‘delete’) The most effective icons are similar ones Many operations are actions making it more difficult to represent them  Use a combination of objects and symbols that capture the salient part of an action Introduction | Interface Types | Which Interface? | Summary 18  Research and design considerations There is a wealth of resources for creating icons  Guidelines, style guides, icon builders, libraries, online tutorials Text labels can be used alongside icons to help identification for small icon sets For large icon sets (for instance, photo editing or word processing) can use the hover function Introduction | Interface Types | Which Interface? | Summary 19  Activity Sketch simple icons to represent the following operations to appear on a digital camera screen:  Turn image 90-degrees sideways  Auto-enhance the image  Crop the image  More options Show your icon to the class and see if they can understand what each represents Introduction | Interface Types | Which Interface? | Summary 20  Multimedia Combines different media within a single interface with various forms of interactivity  Graphics, text, video, sound, and animation Users click on links in an image or text  Another part of the program  An animation or a video clip is played  Users can return to where they were or move on to another place Can provide better ways of presenting information than a single media can Introduction | Interface Types | Which Interface? | Summary 21  Pros and cons Facilitates rapid access to multiple representations of information Can provide better ways of presenting information than can any media alone Can enable easier learning, better understanding, more engagement, and more pleasure Can encourage users to explore different parts of a game or story Tendency to play video clips and animations while skimming through accompanying text or diagrams Introduction | Interface Types | Which Interface? | Summary 22  Multimedia learning app designed for tablet Introduction | Interface Types | Which Interface? | Summary 23  Research and design considerations How to design multimedia to help users explore, keep track of, and integrate the multiple representations  Provide hands-on interactivities and simulations that the user has to complete to solve a task  Provide quizzes, electronic notebooks, and games Multimedia good for supporting certain activities, such as browsing, but less optimal for reading at length Introduction | Interface Types | Which Interface? | Summary 24  Virtual reality Computer-generated graphical simulations providing:  “the illusion of participation in a synthetic environment rather than external observation of such an environment” (Gigante, 1993) Provide new kinds of experience, enabling users to interact with objects and navigate in 3D space Create highly-engaging user experiences Introduction | Interface Types | Which Interface? | Summary 25  Pros and cons Can have a higher level of fidelity with objects that they represent compared to multimedia Induces a sense of presence where someone is totally engrossed by the experience Provides different viewpoints: first and third person Early head-mounted displays were uncomfortable to wear and could cause motion sickness and disorientation Lighter VR headsets are now available (for example, HTC Vive) with more accurate head tracking Introduction | Interface Types | Which Interface? | Summary 26  Application areas Video games Arcade games for social groups Therapy for fears Experience how others feel emotions  For example, empathy and compassion Enrich user’s planning experience for travel destinations Architecture, design, and education Introduction | Interface Types | Which Interface? | Summary 27  Research and design considerations Much research on how to design safe and realistic VRs to facilitate training  For example, flying simulators  Help people overcome phobias (for example, spiders or talking in public) Design issues  How best to navigate through them (for instance, first versus third person)  How to control interactions and movements (for example, by using head and body movements)  How best to interact with information (for instance by using keypads, pointing, and joystick buttons)  Level of realism to aim for to engender a sense of presence Introduction | Interface Types | Which Interface? | Summary 28  Website design Early websites were largely text-based, providing hyperlinks Concern was with how best to structure information to enable users to navigate and access them easily and quickly Nowadays, more emphasis is on making pages distinctive, striking, and aesthetically pleasing Need to think of how to design information for multiple platforms—keyboard or touch?  For example, smartphones, tablets, and PCs Introduction | Interface Types | Which Interface? | Summary 29  Usability versus aesthetics? Vanilla or multi-flavor design?  Ease of finding something versus aesthetic and enjoyable experience Web designers  “thinking great literature” Users  “billboard going by at 120 km/h” Need to determine how to brand a web page to catch and keep ‘eyeballs’ Introduction | Interface Types | Which Interface? | Summary 30  Breadcrumbs for navigation Breadcrumbs are category labels: Enable users to look at other pages without losing track of where they have come from Very usable Enable one-click access to higher site levels Attract first time visitors to continue to browse a website having viewed the landing page Introduction | Interface Types | Which Interface? | Summary 31  In your face Web ads Web advertising is often intrusive and pervasive Flashing, aggressive, persistent, and annoying Often requires action to get rid of Introduction | Interface Types | Which Interface? | Summary 32  Research and design considerations Many books and guidelines on website design Veen’s (2001) three core questions to consider when designing any website: 1. Where am I? 2. Where can I go? 3. What’s here? Introduction | Interface Types | Which Interface? | Summary 33  Mobile interfaces Handheld devices intended to be used while on the move Have become pervasive, increasingly used in all aspects of everyday and working life  For example, phones, fitness trackers, and smartwatches Larger-sized tablets used in mobile settings  Including those used by flight attendants and marketing professionals Introduction | Interface Types | Which Interface? | Summary 35  QR codes and smartphones Introduction | Interface Types | Which Interface? | Summary 36  Fitts’ Law Predicts the time it takes to move to and select a target based on the size of the target and the distance to it. The larger and closer a target is, the faster it can be selected Introduction | Interface Types | Which Interface? | Summary 37  Research and design considerations Mobile interfaces can be cumbersome to use for those with poor manual dexterity or ‘fat’ fingers Key concern is hit area:  Area on the phone display that the user touches to make something happen, such as a key, an icon, a button, or an app  Space needs to be big enough for all fingers to press accurately  If too small, the user may accidentally press the wrong key  Fitts’ law can be used to help design right spacing  Apple Suggests a minimum tappable areas should be 44 points x 44 points for all controls Introduction | Interface Types | Which Interface? | Summary 38  Appliances Everyday devices in home, public places, or car  For example, washing machines, remotes, toasters, printers, and navigation systems) And personal devices  For instance, digital clock and digital camera Used for short periods  For example, starting the washing machine, watching a program, buying a ticket, changing the time, or taking a snapshot Need to be usable with minimal, if any, learning Introduction | Interface Types | Which Interface? | Summary 39  Toaster control Introduction | Interface Types | Which Interface? | Summary 40  Research and design considerations Need to design as transient interfaces with short interactions Simple interfaces Consider trade-off between soft and hard controls  For example, use of buttons or keys, dials, or scrolling Introduction | Interface Types | Which Interface? | Summary 41  Voice User Interfaces Involves a person talking with a spoken language Used most for inquiring about specific information, for example, flight times or to perform a transaction, such as buying a ticket Also used by people with visual impairments  For example, speech recognition word processors, page scanners, web readers, and home control systems Introduction | Interface Types | Which Interface? | Summary 42  Have speech interfaces come of age? Introduction | Interface Types | Which Interface? | Summary 43  Modeling human conversations People often interrupt each other in a conversation  Especially when ordering in a restaurant, rather than let the waiter go through all of the options Speech technology has a similar feature called ‘barge-in’  Users can choose an option before the system has finished listing all of the options available Introduction | Interface Types | Which Interface? | Summary 44  Structuring VUI dialogs Directed dialogs are where the system is in control of the conversation  Where it asks specific questions and requires specific responses More flexible systems allow the user to take the initiative:  For example, “I’d like to go to Paris next Monday for two weeks.” But more chance of error, since caller might assume that the system is like a human Guided prompts can help callers back on track  For example, “Sorry I did not get all that. Did you say you wanted to fly next Monday?” Introduction | Interface Types | Which Interface? | Summary 45  Voice assistants (for example, Alexa) Have become popular in many homes Allow all to use rather than being single use Support families playing games, interactive storytelling, jokes, and so forth Can encourage social and emotional bonding Young children (under 4), however, find it difficult to be understood by the voice assistants  Frustrating for them Introduction | Interface Types | Which Interface? | Summary 46  Research and design considerations How to design systems that can keep conversation on track  Help people navigate efficiently through a menu system  Enable them to recover easily from errors  Guide those who are vague or ambiguous in their requests for information or services Type of voice actor (for example, male, female, neutral, or dialect)  Do people prefer to listen to and are more patient with a female or male voice, a northern or southern accent? Introduction | Interface Types | Which Interface? | Summary 47  Pen-based devices Enable people to write, draw, select, and move objects at an interface using light pens or styluses  Capitalize on the well-honed drawing skills developed from childhood Digital ink, for example, Anoto, use a combination of ordinary ink pen with digital camera that digitally records everything written with the pen on special paper Introduction | Interface Types | Which Interface? | Summary 48  The Anoto pen being used and its internal components Introduction | Interface Types | Which Interface? | Summary 49  Advantages Allows users to annotate existing documents quickly and easily Can be used to fill in paper-based forms that can readily be converted to a digital record using standard typeface Can be used by remote teams to communicate and work on the same documents Introduction | Interface Types | Which Interface? | Summary 50  Touchscreens Single touchscreens are used in walk-up kiosks (such as ticket machines and ATMs) to detect the presence and location of a person’s touch on the display Multi-touch surfaces support a range of more dynamic finger tip actions, for example, swiping, flicking, pinching, pushing, and tapping They do so by registering touches at multiple locations using a grid Now used for many kinds of displays, such as smartphones, iPods, tablets, and tabletops  Supports one and two hand gestures, including tapping, zooming, stretching, flicking, dwelling, and dragging Introduction | Interface Types | Which Interface? | Summary 51  Research and design considerations Provides fluid and direct styles of interaction involving freehand and pen-based gestures for certain tasks Core design concerns include whether size, orientation, and shape of touch displays effect collaboration Much faster to scroll through wheels, carousels, and bars of thumbnail images or lists of options by finger flicking Gestures need to be learned for multi-touch, so a small set of gestures for common commands is preferable More cumbersome, error-prone, and slower to type using a virtual keyboard on a touch display than using a physical keyboard Introduction | Interface Types | Which Interface? | Summary 52  Gesture-based systems Gestures involve moving arms and hands to communicate Uses camera recognition, sensor, and computer vision techniques  Recognize people’s arm and hand gestures in a room  Gestures need to be presented sequentially to be understood (compare with the way sentences are constructed) Introduction | Interface Types | Which Interface? | Summary 53  Gestures used in the operating theater Recognizes core gestures for manipulating MRI or CT images using Microsoft Kinect Introduction | Interface Types | Which Interface? | Summary 54  Research and design considerations How does computer recognize and delineate user’s gestures?  Start and end points?  Difference between deictic and hand waving Introduction | Interface Types | Which Interface? | Summary 55  Haptic interfaces Provide tactile feedback  By applying vibration and forces to a person’s body, using actuators that are embedded in their clothing or a device they are carrying, such as a smartphone Vibrotactile feedback can be used to simulate the sense of touch between remote people who want to communicate Ultrahaptics creates the illusion of touch in midair using ultrasound to make the illusion of 3D shapes Introduction | Interface Types | Which Interface? | Summary 56  Realtime vibrotactile feedback Provides nudges when playing violin incorrectly Uses motion capture to sense arm movements that deviate from model Nudges are short vibrations on arms and hands Introduction | Interface Types | Which Interface? | Summary 57  Research and design considerations Where best to place actuators on body Whether to use single or sequence of ‘touches’ When to buzz and how intense How does the wearer feel it in different contexts? What kind of new smartphone/smartwatch apps can use vibrotactile creatively?  For example, slow tapping to feel like water drops meant to indicate that it is about to rain, and heavy tapping to indicate a thunderstorm is looming Introduction | Interface Types | Which Interface? | Summary 58  Multimodal Interfaces Provide enriched user experiences  By multiplying how information is experienced and detected using different modalities, such as touch, sight, sound, and speech  Support more flexible, efficient, and expressive means of human-computer interaction  Most common is speech and vision Can be combined with multi-sensor input to enable other aspects of the human body to be tracked  For example, eye gaze, facial expression, and lip movements  Provides input for customizing user interfaces Introduction | Interface Types | Which Interface? | Summary 59  Tracking a person’s movements Kinect camera can detect multimodal input in real time using RGA camera for facial recognition and gestures, depth camera for movement tracking, and microphones for voice recognition Used to build model of person and represented as avatar on display programmed to move just like them Introduction | Interface Types | Which Interface? | Summary 60  Research and design considerations Need to recognize and analyze user behavior, for example, speech, gesture, handwriting, or eye gaze Much harder to calibrate these than single modality systems What is gained from combining different input and outputs Is talking and gesturing, as humans do with other humans, a natural way of interacting with a computer? Introduction | Interface Types | Which Interface? | Summary 61  Shareable interfaces Designed for more than one person to use: Provide multiple inputs and sometimes allow simultaneous input by co-located groups Large wall displays where people use their own pens or gestures Interactive tabletops where small groups interact with information using their fingertips For example, DiamondTouch, Smart Table, and Surface Introduction | Interface Types | Which Interface? | Summary 62  A smartboard and an interactive tabletop interface Introduction | Interface Types | Which Interface? | Summary 63  Benefits Provide a large interactional space that can support flexible group working Can be used by multiple users  Can point to and touch information being displayed  Simultaneously view the interactions and have the same shared point of reference as others Can support more equitable participation compared with groups using single PC Introduction | Interface Types | Which Interface? | Summary 64  Research and design considerations Core design concerns include whether size, orientation, and shape of the display have an effect on collaboration Horizontal surfaces compared with vertical ones support more turn-taking and collaborative working in co-located groups Providing larger-sized tabletops does not improve group working but encourages more division of labor Having both personal and shared spaces enables groups to work on their own and in a group  Cross-device systems have been developed to support seamless switching between these, for example, SurfaceConstellations Introduction | Interface Types | Which Interface? | Summary 65  Tangible Interfaces Type of sensor-based interaction, where physical objects, for example, bricks, are coupled with digital representations When a person manipulates the physical object/s, it causes a digital effect to occur, for example, an animation Digital effects can take place in a number of media and places, or they can be embedded in the physical object Introduction | Interface Types | Which Interface? | Summary 66  Osmo Introduction | Interface Types | Which Interface? | Summary 67  Benefits Can be held in one or both hands and combined and manipulated in ways not possible using other interfaces  Allows for more than one person to explore the interface together  Objects can be placed on top of each other, beside each other, and inside each other  Encourages different ways of representing and exploring a problem space People are able to see and understand situations differently  Can lead to greater insight, learning, and problem-solving than with other kinds of interfaces  Can facilitate creativity and reflection Introduction | Interface Types | Which Interface? | Summary 68  VoxBox A tangible system that gathers opinions at events through playful and engaging interaction (Goldsteijn et al., 2015) Introduction | Interface Types | Which Interface? | Summary 69  Research and design considerations What kinds of conceptual frameworks to use to help identify novel and specific features What kind of coupling to use between the physical action and digital effect  If it is to support learning, then an explicit mapping between action and effect is critical  If it is for entertainment, then it can be better to design it to be more implicit and unexpected What kind of physical artifact to use  Bricks, cubes, and other component sets are most commonly used because of flexibility and simplicity  Stickies and cardboard tokens can also be used for placing material onto a surface With what kinds of digital outputs should tangible interfaces be combined? Introduction | Interface Types | Which Interface? | Summary 70  Augmented Reality Augmented reality: Virtual representations are superimposed on physical devices and objects Pokémon Go made it a household game  Used smartphone camera and GPS to place virtual characters onto objects in the environment as if they really are there Many other applications including medicine, navigation, air traffic control, games, and everyday exploring Introduction | Interface Types | Which Interface? | Summary 71  AR that uses forward facing camera Enables virtual try-ons (for example, Snapchat filters) AT mirrors set up in retail stores for trying on make-up, sunglasses, jewelry  Convenient, engaging, and easy to compare more choices  But cannot feel the weight, texture, or smell of what is being tried on Can be used to enable users to step into a character (for example Queen Victoria) Introduction | Interface Types | Which Interface? | Summary 72  Snapchat Lenses Introduction | Interface Types | Which Interface? | Summary 73  Research and design considerations What kind of digital augmentation?  When and where in physical environment?  Needs to stand out but not distract from ongoing task  Needs to be able to align with real world objects  What happens if the AR is slightly off? What kind of device?  Smartphone, tablet, head up display or other? Introduction | Interface Types | Which Interface? | Summary 74  Wearables First developments were head- and eyewear- mounted cameras that enabled user to record what was seen and to access digital information Since then, jewelry, head-mounted caps, smart fabrics, glasses, shoes, and jackets have all been used  Provides the user with a means of interacting with digital information while on the move Applications include automatic diaries, tour guides, cycle indicators, and fashion clothing Introduction | Interface Types | Which Interface? | Summary 75  Google Glass (2014) Why was there so much excitement and concern about people filming what they could see right in front of them? Introduction | Interface Types | Which Interface? | Summary 76  Research and design considerations Comfort  Needs to be light, small, not get in the way, fashionable, and preferably hidden in the clothing Hygiene  Is it possible to wash or clean the clothing once worn? Ease of wear  How easy is it to remove the electronic gadgetry and replace it? Usability  How does the user control the devices that are embedded in the clothing? Introduction | Interface Types | Which Interface? | Summary 77  Robots Main types Remote robots used in hazardous settings  Can be controlled to investigate bombs and other dangerous materials Domestic robots helping around the house  Can pick up objects and do daily chores like vacuuming Pet robots as human companions  Have therapeutic qualities, helping to reduce stress and loneliness Sociable robots that work collaboratively with humans  Encourage social behaviors Introduction | Interface Types | Which Interface? | Summary 78  Social robots: Mel and Paro Cute and cuddly Can open and close eyes and make sounds and movements Source: Images courtesy of Mitsubishi Electric Research Labs. Introduction | Interface Types | Which Interface? | Summary 79  Drones Unmanned aircraft that are controlled remotely and used in a number of contexts  For example, entertainment, such as carrying drinks and food to people at festivals and parties  Agricultural applications, such as flying them over vineyards and fields to collect data about crops, which is useful to farmers  Helping to track poachers in wildlife parks in Africa Can fly low and and stream photos to a ground station where images can be stitched together into maps Can be used to determine the health of a crop, or when it is the best time to harvest the crop Introduction | Interface Types | Which Interface? | Summary 80  Drone being used to survey the state of a vineyard Introduction | Interface Types | Which Interface? | Summary 81  Research and design considerations How do humans react to physical robots designed to exhibit behaviors (for example, making facial expressions) compared with virtual ones? Should robots be designed to be human-like or look like and behave like robots that serve a clearly-defined purpose? Should the interaction be designed to enable people to interact with the robot as if it was another human being or more human-computer-like (for example, pressing buttons to issue commands)? Is it acceptable to use unmanned drones to take a series of images or videos of fields, towns, and private property without permission or people knowing what is happening? Introduction | Interface Types | Which Interface? | Summary 82  Brain-computer interfaces Brain-computer interfaces (BCI) provide a communication pathway between a person’s brain waves and an external device, such as a cursor on a screen Person is trained to concentrate on the task, for example, moving the cursor BCIs work through detecting changes in the neural functioning in the brain BCIs apps:  Games (for example, Brain Ball)  Enable people who are paralyzed to control robots Introduction | Interface Types | Which Interface? | Summary 83  ABC : Arabic Brain Communicator Introduction | Interface Types | Which Interface? | Summary 84  Smart interfaces Smart: phones, speakers, watches, cars, buildings, cites Smart refers to having some intelligence and connected to the internet and other devices Context-aware  Understand what is happening around them and execute appropriate actions, for example, a Nest thermostat Human-building interaction  Buildings are designed to sense and act on behalf of the inhabitants but also allow them to have some control and interaction with the automated systems Introduction | Interface Types | Which Interface? | Summary 85  Which interface? Which interface to use will depend on task, users, context, cost, robustness, and so on Is multimedia better than tangible interfaces for learning? Is speech as effective as a command-based interface? Is a multimodal interface more effective than a mono-modal interface? Will wearable interfaces be better than mobile interfaces for helping people to find information in foreign cities? Are virtual environments the ultimate interface for playing games? Are shareable interfaces better at supporting communication and collaboration compared with using networked desktop PCs? Introduction | Interface Types | Which Interface? | Summary 86  Summary Many innovative interfaces have emerged in last 30 years, including speech, wearable, mobile, brain, and tangible This raises many design and research questions as to decide which to use  For example, how best to represent information to the user so that they can carry out ongoing activity or task New smart interfaces that are context-aware and monitor people  Raising new ethical issues concerned with what data is being collected and what it is used for Introduction | Interface Types | Which Interface? | Summary 87