Introduction To Multimodal Interface Design PDF

Course Title  CS3483 Multimodal Interface Design 1 Staff  Prof. Raymond H.S. Wong ◦ Room: Y6319 ◦ Tel: 3442-8624 ◦ E-mail: [email protected] 2 Assessment  Examination 40%  Projec...

Course Title  CS3483 Multimodal Interface Design 1 Staff  Prof. Raymond H.S. Wong ◦ Room: Y6319 ◦ Tel: 3442-8624 ◦ E-mail: [email protected] 2 Assessment  Examination 40%  Project 30%  Assignment 20%  Tutorial 10%  For a student to pass this course, at least 50% of the maximum mark for the coursework and 30% of the maximum mark for the examination must be obtained. 3 Interface Design (Example 1) 4 Problems with the Design  Cache size settings expressed as a percentage of the disk size.  Minimum setting is 1%  This results in an excessively large cache as capacity of hard drive increases. 5 Improved Design 6 Interface Design (Example 2) 7 Problems with the Design  Large number of tabs makes it difficult to locate any particular one.  The placement of tabs does not follow any logical arrangement.  Clicking any rows in the back row will cause a rearrangement of the tabs. 8 How about this one …… 9 Alternative Design 10 Interface Design (Example 3) 11 Example 3  Most Recently Used (MRU) file concept.  Maintains a brief list of files recently accessed.  Most people tend to return to the same file upon restarting an application. 12 Interface Design (Example 4) 13 Example 4  Easy to forget password for a particular account.  When specifying a new password, the system asks the user to provide a set of questions and answers.  There is an “I forgot…” option which prompts user with the questions. 14 Case Study: Apple Quicktime  How to determine if a direct manipulation interface is well designed?  We compare an old and a new version of Apple Quicktime. 15 Case Study: Apple Quicktime  The designers of Quicktime aim for a simple interface.  The features of the interface should also resemble the physical devices.  Problems ◦ The physical environment is not exactly the same as the interface environment. ◦ The application will inherit the limitations of the physical device. 16 Case Study: Apple Quicktime Thumbwheel design Drawer interface 17 Case Study: Apple Quicktime  The volume control is designed in the form of a thumbwheel.  Problem ◦ The thumbwheel is originally designed to be operated by a thumb. ◦ It will be difficult to identify this control on the interface. 18 Case Study: Apple Quicktime  A drawer interface element is used to hold the advanced controls.  The drawer can be opened or closed using a special button.  Problems ◦ The image on the button does not indicate its function. ◦ Users may prefer some of the controls, like reverse and fast forward, to be always available. 19 Case Study: Apple Quicktime 20 Case Study: Apple Quicktime  Improvements in new version ◦ The thumbwheel is replaced by a slider. ◦ The reverse and fast forward controls have been moved to the main window. 21 What is Human-Computer Interaction? Human Computer Interface 22 Human-Computer Interaction  The study of how people design, implement and use interactive computer systems.  The study of how computers affect individuals, organizations, and society. 23 Requirements of HCI  Ease of use ◦ GUI vs command language ◦ Online help, documentation and training  More powerful forms of communications between users and computers ◦ New interaction techniques ◦ New input and output devices 24 Requirements of HCI  Universal access to information ◦ Large community of users who are not technically sophisticated. ◦ Necessary to develop interfaces that required minimal technical sophistication. 25 Requirements of HCI  Universal access to information ◦ Information-access interfaces  Must be able to deal with diverse types of data, e.g., multimedia information.  Permit new kinds of interaction. 26 Importance of HCI  A large proportion of design and programming effort is devoted to UI.  Good HCI design is critical to ◦ Success of products in marketplace ◦ Safety, usefulness and pleasure of using computer-based systems. 27 Importance of HCI  Good HCI design can decrease costs and increase productivity due to ◦ Fewer errors ◦ Reduced user disruption ◦ Decreased task time ◦ Reduced burden on support staff ◦ Reduced training time 28 Foundations of HCI  Computer science  Psychology  Sociology  Linguistics  Artificial intelligence 29 History of HCI  Direct manipulation of graphical objects.  Windows  Hypertext 30 Direct Manipulation of Graphical Objects  First demonstrated by Ivan Sutherland in Sketchpad ◦ Use light pen to manipulate objects.  Most of the direct manipulation techniques were developed in PARC (currently known as SRI Future Concepts Division) in 1970’s.  “Direct Manipulation” was first coined by Ben Shneiderman (University of Maryland). 31 Windows  Tiled windows ◦ Demonstrated by Douglas Engelbart in his NLS system. ◦ Example: Emacs.  Overlapping windows ◦ Proposed by Alan Kay ◦ Used in the Smalltalk system 32 Hypertext  Original idea is credited to Vannevar Bush  The term “hypertext” is first coined by Ted Nelson.  Ben Shneiderman’s Hyperties ◦ First system where highlighted items in the text could be selected to go to other pages. 33 Hypertext  Apple Hypercard further popularizes the idea to a wide audience.  Tim Berners-Lee’s adoption of hypertext in browser leads to spectacular growth of the World Wide Web. 34 HCI Applications  Mobile interface  Multimedia search engine  Education  Information visualization 35 Mobile Interface  Different interface design requirements for mobile devices such as smartphones and tablets.  Conflicting requirements ◦ Tailoring interfaces appropriate to the device. ◦ Offering a familiar interface so that users can quickly adapt to the mobile interface. 36 Mobile Interface 37 Multimedia Search Engine  In many cases, multimedia contents cannot be easily characterized by specific keywords.  Interfaces equipped with alternative querying mechanisms, such as example- based query specification, are required for effective search. 38 Multimedia Search Engine 39 Education  Adoption of direct manipulation techniques in interface design.  Design of effective learning environments, tools and media for distance learning.  Catering for different student communities ◦ Primary/secondary school students ◦ University students ◦ Further education ◦ On-the-job training 40 Education Chemistry Mathematics 41 Information Visualization  Presentation of large data sets in a suitable graphical formats for trend exploration.  Becomes more important due to information explosion brought on by the Web. 42 Google Map 43 Technological Trends  Wearable devices and ubiquitous computing.  Recognition-based user interfaces.  Conversational interface  3-D technologies. 44 Wearable Devices and Ubiquitous Computing  Wearable devices ◦ Clothing and accessories with computing capabilities.  Example: Apple Watch  Ubiquitous computing ◦ Embedding computers everywhere in our environment.  Example: Projects of MIT’s Tangible Media Group 45 Recognition-based User Interfaces  Current user interface mainly uses keyboard and mouse for input.  Natural user interface will become more and more important ◦ Speech ◦ Hand gestures ◦ Body movements 46 Example: Speech Recognition 47 Example: Gesture Recognition 48 Example: Face Detection 49 Recognition-based User Interface  Fundamental differences from today’s interfaces ◦ Sophisticated pattern recognition techniques are involved in the interface design. ◦ Interface with feedback facilities are required due to recognition error. 50 Conversational Interface  Interfaces which can respond intelligently to human conversations.  An example of a conversational interface. 51 Conversational Interface: Example He says I'm depressed much of the time. I AM SORRY TO HEAR YOU ARE DEPRESSED It's true. I am unhappy DO YOU THINK COMING HERE WILL HELP YOU NOT TO BE UNHAPPY I need some help, that much seems certain. WHAT WOULD IT MEAN TO YOU IF YOU GOT SOME HELP Perhaps I could learn to get along with my mother. TELL ME MORE ABOUT YOUR FAMILY My mother takes care of me. WHO ELSE IN YOUR FAMILY TAKES CARE OF YOU My father. 52 Conversational Interface  The input is inspected for the presence of a keyword.  When such a word is found ◦ The sentence is transformed according to a rule associated with the keyword. ◦ The text is then displayed.  Keywords may have a rank or precedence number. 53 Conversational Interface  A kind of template is applied to transform the original sentence.  Example: ◦ “I am very happy these days” ◦ “How long have you been very happy these days?”  Templates: ◦ “I am X” ◦ “How long have you been X ?” 54 Conversational Interface  Another example ◦ “It seems that you hate me” ◦ “What makes you think I hate you ?” (1) (2) (3) (4) “It seems that | you| hate |me” 55 Conversational Interface  Search for the pattern ◦ “(1) you (3) me”  Reassembly rule ◦ Throw away component (1). ◦ Apply the following rules:  you→I  me→you ◦ Tack on a relevant phrase to the front. 56 Conversational Interface  Formal notation ◦ “(1) you (3) me” ◦ “What makes you think I (3) you” 57 Conversational Interface  Speech recognition ◦ Conversion of sound waves to discrete words.  Natural language understanding ◦ Understand the meaning of the words.  Natural language generation ◦ Generate appropriate responses.  Speech synthesis ◦ Generation of natural-sounding speech. 58 3-D Technologies  Migration from 2-D presentation space to 3-D space.  Applications areas that benefit from 3-D technologies ◦ Training ◦ Simulation ◦ Interactive exploration of complex data environments. 59 Head-Mounted Display 60 Cave Automatic Virtual Environment (CAVE) 61 Data Glove 62 3-D Technologies  Previous 2-D notions need to be revised ◦ How to interact with information ? ◦ How to select objects in 3-D space ? ◦ How to navigate through information spaces ?  Objective: “near-real-time” interactivity to achieve the effect of direct manipulation 63