Windowing Systems & UIMS Concepts

Chapter 8: Implementation Support

• Programming tools provide various levels of support for programmers, including essential functions like windowing systems.
• Windowing systems allow programmers to create applications that function independently of specific hardware devices, a significant aspect of portability.
• Core support is given to handle separate and simultaneous user-system activities.
• Programming tools facilitate the creation of applications that interact properly with users.
• Interaction toolkits bring programming closer to the user's perception level.
• User interface management systems (UIMS) handle the relationship between presentation and functionality.

Introduction

• Previous chapters focused on the abstract design and analysis of interactive systems.
• This chapter focuses on the task of coding the interactive application and its structure.
• Detailed specifications provide instructions for what the interactive application should do.
• Programmers translate the specifications into executable machine instructions, determining how the application will function on the available hardware.

Quest of this Lecture

• This section discusses how human-computer interaction (HCI) impacts programmers.
• Advances in coding have shifted the focus from hardware-specific programming to interaction-technique-specific programming.
• Layers of development tools are critical for programmers. These tools often include windowing systems, interaction toolkits, and user interface management systems.

Elements of Windowing Systems

• A key function of windowing systems is programmer independence from specific hardware.
• A typical graphical workstation uses a visual display screen, a keyboard and often mouse, or other pointing device.

Windowing Systems

• Creating applications that run on a wide variety of devices is crucial.
• Programmers want to interact with an abstract terminal that employs a general language. This language can then be translated for use on many different devices.
• A single translation program (or device driver) is needed for each type of hardware device. Then any application program can access that device.

Elements of Windowing Systems (continued)

• Windowing systems provide programmer independence from hardware details.
• Typical components are visual display, keyboard, and pointing devices (e.g., mouse).

Windowing Systems (continued)

• Programming applications for a wide range of devices requires a standardized abstraction layer.
• The ideal abstract terminal understands a generalized language and can be converted into a specific device's particular code.
• A critical aspect of this is each device having a specialized driver program.

Windowing Systems (continued 2)

• Windowing systems help simplify programming and improve portability.
• Abstract terminals support various operating systems and hardware.
• Device drivers act as an intermediary between the operating system and specific hardware devices.

Imaging Models

• Windowing systems generally use a fixed language for abstraction (named an "imaging model").
• Imaging models are sufficient to describe any image
• Specific primitives are often used for efficiency reasons, such as handling text images as simple pixel images or as more generalized font definitions.

Examples of Imaging Models

• Pixels: Representing the screen as a matrix of points that can be either on or off or assigned a specific color.
• Graphical Kernel System (GKS): An international standard, the screen is treated as a collection of lines and polygons.
• Programmer's Hierarchical Interface to Graphics (PHIGS): An international standard that expands on GKS by allowing editable segments.

Examples of Imaging Models (continued)

• PostScript: A programming language supporting the creation of graphic objects that can be drawn and used for complex images.

Imaging Models (continued)

• Though initially limited to output, these abstract models can serve for some input tasks as well. For instance, pixel coordinate systems can be used to interpret mouse movements.

Elements of Windowing Systems (continued 3)

• Systems process input events (mouse clicks, keyboard presses) within the context of the rendered image.
• This input event should relate to objects shown within the image.
• These processes use abstract terminals that isolate applications from specific hardware details.

Sharing

• Multi-user windowing systems handle concurrent user tasks.
• All users simultaneously share the resources of a single computer.
• Each terminal acts as an independent process.
• Coordination of multiple tasks must be handled by the windowing system.

Sharing (2)

• This streamlining of interaction tasks allows for easier programming in a multi-user enviroment.

Sharing (3)

• Window systems must provide a mechanism to display the separate applications.
• Window systems typically accomplish this by dividing the display into regions allocated to separate processes.

Introduction

• The chapter examines the impact of HCI on programming.
• Advances in coding have led to programming that is more hardware independent and interaction-technique specific.
• Programming support now includes windowing, toolkits and UIMS (User Interface Management Systems).

Elements of Windowing Systems (continued 4)

• Device independence means that application programs are isolated from the details of the specific output devices. (e.g. Printers, Screens, etc.)
• Standard input/output models allow portability across different output devices.
• Common methods for handling images include pixels, PostScript, and graphical kernel systems.

Roles of a Windowing System

• A windowing system acts as a mediator and facilitator between application programs and the multiple hardware devices they use, like keyboards, screens and mice.
• This architecture enables several application programs to share common resources efficiently.

Architectures of Windowing Systems

• Possible software architectures exist with different approaches to multiple-user application management.
• Each application managing its own processes reduces portability of those programs.
• Application management within an operating system kernel promotes program portability, though potential synchronization issues may occur.
• Dedicated application management systems improve portability at the cost of additional complexity within the overall system.

The Client-Server Architecture

• Application Programs interface with an abstract terminal, which interfaces with the Resource Manager, and ultimately the hardware devices.
• Resource Manager tracks usage, and distributes the access to resources such as screens to different clients accessing the terminal.
• Device drivers provide the link between the operating system and specific hardware components.

X Window System

• X Window is industry-standard, and is a classic example of a windowing architecture using client-server implementation.
• It was developed at MIT in the mid-1980s.
• It's based on a pixel-based imaging model.

X Windows (continued)

• X is built on a network protocol specifying the server-client interaction.
• X allows implementation on many different computers and operating systems, achieving portability.

X Windows (continued 2)

• The X Server handles tasks like managing access to the display, interacting with clients, performing actions on the screen, routing events, and directing input events to particular clients.

X Windows Architecture

• X11 server manages communications between applications and the display.
• Client applications (and graphical objects) interact with this to display and manipulate information on screen.
• Device drivers are part of this layer.

X Windows Architecture (continued)

• Pixel imaging model and pointing mechanisms create a user-friendly experience.
• The X protocol manages the communication between the client and server, ensuring interaction consistency.
• Separate window managers control input and output operations and interactions such as how input focus changes and how overlapping windows work.

Programming the Application

• This section focuses on the programming of interactive applications within a client-server architecture. This is the code that controls the specific actions within the "application."
• Applications respond to user input to determine their actions.
• Interactions generally are user-driven, where the actions of the application are determined by inputs from the user.

Read Evaluation Loop

• A key element in interactive programming is the "Read-Evaluation-Loop," where the application responds to input events directed to it.
• The server sends structured input events to the application.
• The application determines how to respond and carry out the action.
• This section details and illustrates the program loop.
• Event handling is a key aspect of this loop.

Programming the Application - Notification-Based

• The main event-processing control loop is separate from the application's code.
• A central notifier is in charge of routing input/output events to the appropriate application.
• Applications register callbacks with this notifier, indicating which actions to execute when specific events are triggered.

Going with the Grain

• Programming styles (modal vs non-modal, direct event-loop handling) should be designed.
• Implementation details should not dictate design decisions.
• Modal vs. non-modal application styles should be considered when designing an interactive application.

Using Toolkits

• Toolkits provide programming abstractions for interaction objects (buttons, menus, etc.).
• They improve coding consistency, allowing creation of similar looking programs.
• Object-oriented programming techniques are easily applied.
• Tools provide generalized input/output object handling.
• The use of toolkits creates a consistent look and feel that reinforces familiarity across different parts of a user interface.

Using Tool Kits (continued)

• WIMP-type interfaces create the impression to users that application objects on the screen are independent entities that can be used directly.
• User input and output activities are directly associated with these objects.
• A user interacting with the mouse "moves" the cursor on screen.
• The link between the mouse input and the cursor's movement on the screen is an example created by toolkits.

Interfaces in Java

• Java provides AWT (Abstract Windowing Toolkit) for handling basic UI elements like buttons, menus and event management.
• Higher-level tools like Swing are built on top of AWT, providing more advanced features.

User Interface Management Systems (UIMS)

• UIMS are extensions of UI toolkits aiming to further separate application tasks from display and input/output tasks.
• UIMS address problems with UI toolkits like buttons, menus and other graphic components.
• Problems include complex tasks like interaction design. UIMS are an intermediate step.
• UIMS provide a structured environment for designers.
• They offer a conceptual framework that clarifies the relationship between application and presentation. Separation of these tasks simplifies maintenance, and improves program portability.

UIMS as Conceptual Architecture

• UIMS offer a clear division between application semantics and presentation.
• This separation leads to improved portability because applications run on various systems, components are reusable
• Reduced costs, better maintainability and potential customization by designers and end-users.
• Separating graphical objects from the actual software "engine" of the program improves maintainability and allows easier application changes.

UIMS Tradition

• This section clarifies typical components and their organization within a UIMS-style architecture.

Seeheim Model

• The Seeheim model describes a conceptual architecture composed of lexical, syntactic and semantic components.
• The presentation, dialogue control and application interface components communicate and work together, but are architecturally separate.

Conceptual vs. Implementation

• The Seeheim design is a separation of tasks, useful for understanding, but not necessarily practical in every implementation.
• This is critical – design should match implementation – otherwise programming will be awkward.

Semantic Feedback

• The amount of feedback given depends on the nature (lexical or syntactic, or semantic) of the event being processed. It is more complex, so feedback is slower for this kind of interaction.

Bypass/Switch

• This technique allows for specific, but controlled, interaction with semantic objects.

More Layers!

• This section clarifies that a conceptual architecture can require multiple layers, potentially increasing overall complexity to enable interaction with specific components. (e.g., device presentation layer)

Arch/Slinky

• Slinky layered approach adds more layers, especially useful for breaking down complex visual and user interactions. It further separates presentation from the operating application.

Monolithic vs. Components

• Seeheim has several large components.
• MVC (model–view–controller) architectures in languages like Smalltalk are examples of components design.
• Separating elements (model, view and controller) improves application design, maintenance and re-use.

MVC Architecture

• MVC architecture is a software design pattern that divides an application into three interrelated parts: model, view, and controller.
• The model represents the internal data and behavior of the component, the view renders this data on the screen and the controller intercepts input events and updates the model & view accordingly.

MVC Issues

• MVC interactions can be complicated; input events only have meaning if related to output event display. (e.g. mouse-click needs context to determine to what output/object on screen the input relates).
• Controllers are often not completely segregated and need to communicate with the view, potentially complicating the design.

PAC Model

• PAC model is an alternative approach that handles interaction design, that can be used to handle data display that more closely resembles the Seeheim design.
• The PAC approach clarifies object state, and supports a better separation of tasks than MVC for user interface development.
• The PAC model clarifies tasks and their interaction.

Implementation of UIMS

• Various techniques are available for creating the dialog with a UIMS architecture.
• Implementing tools such as the "constraints" in a UIMS architecture can be challenging but is worth the effort because the user interaction is easier to understand and tailor to user needs.

Implementation Techniques (continued)

• Menu networks, grammar notations, state transition diagrams, event languages, declarative languages, constraints and graphical specifications are used to create or process a dialog. (These are not necessarily mutually exclusive).

Implementation Consideration

• Window systems (like X Window) promote separability between application functionality, presentation and dialog controllers.
• Application interface callbacks to a notifier allow input events to be directed to the correct part of application code.
• This means components can be implemented independently even if the overall system (application) is complex.

Summary

• Windowing systems, different program designs (read-evaluation vs notification-based), toolkits, and UIMS (User Interface Management Systems) contribute to efficient user interface design.

Explore UIMS vs. interaction toolkits, portability in windowing systems, device independence, and client-server architecture for resource sharing. Learn about the window manager client in X Windows and the read-evaluation loop in UI application programming. Delve into imaging models and UIMS architecture.