SCCVK2073 EVALUATION OF INTERACTIVE APPLICATION

Evaluation

• Evaluation is integral to the design process.
• It involves collecting and analyzing data about users' experiences.
• The goal is to improve the design of artifacts.
• Usability of the system is considered, along with user experience.

Evaluation Goals

• Usability Assessment
• User Experience (UX) Evaluation
• Identifying Design Issues
• Performance Evaluation
• Accessibility Assessment
• Feedback Collection
• Validation of Design Decisions
• Risk Mitigation

Types of Evaluation

• Formative Evaluation: Conducted during the early stages of development.
• Iterative throughout the development process, incorporating feedback into design.
• Summative Evaluation: Conducted after the product/system is developed and implemented.
• Used to assess effectiveness and impact of the product/system.

Evaluation Methods

• User Evaluation: User testing, observing users' interactions to assess effectiveness.
• Expert Evaluation: Heuristic evaluation or usability inspection; using expert evaluators to assess usability and user experience.

Qualitative Approach

• Qualitative evaluations use qualitative and naturalistic methods.
• Methods include in-depth, open-ended interviews, direct observation, and written documents.

Quantitative Approach

• Quantitative evaluation is used to measure and analyze numerical data.
• Methods include surveys and questionnaires, and one-to-one interviews.

Mixed-Method Approach

• Mixed methodology combines quantitative and qualitative data in a single study or series of studies.

Evaluation Ethics

• Strategies to protect the rights and dignity of participants are crucial.
• Safeguards are needed for children or vulnerable populations.
• Strategies include helping others, doing no harm, acting fairly, and respecting others.
• Ethical issues like informed consent, confidentiality, ensuring safety, and feedback collection should be considered.

DECIDE Framework

• A framework to guide evaluation efforts.
• Steps include:
  • Determining overall goals
  • Exploring specific questions to be answered
  • Choosing the evaluation paradigm and techniques
  • Identifying any practical issues (users, equipment, cost, time, etc)
  • Addressing ethical issues
  • Evaluating, interpreting, and presenting data

GOMS Framework

• Developed by Card, Moran, and Newell in 1983.
• A cognitive model for evaluating system performance.
• Components include Goals, Operators, Methods, and Selections (GOMS).

GOMS Family

• Different versions of GOMS framework exist.
• These include: Plain GOMS, KLM, NGOMSL, CPM-GOMS.

Norman's Cycle

• Also known as the Seven Stages of Action.
• A psychological model describing human interaction with computer systems.
• Can be used to help evaluate the efficiency of a user interface.

The Gulfs

• Execution: Distance between the user's intention and the available system actions.
• Evaluation: The difference between the user's perception of the system's state and desired outcome.

Expert Review

• Evaluators with expertise review the design.
• Focus on adherence to heuristics.
• Can be conducted early or late in the development process.
• Requires guiding questions and training for the reviewers.
• Goal is usability concerns.

Functional Testing

• A crucial step in ensuring software quality.
• Explores best practices for testing functionality across platforms.
• Covers various types of testing (unit, component, integration, regression, and user acceptance testing).
• Crucial for formative evaluation.

Defining Functional Requirements

• Description of a program's capabilities and behaviors.
• Key requirements are user workflows and features, expected inputs/outputs/responses.
• Ensuring requirements are measurable, testable, and aligned with users' goals.

Functional Testing for Different Platforms (Web, Desktop, Mobile, and Tablet)

• Detailed test plans for specific platform types, with specific examples of test components outlined.

Challenges in Functional Testing Across Platforms

• Handling device fragmentation, various user interactions, and effective automation.

Best Practices for Effective Functional Testing

• Prioritizing critical workflows, leveraging automation, and incorporating continuous integration into the testing process.
• Best practices for continuous improvement for functional testing.

Usability (UX) Evaluation

• Process to assess and understand how users interact with products or services.
• Involves gathering data on behaviors, attitudes, and preferences.
• Goal is to identify areas for improvement and enhance the user experience.

Types of UX Evaluation Methods

• Usability Testing
• Heuristic Evaluation
• Cognitive Walkthroughs
• User Interviews
• Surveys and Questionnaires
• A/B Testing

Usability Testing Methods (In-person vs. Remote)

• Details on types of usability testing methods, and their strengths and weaknesses.
• Tools and techniques used for both in-person and remote usability testing.

Participant Recruitment and Screening

• Strategies to ensure relevant and diverse participants are included in usability studies.
• Processes for selecting and identifying qualified participants.

Usability Evaluation Metrics (Measures for effectiveness, efficiency, satisfaction, and learnability.)

• Types of measures for each metric, and how to use them in evaluation.

Data Collection and Analysis (Usability)

• Methods for collecting usability data (observation, think-aloud protocols, and post-task interviews).
• Techniques for interpreting qualitative and quantitative feedback.

Reporting Findings and Recommendations (Usability)

• Process for summarizing evaluation results to identify issues and recommend enhancements.

Interaction Design - Evaluation Studies: From Controlled to Natural Settings

• Overview of usability testing objectives and types of studies (controlled and wild settings).

System Testing Evaluations

• System testing's role in ensuring the overall functionality and integration of a complex software system.

Defining System Testing

• Comprehensive Evaluation: Evaluating the entire system
• Integrated Approach: Considering end-to-end scenarios rather than isolated modules.
• Validation of Requirements: Ensures requirements are met and addressed.

Objectives of System Testing

• Functional Verification: Verifying system functions.
• Performance Optimization: Identifying and addressing performance bottlenecks.
• Compliance Validation: Checking compliance with standards.

Scope of System Testing in Formative Evaluation

• End-to-End Functionality, Non-Functional requirements, Integration with External Systems, and Compliance and Regulations.

Approaches to System Testing (Black-Box, White-Box, Agile).

• Overview of different testing methods.

Test Planning and Execution

• Steps for planning and executing test cases.

Analyzing System Test Results (Defect Identification, Performance Evaluation, and Acceptance Criteria)

• Detail on the phases of analyzing test results.

Incorporating Findings into Iterative Design

• Feedback collection followed by design refinement, and repeated for continual development improvement.

System Testing - Performance Testing

• A crucial aspect of software development, ensuring applications handle expected user loads and provide seamless user experience;
• The key principles and techniques used for conducting performance evaluations during the formative stage of an application's lifecycle.

Importance of Performance Testing in Testing in Formative Evaluation

• How to identify bottlenecks during formative stages.
• Optimizing user experience with testing and feedback.
• Mitigating risk by identifying potential issues before launch.

Key Performance Metrics to Measure

• Response Time, Throughput, and Resource Utilization

Selecting Appropriate Testing Techniques (Load Testing, Stress Testing, Endurance Testing, Spike Testing)

• Techniques to choose and use for specific conditions.

Designing Effective Test Scenarios

• Identifying user profiles, defining user journeys, and incorporating realistic data.

Conducting Performance Tests in an Interactive Environment

• Steps in conducting performance testing.

Analyzing and Interpreting Test Results

• Data aggregation, performance analytics, and reporting to stakeholders.

Incorporating Findings into Application Improvements

• Prioritizing findings, implementing optimizations, validating improvements and iterative processes.

System Testing - Exception Handling

• Exception handling as a critical process in formative system testing.

Importance of Exception Handling During Formative Evaluation

• Identifying vulnerabilities during early testing.
• Validating resilience of the system, and improving user experience.

Identifying Potential Exceptions in the System

• Identifying potential system risks and vulnerabilities from various perspectives: input validation, resource constraints, and dependencies.

Designing Test Cases for Exception Scenarios

• Strategies, including boundary condition, simulated failures and error propagation testing for exception scenarios.

Implementing Exception Handling Mechanisms

• Strategies for implementing and testing mechanisms for fault tolerance and exception handling.

Verifying the Effectiveness of Exception Handling

• Methods and metrics used to validate exception handling.

Analyzing and Reporting Exception Handling Outcomes

• Analyzing the collected data, and the process of reporting results and solutions.

Continuous Improvement of Exception Handling Strategies

• Steps in implementing continuous improvement processes.

System Testing - Security Testing

• Overview of security testing in software development.

Importance of Security Testing in Formative Evaluation

• Early detection of security vulnerabilities.
• Proactive strategy approach implemented in early testing phases of the product.
• Continuous improvement to ensure security practices in development cycles.

Common Security Vulnerabilities (Injection Flaws, Broken Authentication, Cross-Site Scripting, and Sensitive Data Exposure)

• Descriptions and examples of common vulnerability types.

Threat Modeling and Risk Assessment

• Processes for identifying and analyzing potential threats, and prioritizing security efforts.
• Approach to making security decisions backed with structured approaches.

Security Testing Methodologies (Reconnaissance, Vulnerability Identification, Exploitation, and Reporting)

• Processes in security testing from planning to reporting.

Automated Security Testing Tools (Burp Suite, OWASP ZAP, Nmap, SQLmap)

• Software tools for automated security testing.

Integrating Security Testing into the Development Lifecycle

• Strategies for embedding security testing throughout the software development lifecycle.

Reporting and Remediation of Security Vulnerabilities

• Process for documenting security issues and recommending solutions.

Introduction to Usability Evaluation

• Focus on user-friendliness and effectiveness.

Defining Usability

• Overview of usability and its key aspects (effectiveness, efficiency, satisfaction, and learnability).

Importance of Usability Evaluation

• Key benefits of conducting usability evaluation.

Usability Techniques (User Evaluation, and Expert Evaluation)

• Methods overview of analyzing product/application quality.

Types of Usability Evaluation

• Overview of different usability evaluation strategies.

Usability Testing Methods (Observation, Think-Aloud Protocol, Interviews, and Remote Usability Testing)

• Specific usability techniques and approaches.

Participant Recruitment and Screening

• Specific techniques used to gather relevant user data.

Usability Evaluation Metrics

• Different measures of usability, including effectiveness, efficiency, satisfaction, and learnability.

Data Collection and Analysis (Usability)

• Different methods for data collection and analysis during usability evaluation

Reporting Findings and Recommendations (Usability)

• Strategies for summarizing evaluation and recommending design enhancements.