Performance Analysis Lecture 2

Applying techniques like measurements and modeling to assess system performance, considering factors like delay, response time, and throughput.

Criteria used to evaluate system performance.

A collection of hardware, software, and firmware.

Requests made by users of a system.

A part of a system that limits its overall performance.

Determining the appropriate resources to provide a desired level of service.

Evaluating the relative performance of different systems, algorithms, or protocols.

Systems where the output depends on current and past inputs, but not on the time itself.

Systems where the output depends on current, past inputs and current time.

Systems with an 'outside', uncontrollable world that might impact workload, failures, or configurations.

Systems where everything is within complete control.

Systems where at least one input or internal state is random, leading to random outputs.

Systems with predictable behavior, where outputs are determined by inputs.

Systems where state changes can happen at any time.

Systems where state changes occur only at specific points in time.

Systems with an uncountable set of possible states.

Systems with a finite or countably infinite set of possible states.

Measures how well an application runs, a subjective and application-dependent metric.

The time it takes to complete a task or request.

The rate at which requests are processed per unit of time.

Maximum throughput achievable under ideal workload.

Maximum throughput that achieves a pre-set response-time limit.

Ratio of usable capacity to nominal capacity.

Fraction of time a resource is busy servicing requests.

Probability of errors or mean time between errors.

Measures how well a network delivers packets, considering time, resource use, and variability of response time.

The delay inside the network for a packet to be delivered.

The number of packets delivered per unit of time.

Packets arriving in a sequence different from the intended order.

Packets that are repeated in the network.

Packets that fail to reach their destination.

Methods for assessing system performance, including direct measurements and theoretical modeling.

Performance models that use mathematical methods and notations.

Computer programs that mimic the behavior of a system, used when the system is complex or unavailable.

A computer program that simplifies a system to study its important aspects. It uses a simplified representation of the system.

A simplified mathematical representation of a system, used to understand system behavior.

Simulation allows for a deeper, more complete understanding of how a system functions.

Simulation models can often be set up and tested relatively quickly, unlike real-world experiments.

Even approximate simulation models can provide useful insights into a system's behavior.

Simulations can take a long time, especially when needing high accuracy results.

Setting up and verifying the accuracy of a simulation model takes time and effort.

Simulation results are often much easier to reproduce than measurement results from experiments.

Simulation has complete control over all parameters, which helps in understanding cause-and-effect relationships.

Without well-defined goals, a simulation is ineffective and unlikely to provide useful insights. Describe what you want to know about the system, then build your experiment.

Don't setup simulation experiments to unfairly show that your system is better than some other system.

The workload in a simulation should match real-world conditions to get accurate results. Include various scenarios (ex: small and large packets).

Use the right method! Choose the best evaluation technique (analytical model, simulation, measurement) appropriate to the questions you are asking.

Simulations can go wrong by having either too much or too little detail. The best balance needs to be created to solve the problem at hand.

A model used to understand congested router performance, but doesn't have sensitivity analysis for result accuracy

Three common errors in presentation and analysis: improper presentation of results, omitting assumptions/limitations, and not accounting for varying traffic types like TCP and UDP.

A structured process for evaluating system performance: Define goals, select metrics, list parameters, select factors/values, evaluation techniques, workloads, design experiments, analyze data, and present results.

rpipes (remote pipes) allow non-blocking remote operations while rpc (remote procedure calls) require blocking until the call returns.

The client, server, and network components involved in the evaluation. Only the parts handling the communication channel are part of the system.

Two common types of services studied: small data transfers and large data transfers.

Focus on operational correctness: call elapsed time, maximum call rate, local CPU time, remote CPU time, and bytes sent per call.

System factors that affect performance, including CPU speed, network speed/reliability, OS overhead. Consider workload parameters as well.

Crucial factors in the evaluation: type of channel (rpipe/rpc), network speed (short/long), size of data, and number of calls.

Using measurements of the system and analytic modeling to estimate performance for scenarios outside of direct experiments.

A synthetic program that creates specific channel requests for evaluation. Resources consumed are also logged.

A full factorial approach to vary all key factors to capture their performance effects.

Using analysis of variance to determine how different factors (network speed, channel type) affect performance in the testing of a system.