1.13

SPECFP Benchmarking Information

• SPECFP measurements typically increase by 15% to 30% compared to base measurements.
• The benchmark was updated in 1995 to include new integer and floating-point benchmarks and remove flawed ones.
• SPEC95 adopted Sun SPARC Station 10/40 as the normalization base due to difficulty in finding original computers.
• SPEC2017 is the most recent version of SPEC benchmarks.
• 82 programs were necessary for SPEC CPU across six generations; 57% of these were unique to a single generation.
• Only six programs (7%) have persisted for three or more generations, with the gcc compiler being the only survivor from SPEC89.
• Additional benchmark suites for various performance areas have been introduced, including graphics and scientific computing since 2008.

Growth of Embedded Computing

• Embedded processors have existed since the development of minicomputers and microprocessors.
• Historical dominant use was in industrial control, featuring low-cost and low-performance processors.
• An 8-bit microcontroller remains the best-selling processor, utilized in cars and home appliances.
• The late 1980s and early 1990s saw expansion into advanced video games, set-top boxes, cell phones, and PDAs.
• Growing networking and information appliance demands triggered an increase in embedded processors and performance requirements.
• Embedded Microprocessor Benchmark Consortium (EEMBC) was established in 1997 to create benchmarks tailored for embedded systems.
• EEMBC released its second-generation benchmarks in 2007; in 2019, Embench was developed to provide free alternative benchmarks.

Historical Perspective on Computing Progress

• Over 50 years, computers have evolved significantly in technology and capabilities.
• Price/performance improved by nearly 100 billion due to inflation adjustments, equating to 58% per year.
• Notable factors include a 10,000 improvement in cost and a 10 million improvement in performance.
• Characterizing performance often relies on kernels, particularly in scientific applications, although they can overstate real application performance.

SPEC and Performance Evaluation Development

• Established in 1988, the System Performance Evaluation Cooperative (SPEC) includes representatives from major computer companies.
• SPEC standardized a set of real programs and inputs for consistent performance evaluation.
• The cooperative emerged during an era of portable operating systems and high-level languages, allowing for meaningful comparisons.
• SPEC benchmarks have evolved to evaluate multiple systems and include OS and I/O activities as factors.
• SPEC92 introduced critical updates, including separate measures for integer and floating-point programs (SPEC INT and SPECFP).
• SPEC base measures, which exclude program-specific optimizations, allow users to have a realistic measure of performance.

Benchmarking Techniques

• Initial benchmarking faced challenges due to diverse operating systems and programming languages, making standardization difficult.
• Synthetic programs like Whetstone, derived from scientific algorithms, gained popularity for benchmarking.
• Kernel benchmarks, which consist of small, time-intensive code snippets from real programs, became a method to assess high-end computing performance.
• Livermore Loops and Linpack serve as prime examples of kernel benchmarks that isolate individual computer features for performance analysis.

This quiz explores the evolution of SPECFP benchmarks and their significance in measuring computer performance. It covers updates made in 1995, including the introduction of new benchmarks and the changes in normalization based on the Sun SPARC Station 1. Test your knowledge on how these benchmarks have impacted performance assessments over the years.