78 Questions
What was Paul Meehl's profession?
Psychologist
In which departments did Paul Meehl have faculty appointments at the University of Minnesota?
Psychology, law, psychiatry, neurology, and philosophy
What was the topic of Paul Meehl's 'disturbing little book'?
Clinical vs. Statistical Prediction: A Theoretical Analysis and a Review of the Evidence
What was the purpose of the 20 studies reviewed in Meehl's book?
To compare the accuracy of clinical and statistical predictions
What type of information did the statistical algorithm use in the study on predicting freshmen grades?
High school grades and one aptitude test
How many of the 14 counselors' predictions were less accurate than the statistical algorithm's prediction in the study on freshmen grades?
11
What was the outcome of Meehl's book on the field of clinical psychology?
It sparked a controversy and a stream of research
How long has the controversy started by Meehl's book continued to engender research?
More than 50 years
What is the approximate number of studies comparing clinical and statistical predictions?
Roughly 200 studies
What is the title of Robyn Dawes's famous article?
The Robust Beauty of Improper Linear Models in Decision Making
What is the logic of multiple regression?
It finds the optimal formula for putting together a weighted combination of the predictors.
What percentage of studies have shown better accuracy for algorithms?
Approximately 60% of studies
What is the advantage of using statistical rules over expert judgment?
They are less expensive
What did Dawes observe about multiple regression?
It adds little or no value to the predictive power of simple models.
What is the term used to describe the domains where algorithms have been shown to be more accurate?
Low-validity environments
What is the implication of Dawes' research?
It is possible to develop useful algorithms without prior statistical research.
What is the topic of Orley Ashenfelter's research?
The future value of fine Bordeaux wines
What is the formula that predicts marital stability?
Frequency of lovemaking minus frequency of quarrels.
Who developed the simple algorithm that has saved the lives of hundreds of thousands of infants?
Virginia Apgar
What three features of the weather does Ashenfelter's formula use to predict the price of a wine?
Average temperature, amount of rain at harvest-time, and total rainfall during the previous winter
Why are experts often inferior to algorithms in making predictions?
Experts try to be clever and consider complex combinations of features
What did Virginia Apgar use to assess the health of newborn infants?
Five variables and three scores.
What is the purpose of the Apgar score?
To determine which babies are in need of immediate intervention.
What is the correlation between Ashenfelter's predictions and actual prices?
Above .90
What is the title of Atul Gawande's book that discusses the virtues of checklists and simple rules?
A Checklist Manifesto
What is the advantage of using algorithms in making predictions?
Algorithms are more accurate
What is the name of the economist who demonstrated the power of simple statistics to outdo world-renowned experts?
Orley Ashenfelter
What is the general conclusion of the research on predictive models?
Simple models are often as accurate as complex models.
What is a key advantage of using formulas over human judgment?
Formulas are consistent and do not suffer from context dependency
What is the term coined by Meehl to describe a rare and decisive event that can override a formula?
The broken-leg rule
What is the approximate percentage of inconsistency found in experienced radiologists when evaluating the same chest X-rays on separate occasions?
20%
What is the main reason for the inferiority of human judgment compared to formulas?
Humans are inconsistent in making summary judgments
What is the likely effect of conducting an interview on the accuracy of a selection procedure in medical school admissions?
It decreases the accuracy of the selection procedure
Why do formulas not suffer from the problems of human judgment?
Formulas always return the same answer given the same input
What is the main reason for the context dependency of human judgment?
The influence of unnoticed stimuli in our environment
What is the conclusion suggested by the research reviewed by Meehl and his followers?
Formulas are superior to human judgment in most cases
What was the reaction of French wine circles to the use of algorithms?
Somewhere between violent and hysterical
What is the likely effect of wine experts' ability to taste the wine on the accuracy of their predictions?
It decreases the accuracy of their predictions
What is the primary concern of clinicians when it comes to algorithmic decisions?
The possibility of a treatable case being denied treatment due to a misclassification
What is the primary reason why humans are unable to maintain the consistency of a formula?
Humans are prone to inconsistencies in their judgment
What is the author's perspective on the use of algorithms in decision-making?
Algorithms can be useful in certain decision-making situations, but not all
What is the expected outcome of the increasing use of algorithms in everyday life?
A softened hostility towards the use of algorithms
What is the primary benefit of using algorithms in decision-making, according to Meehl and other proponents?
They make fewer mistakes than human judgment
What is the author's role in the Israeli Defense Forces?
All of the above
What was the goal of the interview system in the Israeli Defense Forces?
All of the above
What was the problem with the original interview procedure in the Israeli Defense Forces?
It was almost useless for predicting the future success of recruits
What was the author's task in the Israeli Defense Forces?
To design a new interview procedure
What was the author's qualification for being assigned to design a new interview procedure?
A degree in psychology
What was the initial response of clinical psychologists to Meehl's ideas?
Hostility and disbelief
What type of predictions do clinicians typically make in therapy sessions?
Short-term predictions in the context of the therapeutic interview
Why do clinicians fail to make accurate long-term predictions?
Because they have never had the opportunity to learn properly
What is the moral dimension of the debate about clinical and statistical prediction?
The debate is about the role of human judgment versus mechanical combination of variables
Why do people prefer the 'all natural' option, even when informed that it has the same nutritional value and health benefits as the commercially grown option?
Because they have a strong preference for the natural over the synthetic or artificial
What is the reason for the resistance to algorithms making decisions that affect humans?
Because humans have a strong preference for the natural over the synthetic or artificial
What was the reaction of the European wine community to Ashenfelter's formula for predicting the price of Bordeaux wines?
They were resistant to the formula, despite its accuracy
What is the difference between the tasks that clinicians are skilled at and the tasks that they are not good at?
The tasks that clinicians are skilled at are short-term predictions, while the tasks they are not good at are long-term predictions
What is the reason for the illusion of clinical skill among clinicians?
Because they have many hunches that are confirmed during therapy sessions
What is the criticism of the statistical method by experienced clinicians?
That it is mechanical, atomistic, additive, and artificial
What is the key lesson the author learned from the finding?
Intuition adds value only after a disciplined collection of objective information and scoring of separate traits.
What is the purpose of telling the candidates 'Close your eyes' during the interview process?
To add a weighted score to the candidates' overall evaluation.
What is the recommended number of traits to select for a job interview?
6
Why should the information on each trait be collected separately?
To avoid halo effects.
What should the interviewer do to resist the temptation to change the ranking of the candidates?
Firmly resolve to hire the candidate with the highest score.
What is the result of using the interview procedure described in the chapter?
A significant difference in the quality of the people hired.
What should the interviewer do before conducting the interview?
Select a few traits that are prerequisites for success in the position.
What is the benefit of using a disciplined scoring system in the interview process?
It increases the chances of hiring the best candidate.
Why should the interviewer avoid making an overall intuitive judgment about the candidates?
Because it is unlikely to result in the best candidate being hired.
What is the recommended duration for preparing the interview procedure?
Half an hour or so
What was the main reason for the failure of the original interview process?
The interviewers were allowed to focus on the dynamics of the interviewee's mental life
What was the main idea behind the new interview process?
To obtain specific information about the interviewee's life in their normal environment
How did the author combat the halo effect?
By using standardized, factual questions
What was the task of the interviewers in the new interview process?
To elicit relevant facts about the interviewee's past and score each personality dimension
What was the outcome of the new interview process?
The new interview process was moderately useful
What was the result of the 'close your eyes' exercise?
The interviewers' intuitive judgments were as accurate as the sum of the six ratings
What was the main reason for the interviewers' initial resistance to the new interview process?
They felt they were being turned into robots
How many personality traits were evaluated in the new interview process?
6
What was the sequence of the traits evaluation in the new interview process?
The interviewers evaluated the traits in a fixed sequence
What was the purpose of the final score in the new interview process?
To determine the interviewee's fitness for combat duty
Study Notes
Paul Meehl and Clinical vs. Statistical Prediction
- Paul Meehl was a psychologist who wrote a book titled "Clinical vs. Statistical Prediction: A Theoretical Analysis and a Review of the Evidence" in 1954, which challenged the idea that clinical predictions made by trained professionals are more accurate than statistical predictions.
- Meehl reviewed 20 studies that compared clinical predictions with statistical predictions and found that statistical predictions were more accurate in 11 out of 14 cases.
- The book provoked shock and disbelief among clinical psychologists, but it started a stream of research that has continued to this day, with over 200 studies conducted since then.
The Superiority of Algorithms
- Studies have consistently shown that algorithms outperform human experts in making predictions, with algorithms being more accurate in about 60% of cases.
- Algorithms are better than humans because they are not affected by biases, emotions, and other factors that can influence human judgment.
- Humans are also inconsistent in making summary judgments of complex information, which can lead to inaccuracies.
The Apgar Score
- The Apgar score is a simple algorithm developed by Virginia Apgar in 1953 to assess the health of newborn babies.
- The score is based on five variables: heart rate, respiration, reflex, muscle tone, and color, with a score of 0, 1, or 2 for each variable.
- The Apgar score has been credited with reducing infant mortality and is still used in every delivery room today.
The Hostility to Algorithms
- Many people, including clinicians, have been resistant to the idea of using algorithms to make predictions, preferring to rely on human judgment.
- This resistance is often due to a prejudice against the use of machines and algorithms, and a preference for the "natural" and "human" approach.
- However, the use of algorithms can lead to more accurate and fair decisions, and can also help to reduce errors and biases.
The Importance of Algorithms
- Algorithms are becoming increasingly important in our daily lives, and are being used in a wide range of applications, from medicine to finance to sport.
- The use of algorithms can help to improve decision-making, reduce errors, and increase efficiency.
- As algorithms become more widespread, it is likely that the resistance to their use will decrease, and they will become more accepted as a valuable tool.### The Interview Process
- The author was tasked with designing a new interview process to assess recruits for combat duty in the army.
- The previous interview process was deemed "almost useless" in predicting future success of recruits.
- The author was inspired by Paul Meehl's book, which argued that statistical rules are superior to intuitive judgments.
The New Interview Process
- The author designed a new interview process that focused on obtaining specific, factual information about the recruit's life.
- The process involved evaluating six relevant personality traits, including responsibility, sociability, and masculine pride.
- The interviewers were instructed to ask standardized, factual questions and score each trait on a 5-point scale.
- The interviewers were also instructed to go through the six traits in a fixed sequence to avoid halo effects.
Results and Lessons Learned
- The new interview process was a significant improvement over the previous one, with the sum of the six ratings predicting soldiers' performance more accurately.
- The intuitive judgment of the interviewers also performed well, but only after a disciplined collection of objective information.
- The author learned that intuition adds value, but only when combined with a disciplined approach.
- The author also learned not to simply trust intuitive judgment, but not to dismiss it either.
Applying the Lessons to Other Areas
- The message of the chapter can be applied to tasks other than making manpower decisions for an army.
- Implementing interview procedures in the spirit of Meehl and Dawes requires relatively little effort but substantial discipline.
- The author provides an example of how to apply the lessons to hiring a sales representative, including selecting a few traits, making a list of factual questions, and scoring each trait on a 5-point scale.
Learn about the life and work of Paul Meehl, a renowned psychologist who made significant contributions to various fields, including psychology, law, and philosophy. Take this quiz to discover more about his research and interests.
Make Your Own Quizzes and Flashcards
Convert your notes into interactive study material.
Get started for free
