Podcast
Questions and Answers
Why was a 5-point scale implemented in the C-CG, instead of the original scale?
Why was a 5-point scale implemented in the C-CG, instead of the original scale?
- To align with international teaching communication skill standards.
- Because the original scale was not validated in translated versions.
- To exclude physical examinations from the basic skill course.
- Because raters felt a larger selection of ratings would make assessment easier. (correct)
What was the primary goal of the study regarding the short version of the C-CG?
What was the primary goal of the study regarding the short version of the C-CG?
- To translate the C-CG into multiple languages.
- To compare the C-CG with other communication assessment tools.
- To introduce the C-CG in basic medical skills courses.
- To analyse its psychometric properties and determine its suitability for assessing communication skills in undergraduate medical students. (correct)
Which of the following is NOT one of the aspects specifically studied in detail regarding the C-CG?
Which of the following is NOT one of the aspects specifically studied in detail regarding the C-CG?
- Test-retest reliability.
- Item distribution.
- Construct validity.
- Inter-rater reliability. (correct)
How were the videos selected to represent the range of student performance?
How were the videos selected to represent the range of student performance?
Who participated as raters in the study?
Who participated as raters in the study?
What did the training session for the raters include?
What did the training session for the raters include?
What is the significance of the C-CG's acceptance within an international leading declaration?
What is the significance of the C-CG's acceptance within an international leading declaration?
From the initial pool, how many videos were selected for the study, and what determined the selection?
From the initial pool, how many videos were selected for the study, and what determined the selection?
What was the primary goal of discussing individual C-CG ratings item by item within the group after the initial video consultation?
What was the primary goal of discussing individual C-CG ratings item by item within the group after the initial video consultation?
How did the researchers ensure the raters completed the video scoring within the designated timeframe?
How did the researchers ensure the raters completed the video scoring within the designated timeframe?
What statistical measure was used to determine if there was a significant difference between the two assessments?
What statistical measure was used to determine if there was a significant difference between the two assessments?
Which statistical method was employed to reduce the number of variables to a smaller set of factors, thereby simplifying the data structure?
Which statistical method was employed to reduce the number of variables to a smaller set of factors, thereby simplifying the data structure?
Which aspect of the 'basic clinical skills course' is directly assessed using simulated patients?
Which aspect of the 'basic clinical skills course' is directly assessed using simulated patients?
What statistical software was used to conduct the psychometric analysis of the Calgary Cambridge Guides (C-CG)?
What statistical software was used to conduct the psychometric analysis of the Calgary Cambridge Guides (C-CG)?
What threshold was used to determine the statistical significance of the p-value when comparing the two assessments?
What threshold was used to determine the statistical significance of the p-value when comparing the two assessments?
Which type of reliability was assessed using statistical measures like Pearson's r and a t-test over a three-month interval?
Which type of reliability was assessed using statistical measures like Pearson's r and a t-test over a three-month interval?
According to the factor analysis, which of the following communication aspects is represented by one of the four identified factors?
According to the factor analysis, which of the following communication aspects is represented by one of the four identified factors?
In the factor analysis, varimax rotation was used. What is the primary purpose of applying varimax rotation in factor analysis?
In the factor analysis, varimax rotation was used. What is the primary purpose of applying varimax rotation in factor analysis?
In the context of the study, what does 'inter-rater reliability' specifically refer to?
In the context of the study, what does 'inter-rater reliability' specifically refer to?
What ethical consideration was addressed in the study design?
What ethical consideration was addressed in the study design?
In the context of the study, what does a statistically significant t-test result (p = 0.023) between the first and second assessments suggest about the raters' scoring?
In the context of the study, what does a statistically significant t-test result (p = 0.023) between the first and second assessments suggest about the raters' scoring?
Which course activity is correlated with the use of role plays?
Which course activity is correlated with the use of role plays?
The 5-factor solution from the exploratory factor analysis explained 74.1% of the variance. What does this percentage indicate about the factors extracted?
The 5-factor solution from the exploratory factor analysis explained 74.1% of the variance. What does this percentage indicate about the factors extracted?
The study mentions that some factors had only three or fewer items loading on them. What is a potential concern when a factor has very few items loading on it?
The study mentions that some factors had only three or fewer items loading on them. What is a potential concern when a factor has very few items loading on it?
Why is high inter-rater reliability particularly important when using the Calgary-Cambridge Guides (C-CG) for official grading?
Why is high inter-rater reliability particularly important when using the Calgary-Cambridge Guides (C-CG) for official grading?
What should future research primarily focus on regarding the Calgary-Cambridge Guides (C-CG)?
What should future research primarily focus on regarding the Calgary-Cambridge Guides (C-CG)?
In the context of the Calgary-Cambridge Guides (C-CG), what does 'refreshment' refer to?
In the context of the Calgary-Cambridge Guides (C-CG), what does 'refreshment' refer to?
Why might it be necessary to create a version of the Calgary-Cambridge Guides (C-CG) focused on a subset of items for earlier study phases?
Why might it be necessary to create a version of the Calgary-Cambridge Guides (C-CG) focused on a subset of items for earlier study phases?
What is the primary application of the Calgary-Cambridge Guides (C-CG) in medical education?
What is the primary application of the Calgary-Cambridge Guides (C-CG) in medical education?
Which aspect of using the Calgary-Cambridge Guides(C-CG) directly contributes to ensuring fairness towards students during assessment?
Which aspect of using the Calgary-Cambridge Guides(C-CG) directly contributes to ensuring fairness towards students during assessment?
Besides ensuring fairness, what is another critical reason to improve inter-rater reliability when using the Calgary-Cambridge Guides (C-CG) for grading?
Besides ensuring fairness, what is another critical reason to improve inter-rater reliability when using the Calgary-Cambridge Guides (C-CG) for grading?
Which methodology does the content explicitly mention as a way to train medical students in communication skills?
Which methodology does the content explicitly mention as a way to train medical students in communication skills?
What is the primary intended use of the Communication-Checklist Global (C-CG) short version, according to the text?
What is the primary intended use of the Communication-Checklist Global (C-CG) short version, according to the text?
Why is rater training especially important when using the C-CG for assessment?
Why is rater training especially important when using the C-CG for assessment?
What type of validity should the C-CG be validated against in future studies?
What type of validity should the C-CG be validated against in future studies?
A medical school wants to track the progress of their students' communication skills over time and provide targeted feedback to instructors. Which application of the C-CG would be most suitable for this purpose?
A medical school wants to track the progress of their students' communication skills over time and provide targeted feedback to instructors. Which application of the C-CG would be most suitable for this purpose?
A researcher seeks to evaluate whether the C-CG accurately measures the theoretical construct of 'effective communication' in medical professionals. Which type of validity is the researcher primarily concerned with?
A researcher seeks to evaluate whether the C-CG accurately measures the theoretical construct of 'effective communication' in medical professionals. Which type of validity is the researcher primarily concerned with?
Imagine a scenario where the raters using the C-CG have not been adequately trained. Which of the following is the most likely consequence?
Imagine a scenario where the raters using the C-CG have not been adequately trained. Which of the following is the most likely consequence?
A medical program director wants to implement a tool to assess students' communication skills but needs to ensure it aligns with established standards. What should the director do to confirm the C-CG's alignment?
A medical program director wants to implement a tool to assess students' communication skills but needs to ensure it aligns with established standards. What should the director do to confirm the C-CG's alignment?
How can teachers use the information gathered from the C-CG short version to improve their communication skills training?
How can teachers use the information gathered from the C-CG short version to improve their communication skills training?
What was a key strength in the design of the C-CG assessment, contributing to its real-world applicability?
What was a key strength in the design of the C-CG assessment, contributing to its real-world applicability?
Why might the 'negotiating agenda' item on the C-CG limit its validity when assessing younger medical students?
Why might the 'negotiating agenda' item on the C-CG limit its validity when assessing younger medical students?
What do the Intraclass Correlation (ICC) values in Table 3 primarily indicate regarding the C-CG?
What do the Intraclass Correlation (ICC) values in Table 3 primarily indicate regarding the C-CG?
According to the data in Table 3, which communication skill demonstrated the highest consistency across both the first and second assessments?
According to the data in Table 3, which communication skill demonstrated the highest consistency across both the first and second assessments?
Which action most likely increases the validity and relevance of the C-CG when used in the early stages of a medical curriculum?
Which action most likely increases the validity and relevance of the C-CG when used in the early stages of a medical curriculum?
How does the C-CG reflect typical doctor-patient interactions in its design?
How does the C-CG reflect typical doctor-patient interactions in its design?
A medical school wants to assess its communication course effectiveness using C-CG. Based on the information, what adjustment should they consider?
A medical school wants to assess its communication course effectiveness using C-CG. Based on the information, what adjustment should they consider?
If a communication skills program aims to improve students' ability to clarify patient statements, which C-CG metric should be monitored closely?
If a communication skills program aims to improve students' ability to clarify patient statements, which C-CG metric should be monitored closely?
A researcher aims to adapt the C-CG for use in a cultural setting significantly different from the original. What is the most important consideration?
A researcher aims to adapt the C-CG for use in a cultural setting significantly different from the original. What is the most important consideration?
Consider a scenario where the C-CG is used to assess a student's communication skills, and the student receives a low score on 'Establishes dates.' What specific issue does this suggest?
Consider a scenario where the C-CG is used to assess a student's communication skills, and the student receives a low score on 'Establishes dates.' What specific issue does this suggest?
A medical educator observes that students consistently score low on 'Explores patient's concerns re problem' during C-CG assessments. What targeted intervention could improve this?
A medical educator observes that students consistently score low on 'Explores patient's concerns re problem' during C-CG assessments. What targeted intervention could improve this?
According to Table 3, which skill showed the least consistency between the first and second assessments, suggesting potential unreliability or variability in its evaluation?
According to Table 3, which skill showed the least consistency between the first and second assessments, suggesting potential unreliability or variability in its evaluation?
How could the C-CG be adapted to better assess communication skills in telemedicine settings?
How could the C-CG be adapted to better assess communication skills in telemedicine settings?
A medical program aims to use C-CG to evaluate students on skills that include greetings, introductions and showing respect. Which of these skills showed the least amount of inter-rater reliability?
A medical program aims to use C-CG to evaluate students on skills that include greetings, introductions and showing respect. Which of these skills showed the least amount of inter-rater reliability?
What is the most accurate interpretation of the 95% confidence interval (CI) provided alongside each ICC value in Table 3?
What is the most accurate interpretation of the 95% confidence interval (CI) provided alongside each ICC value in Table 3?
Flashcards
C-CG Recognition
C-CG Recognition
An international declaration acknowledges the C-CG as a tool for teaching communication skills.
Assessment Scale
Assessment Scale
A 5-point scale (1 = excellent, 5 = deficient) was implemented to make assessment easier
Video selection for C-CG
Video selection for C-CG
Researchers selected 5 videos showing a range of student performance from excellent to deficient to represent stepwise grading.
Study Aim
Study Aim
Signup and view all the flashcards
Item Distribution
Item Distribution
Signup and view all the flashcards
Construct Validity
Construct Validity
Signup and view all the flashcards
Test-Retest Reliability
Test-Retest Reliability
Signup and view all the flashcards
Study Participants
Study Participants
Signup and view all the flashcards
Inter-rater reliability
Inter-rater reliability
Signup and view all the flashcards
Simulated Patients (SP)
Simulated Patients (SP)
Signup and view all the flashcards
Basic Clinical Skills Course
Basic Clinical Skills Course
Signup and view all the flashcards
Role Plays/Consultations with SPs
Role Plays/Consultations with SPs
Signup and view all the flashcards
Factor analysis
Factor analysis
Signup and view all the flashcards
Negotiates Agenda
Negotiates Agenda
Signup and view all the flashcards
Patient Orientation
Patient Orientation
Signup and view all the flashcards
Structuring the Consultation
Structuring the Consultation
Signup and view all the flashcards
Exploratory Factor Analysis
Exploratory Factor Analysis
Signup and view all the flashcards
Varimax Rotation
Varimax Rotation
Signup and view all the flashcards
Pearson's r
Pearson's r
Signup and view all the flashcards
T-test for dependent samples
T-test for dependent samples
Signup and view all the flashcards
Ceiling/Floor Effects
Ceiling/Floor Effects
Signup and view all the flashcards
Calgary-Cambridge Guides
Calgary-Cambridge Guides
Signup and view all the flashcards
Future C-CG research
Future C-CG research
Signup and view all the flashcards
C-CG version subsets
C-CG version subsets
Signup and view all the flashcards
Simulated patients
Simulated patients
Signup and view all the flashcards
Fairness in grading
Fairness in grading
Signup and view all the flashcards
Referenced Observation
Referenced Observation
Signup and view all the flashcards
Clinical method teaching marriage
Clinical method teaching marriage
Signup and view all the flashcards
Face validity
Face validity
Signup and view all the flashcards
Content validity
Content validity
Signup and view all the flashcards
Concurrent validity
Concurrent validity
Signup and view all the flashcards
Predictive validity
Predictive validity
Signup and view all the flashcards
Kappa statistic
Kappa statistic
Signup and view all the flashcards
Intraclass Correlation (ICC)
Intraclass Correlation (ICC)
Signup and view all the flashcards
Younger medical students
Younger medical students
Signup and view all the flashcards
Typical tasks
Typical tasks
Signup and view all the flashcards
Structured procedure
Structured procedure
Signup and view all the flashcards
Greets patient
Greets patient
Signup and view all the flashcards
Introduces self and role
Introduces self and role
Signup and view all the flashcards
Demonstrates respect
Demonstrates respect
Signup and view all the flashcards
Appropriately moves from open to closed questions
Appropriately moves from open to closed questions
Signup and view all the flashcards
C-CG Purpose
C-CG Purpose
Signup and view all the flashcards
C-CG Short Version Use
C-CG Short Version Use
Signup and view all the flashcards
C-CG Reliability
C-CG Reliability
Signup and view all the flashcards
C-CG Feedback
C-CG Feedback
Signup and view all the flashcards
Rater Training Importance
Rater Training Importance
Signup and view all the flashcards
C-CG Validation
C-CG Validation
Signup and view all the flashcards
C-CG for Teacher Improvement
C-CG for Teacher Improvement
Signup and view all the flashcards
C-CG Target Group
C-CG Target Group
Signup and view all the flashcards
Study Notes
- The study analyzes the psychometric properties of the short version of the Calgary Cambridge Guides (C-CG).
- The study aims to determine if the C-CG is suitable for assessing communication skills in young undergraduate medical students.
Methods
- A translated version of the C-CG was used.
- 30 members from the Department of General Practice rated 5 videotaped encounters between students and simulated patients twice
- Item analysis was conducted to find floor and ceiling effects.
- Construct validity was assessed using exploratory factor analysis.
- Intra-rater reliability was measured over a 3-month interval
- Inter-rater reliability was assessed using the intraclass correlation coefficient.
Results
- The C-CG items showed no ceiling or floor effects.
- Factor analysis extracted five factors.
- Four of the five factors represented important constructs of doctor-patient communication.
- There was correlations at 0.75 (p < 0.0001) for the first and second video assessments.
- Intraclass correlation coefficients ranged from 0.05 to 0.57.
Conclusion
- The study recommends the C-CG as a tool for assessing communication skills in undergraduate medical students.
- Raters need thorough instruction because inter-rater reliability scores were deficient
- Keywords: Undergraduate medical education, questionnaires, physician-patient relations, teaching, observer variation.
Background
- Communicative competence is an important goal of medical education and is considered an essential skill.
- The German Medical Licensure Act of 2004 emphasized teaching communicative and social skills in medical curricula at a young age.
- Instruments like MAAS-Global, LIV-MAAS, LCAS, and C-CG are used to measure communication skills.
- Instruments are often adapted to measure performance on exams like OSCEs and to assess learner performance before and after teaching terms.
- Instruments differ in form, scope, and objectives.
Calgary Cambridge Guide (C-CG)
- C-CG was developed for curricular planning and defining teaching goals in communication skills.
- C-CG covers the medical interview and can be used as an observation guide or assessment tool.
- The 28-item version of the C-CG seems appropriate for a basic skills course.
Study Objectives
- Study analyzed the psychometric properties of the short version of C-CG
- Study determines whether C-CG use can be recommended in the assessment of communication skills in medical students.
- C-CG aspects to study include item distribution, construct validity, test-retest reliability, and inter-rater reliability.
Context
- "Basic clinical skills course" at Göttingen University Medical School includes manual skills and communication skills
- Includes role plays and consultations with simulated patients (SP) in small-group learning sessions.
- Course extends over 12 weeks with 3-hour modules.
- Students attend this course at the beginning of their 3rd year.
The Instrument: C-CG version
- 28 items designed for assessing the history-taking interview.
- Has a 3-point scale ("no", "yes, but", "yes").
- Sub-divided in 6 parts: 'initiating the session', 'gathering information', 'understanding the patient perspective', 'providing a structure for the consultation', 'building a relationship', and 'closing the session'.
- 3 researchers translated the C-CG version independently into German ("forward" translation).
- A native speaker translated the preliminary instrument "backward" into English.
- Two senior lecturers reviewed all translations and developed the pre-final version.
- Final version pre-tested with student tutors, who found a larger selection of ratings would make assessment easier.
- A 5-point scale (1 = excellent and 5 = deficient) was implemented.
Material Preparation
- A sample of 5 videos was selected from 117 SP consultation videos to represent student performance range
- Two authors screened the video material and selected 5 video consultations which showed a stepwise grading from excellent to deficient performances.
- The videotapes were converted to digitised files on DVD.
Participants and Training
- Members of the Institute of General Practice (medical doctors, sociologists, psychologists, and student tutors) were asked to take part in the study as raters
- Group was trained in a 90-minute session, including a short presentation of the experiment and the C-CG.
- Raters carried out an individual rating after watching an 8-minute-video of an SP consultation
- Individual ratings were then discussed item per item to reach consensus about scoring
- All raters received a DVD with the 5 selected SP-consultations and the C-CG in printed form and instructed to score the videos within 4 weeks.
- Rating procedure was repeated after 3 months.
Statistical Analysis
- SAS 9.3 was used for all analyses.
- Score distributions were calculated to evaluate possible floor and/or ceiling effects.
- Construct validity was investigated by exploratory factor analysis using varimax rotation.
- Intra-rater reliability was measured with Pearson's r, a t-test, and descriptive analysis
- Inter-rater reliability was assessed with the intraclass correlation coefficient (ICC).
Item Analysis
- Measures of distribution of the scores for all 28 items, the 5 scales and the overall score of the C-CG.
- Summed for all raters and all videos for the first rating round.
- Means are slightly skewed to the upper end of the scale.
- Raters used of all scores and the IQR ranges as well as the 10% to 90% ranges were rather broad.
- The values for the second assessment were nearly identical.
Construct Validity
- A 5 factors solutions was extracted based on the 'eigenvalue' criterion (>1.0).
- The corresponding factor scorings occurred after varimax rotation.
- Technicalities include opening and closing a session with a patient, structuring the consultation, formal aspects of communication and patient orientation
- The 5-factor solution explained 74.1% of the whole variance.
Test-retest Reliability
- Raters' mean total score at the first assessment was 2.37 (SD 0.7).
- Raters gave better scores at the second assessment (mean 2.26, SD 0.7).
- Ratings at the first and second rating round correlated at 0.75 (Pearson's r, p<0.0001)
- 34.1 to 66.7% instances, the scores of both assessments of a rater were identical.
- Items with the best agreements were: 'closes interview by summarising briefly' and 'greets patient'.
Inter-rater Reliability
- ICCs for each item were moderate ranging from 0.05 to 0.57.
- Items with an ICC above 0.4 at the first rating round were: 'encourages patient to tell story', listens attentive' demonstrates respect', 'demonstrates appropriate non-verbal behaviour' and 'is not judgemental'.
Discussion
- Recommend the C-CG as an instrument for assessing communication skills in undergraduate medical students.
- Suggest that it should be used for regular monitoring of the success of curriculum.
- Raters need a thorough instruction before using the C-CG as inter-rater reliability is potentially an issue.
- Wide range between 1 and 5 shows the ability of the C-CG to detect differences and graduations within single communicative skills.
- The C-CG seems well-suited to identify good compared to poor performers.
Studying That Suits You
Use AI to generate personalized quizzes and flashcards to suit your learning preferences.
Related Documents
Description
This study examines the C-CG's 5-point scale implementation, goals, and rater methodology. It explores video selection, rater training, and statistical measures used in the assessment. The study also investigates the C-CG's international recognition.