C-CG Study: Goals, Raters, and Scale Implementation
55 Questions
0 Views

Choose a study mode

Play Quiz
Study Flashcards
Spaced Repetition
Chat to Lesson

Podcast

Play an AI-generated podcast conversation about this lesson

Questions and Answers

Why was a 5-point scale implemented in the C-CG, instead of the original scale?

  • To align with international teaching communication skill standards.
  • Because the original scale was not validated in translated versions.
  • To exclude physical examinations from the basic skill course.
  • Because raters felt a larger selection of ratings would make assessment easier. (correct)

What was the primary goal of the study regarding the short version of the C-CG?

  • To translate the C-CG into multiple languages.
  • To compare the C-CG with other communication assessment tools.
  • To introduce the C-CG in basic medical skills courses.
  • To analyse its psychometric properties and determine its suitability for assessing communication skills in undergraduate medical students. (correct)

Which of the following is NOT one of the aspects specifically studied in detail regarding the C-CG?

  • Test-retest reliability.
  • Item distribution.
  • Construct validity.
  • Inter-rater reliability. (correct)

How were the videos selected to represent the range of student performance?

<p>Chosen to show a stepwise grading from excellent to deficient performances, as determined by two authors. (D)</p> Signup and view all the answers

Who participated as raters in the study?

<p>Members from the Institute of General Practice, including medical doctors, sociologists, psychologists, and student tutors. (A)</p> Signup and view all the answers

What did the training session for the raters include?

<p>A short presentation of the experiment and the C-CG. (A)</p> Signup and view all the answers

What is the significance of the C-CG's acceptance within an international leading declaration?

<p>It highlights the instrument's recognition and importance in teaching communication skills globally. (D)</p> Signup and view all the answers

From the initial pool, how many videos were selected for the study, and what determined the selection?

<p>5 videos, selected to represent the range of quality of student performance. (C)</p> Signup and view all the answers

What was the primary goal of discussing individual C-CG ratings item by item within the group after the initial video consultation?

<p>To arrive at the most accurate and agreed-upon scoring for the observed consultation. (A)</p> Signup and view all the answers

How did the researchers ensure the raters completed the video scoring within the designated timeframe?

<p>By sending reminder emails and making telephone calls. (B)</p> Signup and view all the answers

What statistical measure was used to determine if there was a significant difference between the two assessments?

<p>T-test for dependent samples (C)</p> Signup and view all the answers

Which statistical method was employed to reduce the number of variables to a smaller set of factors, thereby simplifying the data structure?

<p>Exploratory factor analysis (D)</p> Signup and view all the answers

Which aspect of the 'basic clinical skills course' is directly assessed using simulated patients?

<p>Communication skills, including history-taking. (B)</p> Signup and view all the answers

What statistical software was used to conduct the psychometric analysis of the Calgary Cambridge Guides (C-CG)?

<p>SAS 9.3 (B)</p> Signup and view all the answers

What threshold was used to determine the statistical significance of the p-value when comparing the two assessments?

<p>p &lt; 0.05 (D)</p> Signup and view all the answers

Which type of reliability was assessed using statistical measures like Pearson's r and a t-test over a three-month interval?

<p>Test-retest reliability (D)</p> Signup and view all the answers

According to the factor analysis, which of the following communication aspects is represented by one of the four identified factors?

<p>Technicalities of opening and closing a session. (A)</p> Signup and view all the answers

In the factor analysis, varimax rotation was used. What is the primary purpose of applying varimax rotation in factor analysis?

<p>To simplify the factor structure and enhance interpretability (B)</p> Signup and view all the answers

In the context of the study, what does 'inter-rater reliability' specifically refer to?

<p>The ability of different raters to use the C-CG intuitively and consistently. (A)</p> Signup and view all the answers

What ethical consideration was addressed in the study design?

<p>Review and approval by the University of Göttingen's ethical review board. (B)</p> Signup and view all the answers

In the context of the study, what does a statistically significant t-test result (p = 0.023) between the first and second assessments suggest about the raters' scoring?

<p>There was a significant, though marginal, shift in the raters' scoring between the two assessments. (B)</p> Signup and view all the answers

Which course activity is correlated with the use of role plays?

<p>Simulated patient consultations (D)</p> Signup and view all the answers

The 5-factor solution from the exploratory factor analysis explained 74.1% of the variance. What does this percentage indicate about the factors extracted?

<p>The extracted factors account for a substantial amount of the total variability in the data. (C)</p> Signup and view all the answers

The study mentions that some factors had only three or fewer items loading on them. What is a potential concern when a factor has very few items loading on it?

<p>The factor may not be adequately represented, potentially lacking in breadth and stability. (B)</p> Signup and view all the answers

Why is high inter-rater reliability particularly important when using the Calgary-Cambridge Guides (C-CG) for official grading?

<p>To ensure fairness and avoid assessment dependence on individual rater interpretation. (C)</p> Signup and view all the answers

What should future research primarily focus on regarding the Calgary-Cambridge Guides (C-CG)?

<p>Re-assessing and refining the underlying factor structure of the C-CG instrument itself. (B)</p> Signup and view all the answers

In the context of the Calgary-Cambridge Guides (C-CG), what does 'refreshment' refer to?

<p>Repeating the training to ensure proper application of the guides. (D)</p> Signup and view all the answers

Why might it be necessary to create a version of the Calgary-Cambridge Guides (C-CG) focused on a subset of items for earlier study phases?

<p>To tailor the assessment to the specific competencies expected at that stage of learning. (B)</p> Signup and view all the answers

What is the primary application of the Calgary-Cambridge Guides (C-CG) in medical education?

<p>Defining curriculum and organizing teaching in communication training programs. (A)</p> Signup and view all the answers

Which aspect of using the Calgary-Cambridge Guides(C-CG) directly contributes to ensuring fairness towards students during assessment?

<p>Achieving high inter-rater reliability among assessors. (A)</p> Signup and view all the answers

Besides ensuring fairness, what is another critical reason to improve inter-rater reliability when using the Calgary-Cambridge Guides (C-CG) for grading?

<p>To avoid individual assessment's dependence on a rater’s personal interpretation (D)</p> Signup and view all the answers

Which methodology does the content explicitly mention as a way to train medical students in communication skills?

<p>Training with lay actors to simulate patient interactions. (C)</p> Signup and view all the answers

What is the primary intended use of the Communication-Checklist Global (C-CG) short version, according to the text?

<p>To evaluate students' communication skills and inform teachers about their training progress. (A)</p> Signup and view all the answers

Why is rater training especially important when using the C-CG for assessment?

<p>To enhance the reliability of the assessment results. (A)</p> Signup and view all the answers

What type of validity should the C-CG be validated against in future studies?

<p>Construct validity, especially convergent validity (D)</p> Signup and view all the answers

A medical school wants to track the progress of their students' communication skills over time and provide targeted feedback to instructors. Which application of the C-CG would be most suitable for this purpose?

<p>Using the C-CG short version to regularly assess students and inform teachers whether they have reached their training goals. (A)</p> Signup and view all the answers

A researcher seeks to evaluate whether the C-CG accurately measures the theoretical construct of 'effective communication' in medical professionals. Which type of validity is the researcher primarily concerned with?

<p>Construct validity (B)</p> Signup and view all the answers

Imagine a scenario where the raters using the C-CG have not been adequately trained. Which of the following is the most likely consequence?

<p>The reliability of the assessment results will be compromised. (A)</p> Signup and view all the answers

A medical program director wants to implement a tool to assess students' communication skills but needs to ensure it aligns with established standards. What should the director do to confirm the C-CG's alignment?

<p>Administer the C-CG alongside other communication assessment tools and compare the results. (B)</p> Signup and view all the answers

How can teachers use the information gathered from the C-CG short version to improve their communication skills training?

<p>By understanding if they have reached their training goals and whether they have improved compared to the previous semester. (A)</p> Signup and view all the answers

What was a key strength in the design of the C-CG assessment, contributing to its real-world applicability?

<p>The raters having varied backgrounds, mirroring the interdisciplinary nature of medical training. (D)</p> Signup and view all the answers

Why might the 'negotiating agenda' item on the C-CG limit its validity when assessing younger medical students?

<p>The skill of negotiating agendas may not yet be developed or emphasized in early medical curricula. (D)</p> Signup and view all the answers

What do the Intraclass Correlation (ICC) values in Table 3 primarily indicate regarding the C-CG?

<p>The level of agreement or consistency among raters in their assessment of communication skills. (B)</p> Signup and view all the answers

According to the data in Table 3, which communication skill demonstrated the highest consistency across both the first and second assessments?

<p>Encourages patient to tell story (C)</p> Signup and view all the answers

Which action most likely increases the validity and relevance of the C-CG when used in the early stages of a medical curriculum?

<p>Removing the 'negotiating agenda' item from the assessment. (C)</p> Signup and view all the answers

How does the C-CG reflect typical doctor-patient interactions in its design?

<p>By including items that measure structured diagnostic procedures and effective communication skills. (B)</p> Signup and view all the answers

A medical school wants to assess its communication course effectiveness using C-CG. Based on the information, what adjustment should they consider?

<p>Deleting the ‘negotiating agenda’ item, especially for younger students. (C)</p> Signup and view all the answers

If a communication skills program aims to improve students' ability to clarify patient statements, which C-CG metric should be monitored closely?

<p>Clarifies patient’s statements. (A)</p> Signup and view all the answers

A researcher aims to adapt the C-CG for use in a cultural setting significantly different from the original. What is the most important consideration?

<p>The cultural relevance and interpretation of each item. (D)</p> Signup and view all the answers

Consider a scenario where the C-CG is used to assess a student's communication skills, and the student receives a low score on 'Establishes dates.' What specific issue does this suggest?

<p>The student has difficulty in creating a timeline of events related to the patient's health. (D)</p> Signup and view all the answers

A medical educator observes that students consistently score low on 'Explores patient's concerns re problem' during C-CG assessments. What targeted intervention could improve this?

<p>Training on active listening and empathetic questioning techniques. (A)</p> Signup and view all the answers

According to Table 3, which skill showed the least consistency between the first and second assessments, suggesting potential unreliability or variability in its evaluation?

<p>Clarifies patient’s statements. (C)</p> Signup and view all the answers

How could the C-CG be adapted to better assess communication skills in telemedicine settings?

<p>By adding items that evaluate the effective use of technology in communication. (A)</p> Signup and view all the answers

A medical program aims to use C-CG to evaluate students on skills that include greetings, introductions and showing respect. Which of these skills showed the least amount of inter-rater reliability?

<p>Greets patient (D)</p> Signup and view all the answers

What is the most accurate interpretation of the 95% confidence interval (CI) provided alongside each ICC value in Table 3?

<p>The range within which the true ICC value is likely to lie, with 95% confidence. (B)</p> Signup and view all the answers

Flashcards

C-CG Recognition

An international declaration acknowledges the C-CG as a tool for teaching communication skills.

Assessment Scale

A 5-point scale (1 = excellent, 5 = deficient) was implemented to make assessment easier

Video selection for C-CG

Researchers selected 5 videos showing a range of student performance from excellent to deficient to represent stepwise grading.

Study Aim

Aims to analyse the psychometric properties of the short version of C-CG for assessing communications skills in young undergraduate medical students.

Signup and view all the flashcards

Item Distribution

Does the C-CG provide a differentiated assessment?

Signup and view all the flashcards

Construct Validity

Does the C-CG represent meaningful aspects of communication?

Signup and view all the flashcards

Test-Retest Reliability

Can the C-CG be used reliably from semester to semester?

Signup and view all the flashcards

Study Participants

Medical doctors, sociologists, psychologists, and student tutors.

Signup and view all the flashcards

Inter-rater reliability

Consistency of scores when different raters use the C-CG.

Signup and view all the flashcards

Simulated Patients (SP)

Individuals trained to act as patients to simulate real clinical encounters for training.

Signup and view all the flashcards

Basic Clinical Skills Course

A course including manual skills and communication taught in small groups.

Signup and view all the flashcards

Role Plays/Consultations with SPs

A teaching method involving role-playing with simulated patients.

Signup and view all the flashcards

Factor analysis

A method of statistical analysis to determine distinct categories within a set of data.

Signup and view all the flashcards

Negotiates Agenda

Agreeing on the topics for discussion during a consultation.

Signup and view all the flashcards

Patient Orientation

Key doctor behaviour, showing interest and care for the patient’s well-being.

Signup and view all the flashcards

Structuring the Consultation

Facilitating the patient's expression of concerns, building a relationship.

Signup and view all the flashcards

Exploratory Factor Analysis

A statistical method used to explore the underlying structure of a dataset and reduce its dimensionality by identifying factors.

Signup and view all the flashcards

Varimax Rotation

A rotation method in factor analysis that simplifies the factors by maximizing the variance of the loadings, making them easier to interpret.

Signup and view all the flashcards

Pearson's r

A statistical measure that assesses the linear relationship between two continuous variables.

Signup and view all the flashcards

T-test for dependent samples

A statistical test used to determine if there is a significant difference between the means of two related groups.

Signup and view all the flashcards

Ceiling/Floor Effects

Effects where scores cluster at the high end (ceiling) or low end (floor) of a measurement scale, limiting discrimination.

Signup and view all the flashcards

Calgary-Cambridge Guides

A framework used to define curriculum and organize teaching in communication training programs.

Signup and view all the flashcards

Future C-CG research

Focuses on refining the factor structure of the C-CG instrument.

Signup and view all the flashcards

C-CG version subsets

Creating specialized versions of the C-CG for specific study phases.

Signup and view all the flashcards

Simulated patients

Using actors to simulate real patients in medical training scenarios.

Signup and view all the flashcards

Fairness in grading

Process of ensuring consistency and minimizing subjective bias in assessments, especially in communication skills evaluation.

Signup and view all the flashcards

Referenced Observation

Tool for objective observation, skill development & feedback in clinical education.

Signup and view all the flashcards

Clinical method teaching marriage

Tool to ensure course content and student progress are aligned with clinical teaching goals.

Signup and view all the flashcards

Face validity

Whether the instrument appears to measure what it intends to measure from the perspective of the test-taker.

Signup and view all the flashcards

Content validity

The extent to which a measure covers all facets of a concept.

Signup and view all the flashcards

Concurrent validity

Correlation between a test and a criterion measure taken at the same time.

Signup and view all the flashcards

Predictive validity

Correlation between a test and a criterion measure taken in the future.

Signup and view all the flashcards

Kappa statistic

A measure of agreement, taking into account the possibility of agreement occurring by chance.

Signup and view all the flashcards

Intraclass Correlation (ICC)

A statistic that measures the degree of consistency among multiple measurements of the same subject.

Signup and view all the flashcards

Younger medical students

Beginning to learn their role as a communicator.

Signup and view all the flashcards

Typical tasks

Skills and abilities that medical students should learn communication courses of doctor-patient encounters.

Signup and view all the flashcards

Structured procedure

A structured procedure in diagnostics, treatment, referral.

Signup and view all the flashcards

Greets patient

Greeting patients politely at start of consultation.

Signup and view all the flashcards

Introduces self and role

Explaining who you are and what you do.

Signup and view all the flashcards

Demonstrates respect

Showing respect towards the opinions of others and treating patients with decency.

Signup and view all the flashcards

Appropriately moves from open to closed questions

Students should also practice moving between general (open) and specific (closed) questions.

Signup and view all the flashcards

C-CG Purpose

A tool designed for planning curricula and setting communication skills teaching objectives.

Signup and view all the flashcards

C-CG Short Version Use

The short version is suitable for evaluating communication skills.

Signup and view all the flashcards

C-CG Reliability

Student communication skills can be reliably assessed using the C-CG.

Signup and view all the flashcards

C-CG Feedback

Regularly inform educators whether they have met their training goals and improved compared to the previous semester.

Signup and view all the flashcards

Rater Training Importance

Essential for ensuring trustworthy results when using the instrument.

Signup and view all the flashcards

C-CG Validation

Should be validated against a gold standard to verify its construct validity, especially convergent validity.

Signup and view all the flashcards

C-CG for Teacher Improvement

Can indicate if teachers are reaching communication skill training objectives and if their teaching has improved over time.

Signup and view all the flashcards

C-CG Target Group

Can assess communication skills in undergraduate medical students reliably.

Signup and view all the flashcards

Study Notes

  • The study analyzes the psychometric properties of the short version of the Calgary Cambridge Guides (C-CG).
  • The study aims to determine if the C-CG is suitable for assessing communication skills in young undergraduate medical students.

Methods

  • A translated version of the C-CG was used.
  • 30 members from the Department of General Practice rated 5 videotaped encounters between students and simulated patients twice
  • Item analysis was conducted to find floor and ceiling effects.
  • Construct validity was assessed using exploratory factor analysis.
  • Intra-rater reliability was measured over a 3-month interval
  • Inter-rater reliability was assessed using the intraclass correlation coefficient.

Results

  • The C-CG items showed no ceiling or floor effects.
  • Factor analysis extracted five factors.
  • Four of the five factors represented important constructs of doctor-patient communication.
  • There was correlations at 0.75 (p < 0.0001) for the first and second video assessments.
  • Intraclass correlation coefficients ranged from 0.05 to 0.57.

Conclusion

  • The study recommends the C-CG as a tool for assessing communication skills in undergraduate medical students.
  • Raters need thorough instruction because inter-rater reliability scores were deficient
  • Keywords: Undergraduate medical education, questionnaires, physician-patient relations, teaching, observer variation.

Background

  • Communicative competence is an important goal of medical education and is considered an essential skill.
  • The German Medical Licensure Act of 2004 emphasized teaching communicative and social skills in medical curricula at a young age.
  • Instruments like MAAS-Global, LIV-MAAS, LCAS, and C-CG are used to measure communication skills.
  • Instruments are often adapted to measure performance on exams like OSCEs and to assess learner performance before and after teaching terms.
  • Instruments differ in form, scope, and objectives.

Calgary Cambridge Guide (C-CG)

  • C-CG was developed for curricular planning and defining teaching goals in communication skills.
  • C-CG covers the medical interview and can be used as an observation guide or assessment tool.
  • The 28-item version of the C-CG seems appropriate for a basic skills course.

Study Objectives

  • Study analyzed the psychometric properties of the short version of C-CG
  • Study determines whether C-CG use can be recommended in the assessment of communication skills in medical students.
  • C-CG aspects to study include item distribution, construct validity, test-retest reliability, and inter-rater reliability.

Context

  • "Basic clinical skills course" at Göttingen University Medical School includes manual skills and communication skills
  • Includes role plays and consultations with simulated patients (SP) in small-group learning sessions.
  • Course extends over 12 weeks with 3-hour modules.
  • Students attend this course at the beginning of their 3rd year.

The Instrument: C-CG version

  • 28 items designed for assessing the history-taking interview.
  • Has a 3-point scale ("no", "yes, but", "yes").
  • Sub-divided in 6 parts: 'initiating the session', 'gathering information', 'understanding the patient perspective', 'providing a structure for the consultation', 'building a relationship', and 'closing the session'.
  • 3 researchers translated the C-CG version independently into German ("forward" translation).
  • A native speaker translated the preliminary instrument "backward" into English.
  • Two senior lecturers reviewed all translations and developed the pre-final version.
  • Final version pre-tested with student tutors, who found a larger selection of ratings would make assessment easier.
  • A 5-point scale (1 = excellent and 5 = deficient) was implemented.

Material Preparation

  • A sample of 5 videos was selected from 117 SP consultation videos to represent student performance range
  • Two authors screened the video material and selected 5 video consultations which showed a stepwise grading from excellent to deficient performances.
  • The videotapes were converted to digitised files on DVD.

Participants and Training

  • Members of the Institute of General Practice (medical doctors, sociologists, psychologists, and student tutors) were asked to take part in the study as raters
  • Group was trained in a 90-minute session, including a short presentation of the experiment and the C-CG.
  • Raters carried out an individual rating after watching an 8-minute-video of an SP consultation
  • Individual ratings were then discussed item per item to reach consensus about scoring
  • All raters received a DVD with the 5 selected SP-consultations and the C-CG in printed form and instructed to score the videos within 4 weeks.
  • Rating procedure was repeated after 3 months.

Statistical Analysis

  • SAS 9.3 was used for all analyses.
  • Score distributions were calculated to evaluate possible floor and/or ceiling effects.
  • Construct validity was investigated by exploratory factor analysis using varimax rotation.
  • Intra-rater reliability was measured with Pearson's r, a t-test, and descriptive analysis
  • Inter-rater reliability was assessed with the intraclass correlation coefficient (ICC).

Item Analysis

  • Measures of distribution of the scores for all 28 items, the 5 scales and the overall score of the C-CG.
  • Summed for all raters and all videos for the first rating round.
  • Means are slightly skewed to the upper end of the scale.
  • Raters used of all scores and the IQR ranges as well as the 10% to 90% ranges were rather broad.
  • The values for the second assessment were nearly identical.

Construct Validity

  • A 5 factors solutions was extracted based on the 'eigenvalue' criterion (>1.0).
  • The corresponding factor scorings occurred after varimax rotation.
  • Technicalities include opening and closing a session with a patient, structuring the consultation, formal aspects of communication and patient orientation
  • The 5-factor solution explained 74.1% of the whole variance.

Test-retest Reliability

  • Raters' mean total score at the first assessment was 2.37 (SD 0.7).
  • Raters gave better scores at the second assessment (mean 2.26, SD 0.7).
  • Ratings at the first and second rating round correlated at 0.75 (Pearson's r, p<0.0001)
  • 34.1 to 66.7% instances, the scores of both assessments of a rater were identical.
  • Items with the best agreements were: 'closes interview by summarising briefly' and 'greets patient'.

Inter-rater Reliability

  • ICCs for each item were moderate ranging from 0.05 to 0.57.
  • Items with an ICC above 0.4 at the first rating round were: 'encourages patient to tell story', listens attentive' demonstrates respect', 'demonstrates appropriate non-verbal behaviour' and 'is not judgemental'.

Discussion

  • Recommend the C-CG as an instrument for assessing communication skills in undergraduate medical students.
  • Suggest that it should be used for regular monitoring of the success of curriculum.
  • Raters need a thorough instruction before using the C-CG as inter-rater reliability is potentially an issue.
  • Wide range between 1 and 5 shows the ability of the C-CG to detect differences and graduations within single communicative skills.
  • The C-CG seems well-suited to identify good compared to poor performers.

Studying That Suits You

Use AI to generate personalized quizzes and flashcards to suit your learning preferences.

Quiz Team

Related Documents

Description

This study examines the C-CG's 5-point scale implementation, goals, and rater methodology. It explores video selection, rater training, and statistical measures used in the assessment. The study also investigates the C-CG's international recognition.

More Like This

Use Quizgecko on...
Browser
Browser