4a: Measurements & Scales PDF

**[Core Principles in Mental Health Research:]** **[4a: Measurements & Scales]** [Learning Objectives:] - To review the core principles of measurement selection, with a focus on practical and psychometric considerations. - To practically apply principles of questionnaire design and p...

**[Core Principles in Mental Health Research:]** **[4a: Measurements & Scales]** [Learning Objectives:] - To review the core principles of measurement selection, with a focus on practical and psychometric considerations. - To practically apply principles of questionnaire design and psychometric properties and outline a plan for how these would be tested - To understand some of the challenges in conducting research, including outcome measurement in a culturally diverse population [Will the questionnaire pick changes over time and differences between groups?] - Responsiveness: does the questionnaire pick up changes over time sufficiently well? - May be defined in relation to stakeholder views: is a difference agreed by patients and/or clinicians as important detected by the instrument? - Or there are ways of calculating from statistical distribution e.g. statistical definitions e.g. standardised response mean calculated from average pre-/post difference and standard deviation - Responsiveness is often included alongside reliability and validity as one of core psychometric properties in recent checklists of essential psychometric properties (e.g. COMET checklist) - What's important to clinicians/patients?? Gap usual [Does the questionnaire produce a useful range of scores?] - Precision: Do scores vary sufficiently among respondents for measurements to be useful? - Floor and ceiling effects: Where many people score at the upper or lower end of a scale so results are not useful - Could be a problem with the scale or use in an inappropriate population e.g. a scale to measure symptoms among people with severe mental health problems used in general population - Above are sometimes subsumed within definition of responsiveness -- some variations between lists of core psychometric properties - May have to remove certain items and retest to determine reliability and validity [Is the questionnaire appropriate for the study in which it is being used?] - Appropriateness is generally study-specific rather than a property of questionnaires. - Is this an appropriate tool for the study question? - E.g. Does what's measured map exactly onto the study research question? - Is it designed for/suitable for this population? - E.g. a measure of resilience developed and piloted with undergraduates may not be suitable for people with severe mental health problems -- new pilot study often needed [Is it going to be practical to use this tool? ] - Feasibility: can you use the tool in the proper way in practice. Impediments (generally study/situation specific) might include: - Lack of time or a lengthy tool - Lack of access to training or of money to pay for an expensive tool - Not having researchers who have the required training to use the tool - Some tools have charges -- must consider when planning a study (free versions online might not have scoring) [Is the questionnaire acceptable to the people we plan to use it with?] - Acceptability is very important. Reasons for an instrument being unacceptable (which can change over time) include: - People find it rude or stigmatising - Language is hard to comprehend, or seems irrelevant - Tone is pessimistic or distressing - It looks off-putting, is glitchy, long or hard to use - Response rate or partial completion is an indicator - Another reason why piloting/stakeholder consultation is essential, even for previously used instruments [Cross-cultural validity *(sometimes defined as a component of construct validity)*] - Cross-cultural validity: defined as measurement invariance -- a scale performs the same way across cultures - May aim for this or for instruments developed for or adapted to specific groups - Either way -- should be confident instrument is valid among people you use it with - Can be challenging in highly diverse contexts e.g. London [Cross-cultural validation and instrument translation] - Processes required to produce a valid version of an instrument in another culture/language include: - Translation and back-translation - Exploring whether language/items have same meaning, often through qualitative work (e.g., idioms, colloquialisms) - Exploring whether important aspects of the concept being measure are being covered - Investigating psychometric properties (reliability, validity, responsiveness), piloting [Conducting valid research in a diverse environment] - Concepts of cross-cultural validity can be challenging in a diverse contemporary environment e.g., where many people's identities are complex and mixed, not straightforwardly part of a specific culture - An apparently cross-culturally valid instrument could miss important aspects of minorities' experiences e.g., impacts of racism -- measuring the same concepts in everyone may not always be right - When developing and testing measures: study samples should in any case be diverse and representative of local populations [Some important challenges to conducting research that is relevant for people from Black and Minority Ethnic backgrounds (an introduction)] - Small minority of mental health academics/researchers are from BAME backgrounds (we hope some of you will help remedy this!) (1%!!!) - Study samples tend to under-recruit people from minorities whose mental health appears impacted by injustice and inequality - Measures and study methods may not reflect important aspects of minority experience e.g. impacts of racism and discrimination - Questions that are important from a BAME perspective are not addressed: e.g. Very limited research on why people from Black backgrounds in UK are much more likely to be admitted to hospital compulsorily, even though this has been demonstrated again and again........