Developing Pedestrians' Red-light Violation Behavior Questionnaire (PRVBQ) PDF

Original Article Bull Emerg Trauma 2020;8(2):98-106. Developing Pedestrians’ Red-light Violation Behavior Questionnaire (PRVBQ); Assessme...

Original Article Bull Emerg Trauma 2020;8(2):98-106. Developing Pedestrians’ Red-light Violation Behavior Questionnaire (PRVBQ); Assessment of Content Validity and Reliability Mahdi Moshki¹, Abdoljavad Khajavi2, Homayoun Sadeghi-Bazargani3, Shahram Vahedi4, Saeid Pour- Doulati*5 1 Department of Health Education and Health Promotion, School of Health Sciences; Social Development and Health Promotion Research Center, Gonabad University of Medical Sciences, Gonabad, Iran 2 Community Medicine Department, School of Medicine, Gonabad University of Medical Sciences, Gonabad, Iran 3 Road Traffic Injury Research Center, Tabriz University of Medical Sciences, Tabriz, Iran 4 Department of Education and Psychology, University of Tabriz, Tabriz, Iran 5 Social Development and Health Promotion Research Center, Gonabad University of Medical Sciences, Gonabad, Iran *Corresponding author: Saeid Pour-Doulati Received: September 14, 2019 Address: Center for non-communicable diseases control and prevention, East Revised: January 24, 2020 Azerbaijan Province Health Center, Postal code: 5143814998, Tabriz, Iran. Tel: +98-41-32372936; Cell: +98-914-3077517; Fax: +98-914-32330196; Accepted: February 17, 2020 e-mail: [email protected] Objective: To develop a self-completion pedestrians’ red-light violation behavior questionnaire (PRVBQ) based on the theory of planned behavior (TPB) and assess the content validity and reliability. Methods: This study was conducted in three phases of (i) PRVBQ development study; (ii) Content validity study including face validity; and (iii) Reliability assessment. The directed content analysis method was used for the analysis of the qualitative interviews. The item impact score was used for face validity. Content validity index (CVI) in the item level and average scale level, and content validity ratio (CVR) were determined. Intra- class Correlation Coefficient (ICC) and Cronbach’s alpha were assessed for test-retest reliability and internal consistency, respectively. Results: Draft questionnaire including 86 items was constructed. Sixteen items were eliminated due to low face and content validity, remaining 70 items in total. The PRVBQ was rated as having good content validity (individual items CVI ranged from 0.80 to 1, and overall PRVBQ CVI-Average=0.95, p=0.05). The direct measures (reflective indicators) showed excellent internal consistency with Cronbach’s alpha=0.9. All items showed excellent agreement. Conclusion: This study using a comprehensive process of development and assessment of content validity and reliability developed a content valid and reliable questionnaire predicting pedestrians’ red light violation behavior. Keywords: Pedestrian red-light violation behavior questionnaire; Validity; Reliability. Please cite this paper as: Moshki M, Khajavi A, Sadeghi-Bazargani H, Vahedi S, Pour-Doulati S. Developing Pedestrians’ Red-light Violation Behavior Questionnaire (PRVBQ); Assessment of Content Validity and Reliability. Bull Emerg Trauma. 2020;8(2):98-106. doi: 10.30476/BEAT.2020.46449. Journal compilation © 2020 Trauma Research Center, Shiraz University of Medical Sciences  Pedestrian red-light violation questionnaire Introduction role in explaining pedestrian’s unsafe road crossing behavioral intention. The habitual behavior is a T he increasing growth of motor vehicles and insufficient attention to pedestrian safety have put them at risk of injury, disability, and death. mental concept that can be automatically triggered by the environment. Repeating a previous behavior strengthens the habit. Therefore, past behavior, Pedestrians account for 23 percent of road traffic along with the ATT, SN, and BPC can contribute deaths worldwide. Pedestrians, owing to having no to behavioral intention, which in turn determines shield at all to protect them in case of a collision and the future behavior. Past behavior, explaining 42 having very low mass in comparison with motorized percent of the variance in a pedestrians’ intention to vehicles, are more vulnerable than other road users jaywalk, was introduced as the strongest predictor. Prevention of pedestrian injuries is an important of the pedestrians’ unsafe road crossing behavioral policy of the health care systems. Evidence suggests intention in China. that pedestrian injuries are both predictable and Considering the high rate of pedestrian injury preventable. and mortality and their unsafe road crossing Developing effective prevention strategies requires behavior as a major risk factor, investigating the gaining a deep understanding of traffic accident pedestrians’ risky crossing behavior for developing causes. The human component is the main cause effective preventive interventions is highly felt. of traffic accidents in contrast to environment Understanding the reasons for pedestrians’ unsafe and vehicle as the other two components of traffic road crossing behavior needs to develop valid and accidents. Prohibited road-crossings at signalized reliable measures. The development of a valid and intersections, where automatic traffic signals indicate reliable measurement instrument is a very critical to pedestrians; when they should cross, was known point, particularly in social-psychological and as the most common violations of traffic rules. health-related behavior research. Validity ensures Pedestrians in developing countries are more likely that the measurement instrument is measuring to commit a traffic violation and display more risky what it anticipates to determine and is reflecting the behaviors. intended theoretical concept. Therefore, investigating the underlying factors Evaluating content validity (including face validity) of pedestrians’ risky road-crossing behaviors is of a measurement instrument is the most important essential to develop an evidence-based and effective and a critical early step in the construct validity of intervention. The theory of planned behavior (TPB; an instrument. Content validity refers to the [4-7] ) is a widely used socio-psychological model degree to which items of a measurement instrument helping researches and safety intervention planners adequately represent the content domain. If an to understand pedestrians’ unsafe road crossing instrument lacks content validity, it is impossible behavior [3, 8-14]. Based on the TPB behavioral to establish reliability for it. Most TPB based intention and perceived behavioral control (PBC) questionnaires used for predicting pedestrians’ are proximal predictors of actual behavior. In this unsafe road crossing behavior have not presented model, three latent variables including, attitude sufficient evidence of validity and reliability. (ATT) toward the behavior, subjective norms (SN), Therefore, the purpose of the present study was and PBC, predict intention to perform a behavior. to develop a self-completion pedestrians’ red-light “Intention” is the antecedent variable of behavior. violation behavior questionnaire (PRVBQ) based on The latent variables are needed to be measured the TPB and the extended variable (PB), to use it for indirectly by questionnaire responses. the predictive application and to assess the content TPB based questionnaires may include direct validity and reliability of scores in the sample of measures (i.e., reflective indicators), indirect adult pedestrians of Tabriz city, Iran (in Persian). measures (i.e., formative indicators), or both measures for each latent variable of ATT, SN, and Materials and Methods PBC. Indirect measures are constructed based on the expectancy-value theory. The direct measures, This study was part of a larger study approved by e.g. ask respondents about the opinion of important the Local Ethics Committee and research council of people in general and indirect measures, e.g. ask Gonabad University of Medical Sciences. Gonabad, about the strength of normative beliefs with respect Iran (approval code: IR.GMU.REC.195.19). The to each reference group and motivation to comply present study was a sequential exploratory mixed with them. Direct measures used for the prediction method (qualitative and quantitative) that took place and indirect measures used for determining the in three phases between Jun 2016 and November underlying beliefs of specific behaviors, but neither 2017 and was conducted in Tabriz, Iran. approach is perfect. Therefore, it is recommended that each TPB questionnaire uses both direct and Construct indirect measures. The construct of the study is PRVB, which has In addition to the main constructs of the TPB, past the potential for the crash, leading to injury and behavior (PB) is a variable that plays a significant death. This study was conducted in 3 phases www.beat-journal.com 99 Moshki M et al. (Figure 1) including, (i) PRVBQ development study consisted of belief elicitation for item generation and instrument construction; (ii) Content validity study including face validity; and (iii) Reliability assessment including internal consistency and test re-test reliability. Phase 1: PRVBQ Development Study Including Belief Elicitation for Item Generation and Instrument Construction This qualitative study provided the relevancy and comprehensives of the items being used in the questionnaire. Thirty pedestrians using semi- structured open-ended questions were interviewed to elicit salient outcomes, social referents, and circumstances regarding pedestrians’ traffic light violations. Interviews were continued until saturation was reached. Directed content analysis was used by two independent skilled coders to analyze the transcribed interviews and capture salient beliefs. Then, the beliefs obtained from the directed content Fig. 1. Three phases of content validity and reliability study. analysis were subjected to the frequency analysis. The rule of selecting the beliefs expressed by at least of the PRVBQ (Cognitive interview study) 10 percent of the respondents (one of the three rules This stage provided information on the suggested by Aizen and Fishbein) was applied to comprehensiveness and especially the include the most frequent beliefs in the modal set comprehensibility and relevancy of the items. (Detailed information has been provided elsewhere First, four of colleagues were asked to fill in the ). Based on the information obtained from this questionnaire. We provided them a cover letter study and based on the manuals for constructing a consisted of a brief description of research, study TPB questionnaire provided by Aizen and Fishbein title and its objectives, conceptual model, its [16, 21], and Francis et al.’s study , the first draft dimensions, and instructions on how to fill in the of PRVBQ was constructed. The main constructs questionnaire. These provisions have also been of the TPB consisted of ATT, SN, PBC, and provided in other stages of the content validity behavioral intention (BI), plus extended construct and reliability study, where we have provided the of PB, that were used for items formulation. Since questionnaire to pedestrians or experts. Then, the neither approach of direct and indirect measures was entire questionnaire was presented requesting them to perfect, we used both in developing PRVBQ for the review the draft PRVBQ and comment on relevancy, prediction of PRVB (Figure 2). comprehensiveness, and appropriateness of items to the target population. After adaptation based on Phase 2: Content Validity Study Including Face their feedbacks, eight people who were similar to the Validity target population drew up for the cognitive interview. 1- Qualitative evaluating face and content validity The target population was urban pedestrians living Fig. 2. Conceptual model used for PRVBQ development. 100 Bull Emerg Trauma 2020;8(2)  Pedestrian red-light violation questionnaire in the Tabriz city, aged≥18 years, lacking physical Not relevant, 2: Somehow relevant, 3: Quite relevant, and mental disabilities. The concurrent probing and 4: Highly relevant). Item level content validity was used for the cognitive interview. A skilled index (I‑CVI) was calculated by dividing the number interviewer based on the predefined interview guide of experts who gave each item scores 3 or 4 by the asked the interviewees to complete the questionnaire total number of experts participating in the panel. and asked them about their understanding of the Having three different I-CVI for each item in terms PRVBQ instructions, items and response options of relevancy, clarity, and simplicity; we computed to determine manifest complications over items the average I-CVI for each item by adding three for testing the comprehensibility, relevancy, I-CVIs and divided by three. According to Lynn’s acceptability, and feasibility of the measurement criteria, if the number of experts is between 6 and instrument. The technics of think-aloud (respondents 10, I-CVI equal to or higher than 0.78 is considered verbalized their thoughts while reading each to be excellent. Items with an I-CVI between question and chose the answer) and paraphrasing 0.70 to 0.78 were revised and items with a I-CVI (respondents were asked to rephrase an item in his/ lower than 0.70 were deleted. Scale level content her own words) were employed for understanding. if validity index (S-CVI/Ave) was calculated by the item was misunderstood and could be rephrased. adding together the items with an I-CVI above 0.78 Participants provided many recommendations to and divided by the total number of items. S‑CVI/ further improvement of the face and content validity Ave≥0.9 was considered acceptable, according to of the PRVBQ. the recommendation of Polit et al.. As substantial To ensure the rigor of the analyses and prevent item improvement not necessitated, and all aspects of bias, two independent researchers were involved in the construct adequately covered by the initial pool the analyses. All interviews were audio-recorded of items, the second round of CVI study was not and transcribed verbatim. We made necessary conducted. The remaining items were subjected to modifications to the interview guide, questionnaire the CVR study. Then, CVR was calculated for each items, response options and recall period based on item. CVR specifies whether an item is necessary to the respondents’ comments from the first round be included in the questionnaire or not. An expert of the cognitive interviews. Following the first panel (N=8) was requested to specify whether an item round of cognitive interviews, an expert’s panel is necessary for operating a construct in a set of items provided a cover letter and the entire questionnaire or not by specifying each item as “not necessary, to review the PRVBQ and comment on relevancy, useful but not essential, or essential”. CVR was comprehensiveness, and appropriateness to the target calculated by the following formula: CVR=(Ne-N/2)/ population. Necessary modifications were made on (N/2), in which the Ne is the number of panelists the PRVBQ based on the expert comments. The indicating “essential” and N is the total number of second round of cognitive interviews with eight panelists. Lawshe table was used for determining the pedestrians was conducted to confirm the face numeric value of the CVR. Having eight experts, a and content validity. In this round, based on the minimum value of 0.75 was considered an acceptable respondents’ comments, minor revisions were made. level of significance. Then the revised questionnaire was subjected to the quantitative face and content validity. Phase 3- Reliability Internal consistency and temporal reliability 2- Quantitative Evaluating of Face and Content assessed for the reliability of the instrument. Validity of the PRVBQ 2-1- Face Validity 1- Temporal Reliability A group of pedestrians (N=10) after providing them Test re-test reliability was used for assessing a cover letter were requested to rate the importance of temporal reliability. Fifty pedestrians over a two- each item on a 5-point Likert scale. The item impact week interval answered the questionnaire. The score (frequency × importance) was calculated. The correlation between the individual questions impact score of ≥1.5 was considered acceptable. demonstrated the stability of the instrument. For assessing the temporal stability, Intra-class 2-2- Content Validity Correlation Coefficient (ICC) was calculated for Quantitative content validity was calculated by each item. According to Cicchetti , Cicchetti measuring the content validity index (CVI) based and Sparrow , and Fleiss , ICC values were on Waltz and Bausell approach , and the content interpreted for the reliability as follows: values≥0.74 validity ratio (CVR). CVI, the most widely reported were considered excellent, values from 0.60 to 0.74 approach for content validity, was first calculated. were depicted good, values from 0.40 to 0.59 were An expert panel (N=10) in the field of road safety, regarded fair, and values≤0.40 were illustrated as health promotion, and psychometric was requested poor (Figure 3). to score the relevance, and comprehensibility (in terms of clearness and simplicity) of each item using 2- Equivalence Reliability a 4-point ordinal rating scale e.g. for relevancy, (1: As noted earlier, we included direct and indirect www.beat-journal.com 101 Moshki M et al. Fig. 3. Content validity study flowchart. Table 1. The demographic characteristics of the belief elicitation participants. Age (N=30) Sex (N=30) Education (N=30) Occupation (N=30) Marriage (N=30) Crossing (N=30) Mean 4.93 Male 15 Under diploma 1 Public 14 Married 21 Cross 15 employee Median 39.00 Female 15 Diploma 6 Private 2 Single 9 Not cross 15 employee SD 13.06 Associate degree 1 Self-employee 6 Min 21 Bachelor degree 18 Retired 4 Max 75 Master degree 3 Student 2 Doctorate 1 Housewife 2 measures (i.e., reflective and formative indicators, a significance level of 0.05. respectively) in the PRVBQ. Since the behavioral, normative, and control belief composites were Results formative indicators of ATT, SN, and PBC, respectively, there was no requirement of internal Phase 1- PRVBQ Development Study Consists of consistency for them. Therefore, we calculated Belief Elicitation for Item Generation and Draft internal consistency only for direct measures. Instrument Construction Cronbach’s alpha coefficient, the most widely used The demographic characteristics of the belief statistic to assess the internal consistency of a elicitation participants can be found in Table 1. scale, indicated how well the items on a tool jointly measure the same construct [29, 30]. To test internal Belief Elicitation for Item Generation consistency, 50 pedestrians completed a copy of Four to twenty sub-categories were generated the questionnaire. We calculated Cronbach’s alpha through directed content analysis for each of the coefficient to assess the internal consistency of the ten predefined categories of the TPB. Consistent entire questionnaire and for each subscale. Values with the TPB questionnaire, these categories equal or above 0.7 were considered as acceptable were advantages, disadvantages, positive feelings, level [26, 31]. Cronbach’s alpha and ICC were negative feelings, approving referents, disapproving calculated using the Statistical Package for Social referents, behaving referents, not-behaving referents, Sciences (SPSS, version 24.0, Chicago, IL, USA) at facilitators, and barriers. We added the elicited 102 Bull Emerg Trauma 2020;8(2)  Pedestrian red-light violation questionnaire advantage beliefs and disadvantage beliefs together to produce valid responses by the pedestrians. To to create a set of instrumental beliefs. Similarly, to further improve the face and content validity of the create a set of experiential beliefs we combined the PRVBQ, pedestrians provided many constructive beliefs of like and dislike together. suggestions. The revisions were made to fulfill these Finally, all instrumental beliefs and experiential recommendations. beliefs were put together to yield a set of attitudinal beliefs. Normative and control beliefs have been Expert Panel Input made like the same process of attitudinal beliefs. An expert panel reviewed the revised PRVBQ After producing attitudinal, normative and control and made some critical comments on relevancy, beliefs, using the Aizen and Fishbein’s rule of “belief comprehensiveness, and appropriateness to the mentioned by at least 10% of participants”, the most target population. Based on their inputs, necessary frequently mentioned beliefs were identified to be modifications made on the PRVBQ to increase the used for crafting the primary draft of PRVBQ based relevancy and comprehensiveness of the instrument. on the TPB constructs and additional construct of “past behavior” comprised of 86 items in total. Second Cognitive Interview Study The most important outcomes of the PRVB were Overall, the revised PRVBQ, like the first round getting injured, time-saving, breaking the law, of cognitive debriefing interviews, was relevant to disturbing the city system, lowering the level of the participant pedestrians. Although this version culture, violating citizenship rights, and financial was more comprehensive and acceptable than the damage. The most important social referents initial draft version, a few minor problems were were family members, friends, educated people, detected in this version, likely due to the complexity colloquies, relatives, and youth. The most important of the employed conceptual framework. So we circumstances regarding PRVB were “Being in a made necessary modifications to the instrument to hurry” “No police presence” “Fear of accident” increase the comprehensibility of the instructions, “Complying with the law” “Not crossing other items, and response options of the instrument. The pedestrians” and “Physical ability”. revised PRVBQ with 86 items was subjected to the quantitative content validity including face validity. Phase 2- Content Validity Study Including Face Validity of the Instrument Quantitative Evaluating of Face and Content Qualitative evaluating face and content validity of Validity of the PRVBQ the PRVBQ (Cognitive interview study) Face Validity First Cognitive interview study All 86 items of the questionnaire had an impact Cognitive debriefing of the interviews indicated score higher than 1.5 were retained and subjected to that the PRVBQ was relevant to the PRVB, but many the next step of content validity (Table 2). items were needed to be revised to improve the Based on the results of the I-CVI for all 86 items comprehensiveness of items. Response options also of the questionnaire, eight items (i.e., items 3, 6, 10, recognized problematic and needed to be revised 13, 17, 21, 57, and 65), having I-CVI of less than Table 2. Items of PRVBQ Scales before and after face and content validity. Measures Scale Sub-scale Domain (N*) Item deletion Initial After After After Items FV CVI CVR Direct Attitude Behavioral belief Strength 7 7 5 5 Measures Outcome evaluations Cognitive 7 7 5 5 Affect 7 7 5 5 Injunctive Injunctive belief Strength 7 7 7 5 Norm Motivation to comply 7 7 7 5 Descriptive norm Descriptive belief Strength 7 7 7 5 Identification with Referent 7 7 7 5 Perceived Behavioral Control belief strength 8 8 7 7 Control Power of control Factors 8 8 7 7 Indirect Attitude Direct attitude 4 4 4 4 Measures Subjective norm Direct perceived norm 4 4 4 4 Perceived Behavioral Direct perceived Control Self-efficacy 3 3 3 3 Control Autonomy 3 3 3 3 Behavioral Intention Behavioral intention 4 4 4 4 Extended Past behavior Past behavior 3 3 3 3 Measure Total items 86 86 78 70 Content Validity Index (CVI) www.beat-journal.com 103 Moshki M et al. Table 3. Intra-class correlation coefficient (ICC) of the composite beliefs. Item α ICC P value IATa1 0.806 0.675 (0.490-0.802) 0.001 IAT2 0.873 0.774 (0.634-0.865) 0.001 IAT3 0.882 0.789 (0.656-0.874) 0.001 IAT4 0.818 0.692 (0.514-0.813) 0.001 IAT5 0.840 0.724 (0.560-0.834) 0.001 EAT b1 0.841 0.725 (0.562-0.834) 0.001 EAT2 0.884 0.793 (0.661-0.877) 0.001 EAT3 0.855 0.747 (0.593-0.848) 0.001 EAT4 0.817 0.691 (0.512-0.812) 0.001 EAT5 0.864 0.761 (0.641-0.857) 0.001 INOc1 0.780 0.640 (0.442-0.778) 0.001 INO2 0.966 0.934 (0.886-0.962) 0.001 INO3 0.834 0.716 (0.548-0.828) 0.001 INO4 0.835 0.716 (0.549-0.828) 0.001 INO5 0.791 0.655 (0.462-0.788) 0.001 DNOd1 0.788 0.650 (0.456-0.785) 0.001 DNO2 0.805 0.674 (0.489-0.801) 0.001 DNO3 0.882 0.788 (0.655-0.874) 0.001 DNO4 0.856 0.748 (0.595-0.849) 0.001 DNO5 0.829 0.708 (0.537-0.823) 0.001 PBCe1 0.876 0.779 (0.641-0.868) 0.001 PBC2 0.884 0.792 (0.661-0.871) 0.001 PBC3 0.775 0.633 (0.432-0.774) 0.001 PBC4 0.886 0.795 (0.664-0.878) 0.001 PBC5 0.911 0.837 (0.730-0.904) 0.001 PBC6 0.897 0.814 (0.693-0.890) 0.001 PBC7 0.875 0.778 (0.640-0.868) 0.001 a Instrumental Attitude; bExperiential Attitude; cInjunctive Norms; dDescriptive Norms; ePerceived Behavioral Control 0.78 were excluded after careful consideration. The with reliability study. I-CVI of the 78 remaining items was between 0.8 Based on the qualitative belief elicitation study, in and 1 with S-CVI/Ave equal to 0.95. total, 22 beliefs concerning salient outcomes, social referents, and circumstances regarding pedestrians’ Content Validity Ratio (CVR) red light violation has been recognized. Using Eight items (i.e., items 27, 28, 34, 35, 41, 42, 48, 49) these beliefs and based on the TPB constructs and having CVR less than 0.75 were excluded from the additional construct of PB, we formulated the initial questionnaire, remaining 70 items to the reliability form of questionnaire items comprised of 86 items. study (Table 2). Previous nine studies of TPB based questionnaire Phase 3: Reliability assessment including internal studying pedestrian behavior, except one , have consistency and test re-test reliability not reported conducting a belief elicitation study for The Cronbach’s alpha reliability coefficient for item generation [3, 8, 10-14, 32]. the collective 22-item direct measures was 0.90, PRVBQ (86 items) was first subjected to the indicating excellent internal consistency reliability, two-round qualitative cognitive interview with ranged from 0.83 to 0.97. All items showed excellent pedestrians and an expert panel in between. Based agreement, ICC=0.88 (95% CI [0.80, 0.93]). on the results of the two rounds interview and expert panel comments, most items and response options Discussion were subjected to thorough revision. Previous nine studies of TPB based questionnaire studying The present study demonstrated the development of pedestrian behavior [3, 8-14, 32], except Barrero a new questionnaire to understand pedestrians’ red- et al., have not reported conducting a cognitive light violation behavior and assessment of content interview for testing face and content validity. validity and reliability of this instrument. Developing Due to the scrutinized item generation process a measurement instrument with acceptable content and thorough revision of almost all 86 items after validity is an iterative and lengthy process. This cognitive debriefing interview, none of the items was process started with instrument development, deleted due to item impact score lower than 1.5. Nine including belief elicitation for item generation items of the questionnaire with an I-CVI