NR449 W6 Review

Introduction to Evaluating Quantitative Data

• Analyzing quantitative data gives easily understood results for researchers
• Quantitative data is either descriptive or inferential depending on the study goals
• Evaluating descriptive data is important for learning to evaluate quantitative data
• Quantitative data aids researchers and readers in seeing if interventions are effective for practice
• Levels of measurement impact the analysis of quantitative data
• There exist differences between statistical and clinical significance
• Procedures influence the data's narrative
• Appropriate decisions are critical based on the analysis results
• Misrepresented or incorrect data impacts nursing practice

Common Elements in Quantitative Analysis

• Understanding critical elements is important when analyzing quantitative data
• Raw data is stored either manually or electronically for retrieval
• Raw data is placed into a dataset and structured based on the needs
• Data preparation and cleaning ensures data is easily manipulated
• Tabulation presents information in a table form with rows and columns; charts and graphs help show trends and patterns
• Summary statistics involves statistical processes like frequency distributions, averages, dispersion and correlations
• Descriptive analysis identifies patterns in data using cross-tabulations, variable comparisons, and subgroup analysis
• Statistical Analysis of Differences and Associations includes confidence intervals and statistical testing
• Presentation of Data and Analysis depends on the audience, using tables and statistics or descriptive procedures

Research Process

• Involves theory, hypothesis, research design, devising measures of concept, selecting a research site, and respondents
• Includes data collection, process data, analysis and interpretations, findings/conclusions, and report writing

Quantitative Analysis General Rules

• Represented by numbers, the reliability of data collection and accuracy are critical for correct conclusions
• Planning and reporting are determined when the methods and procedures are designed
• Statistical tests should be selected before an experiment to minimize researcher bias
• Examples of tests include ANOVA, chi square, regression analysis, and z or t tests
• Test selection is based on factors such as research question, groups tested, and variable measurement levels
• Researchers must run all identified tests to reduce bias
• Researchers must report all test results, as selecting specific results can be unethical

Goals of analysis

• Evaluating differences between groups
• Assessing the nature and direction of relations between subjects or variables
• Capacity to predict outcomes and sort data
• Data reduction by grouping variables into classifications

Data assumptions

• Parametric tests assume data fit a specified distribution (normal bell-shaped)
• Nonparametric tests do not assume or rely on specific data distribution
• Robust tests yield reliable results even if the assumptions are violated
• Univariate analysis uses single variable analysis (descriptive statistics) or a single dependent variable analysis (inferential analysis)
• Bivariate analysis examines the relationship between two variables, used to see if one variable can predict a specific outcome
• Multivariate analysis allows simultaneous analysis of multiple variables

Evaluating the Outcomes of a Research Study

• Readers evaluate methods, procedures, and results
• Methods and procedures identify each test planned and executed
• Results or findings list relevant results and numbers for understanding outcomes
• Test statistics and p-values should be noted
• Degrees of freedom represent the effects on sample size
• Confidence intervals determine the intervention's effect
• Graph and table inclusion to support results
• P-values are identified as actual probability of error, or the value listed for statistical significance

Statistical Significance

• Required when using evidence in practice, but shows minimal information on client care outcomes
• Statistical significance measures quantify the probability of research results being due to coincidence
• A "p" value ≤ 0.05 measures statistical significance; a value over 0.05 indicates statistical significance; a value over 0.05 indicates probability of error
• Profoundly dependent on the study’s sample size; with large or small intervention effects, it can appear statistically significant

Clinical Significance

• Relies on inferential statistics for strong evidence in interventions impacting client outcomes
• Refers to the magnitude of treatment effect determining the impact on clinical practice
• Dependent on implications for existing practice-treatment effect size, influencing treatment decisions
• Change should make a tangible impact to participants’ lives and be cost effective

Quantitative Analysis Errors

• Misinterpreting statistical significance as clinical significance is a common error
• This misinterpretation could mean that the expected interventions might not produce the expected outcomes in the clinician practice
• Sampling errors or random events can cause misleading results

Factors that can undermine the value of quantitative research include:

• Inaccurate or misleading findings
• Dishonest responses
• Data not always representing change
• Lack of clarity
• Inferring results beyond the sample
• Degree of interpretation

Introduction to Evaluating Qualitative Data

• Qualitative data differs from quantitative as it is presented in a non-numeric form
• Nurse researchers gather data via methods such as focus groups, interviews, questionnaires, or observations
• Qualitative data relies on the nurse researcher’s ability to interpret data to find meaning
• Qualitative data analysis is an active process to ensure trustworthiness to the reader
• Qualitative analysis can change and adapt as the study and data evolve
• Developing an understanding the qualitative analysis process leads to understanding standards that guide qualitative research
• Identifying a systematic approach to analyzing utilization of data in practice
• Determines how data misrepresentation has an impact

Qualitative Analysis

• Complex and vast, qualitative data is reduced into meaningful elements
• Ensuring reliability to draw qualitative inferences is critical for evidence supporting a holistic view of nursing

Qualitative Analysis Characteristics

• Researchers follow actions during qualitative analysis, going back and forth between actions as data comes in
• Data collection methods include focus groups, interviews, questionnaires and observations

Standout characteristics include:

• Constant comparison of new findings to existing ones to support or reject conclusions
• Theoretical sampling involves recruiting additional sample members to loosen inclusion criteria

Qualitative Analysis Process

• Qualitative data analysis is an interactive process where researchers use cognitive processes in phases to evaluate patterns and themes
• Comprehending encompasses the researcher’s ability to make sense of the data to gain a tone
• Synthesizing occurs when a researcher sifts through data using inductive reasoning to begin piecing data together
• Theorizing starts when the researcher gets to the point of what has emerged from the data
• Recontextualizing involves different settings for the theory

Qualitative Analysis Themes and Codes

• Qualitative data must be organized and managed for themes and codes throughout the collection process
• Reducing raw data involves performing a review of the data with no presumptions
• Identifying themes involves providing tones, depth, and general themes from participants
• Comparing themes involves a comparison between the themes for clarity
• Creating a coding scheme involves a schematic, codebook, and unit of analysis
• Reliability of coding schemes uses themes throughout data to exhibit collection methods

Reliability and Validity of Qualitative Analysis

• Qualitative data analysis should be reliable and valid for trust
• Applied criteria include credibility, dependability, confirmability, and transferability
• Credibility involves time, triangulation and external checks

Dependability

• Confirming inter-rater or inter-coder reliability between coders
• Cohen’s kappa measures agreement and probability of error
• Inquiry audit is a review of data, documents, procedures, and results by an external reviewer

Confirmability

• Two or more independent researchers achieve congruence
• Audit trail is a detailed description of the researchers’ decision rules for categorization and inferences

Transferability

• Findings can be transferred to other settings
• Thick description gives richness and details of the setting and transactions

Qualitative Analysis Reporting and Conclusions

• Reporting reflects how well the study fits the data
• Researchers include literature support and direct quotes from participants
• Conclusions include the sample, setting, coding, and description
• Researches will include direct quotes for illustration and summarize by recommending future research

Evaluating the Qualitative Analysis

• Complex for the reader compared to quantitative studies
• No defined rules to verify data validity
• The reader must go on faith that the researcher conducted the study appropriately
• The reader identifies approaches, procedures, and methods used to support credibility

Qualitative Analysis in Nursing Practice

• Can capture experiences of healthcare teams, including client and family
• Nurses reviewing the study identify themes relating to the overall health of the client
• Case studies identify client need areas related to ethnography, phenomenology, and grounded theory

Aspects evident to the nurse reader include:

• A detailed description of the analysis method Independent Review
• Triangulation
• Described coding procedures
• Peer or external audits if they are reported
• Studies are beneficial if research has rigor

Quantitative Data Analysis

• Inferential analysis determines how confident one can be that an intervention caused the outcome
• Quantitative analysis relies on the probability of error, and certainty of the estimates
• Tests should be selected a priori, run all identified tests, and report all test results
• ANOVA, regression, t, or Z tests are examples of statistical tests

Goals, Assumptions and Variables

• Analysis evaluates group differences and relationships between variables
• Analysis predicts outcomes and sorts data
• Parametric tests need data to fit a normal bell-shaped distribution
• Nonparametric tests do not rely on specific distribution patterns
• Univariate, bivariate, and multivariate analysis are based on the number of variables used

Overview of Quantitative Analysis

• Inferential analysis determines if a specific result can be expected in a larger population
• Standard error is an error arising from the sampling process

Statistical Significance

• Compares differences to standard error and reports as p value
• Very small p value = statistical significance

Clinical Significance

• Determines the extent to which an intervention can make a real difference in patients’ lives
• Point estimates, confidence intervals, and effect size are components of clinical significance

Study Designs

• Quantitative analysis is used on experimental, quasi-experimental, causal-comparative, and case-control study designs

T Test Information

• If an outcome can be expressed as mean, a z or t test can be used
• Three versions of the t-test include: one-sample, independent-samples, and paired-samples
• T-test results determine the effectiveness of an intervention

Chi Square Test

• A chi-square test is utilized for tests that do not use the mean
• A chi-square test includes the test of model fit, independence, or association
• Assumptions should include data that are not normally distributed, and variables that are measured in a categorical manner

ANOVA

• ANOVA is one of the most used statistical tests
• ANOVA is effective for studies utilizing experimental and quasi-experimental designs
• ANOVA may determine if factors have interaction effects
• Factors are broad categories by which subjects are categorized into levels
• Univariate, repeated-measures, analysis of covariance (ANCOVA), and MANOVA are types of ANOVA

Non-Normal Data

• Nonparametric tests are used for data that are ranked, ordinal, or skewed
• Mann–Whitney U test, Wilcoxon Signed Rank test, and Kruskal–Wallis test are examples

Sections of a Research Report

• Reports should include descriptive statistics about the sample and variables
• They analyze sample subgroups for group equivalency
• Statistics evaluate error and magnitude of effect
• Numbers should reflect confidence in estimates

Practicing Quantitative Results

• Determine if a result is due to error, and whether a statistically significant finding is clinically significant
• A systematic appraisal is needed

Creating a Quantitative Analysis

• The level of measurement, statistics used to represent variables, number of groups to compared, and availability of statistical software are key factors
• The research question is important

Utilizing Quantitative Results

• Inferential analysis allows researchers to draw conclusions about a population
• Statistical significance determines results are not due to standard error
• Clinical significance shows if an intervention is useful
• Nurses should focus on statistical selection, key numbers that reflect error and level of certainty

Analysis and Reporting of Qualitative Results

• Qualitative studies generate large amounts of data that must be reduced to meaningful units
• Qualitative research findings should meet trustworthiness criteria
• Steps in qualitative research include preparing the data, conducting an in-depth analysis, representing the reduced data, and interpreting the larger meanings

Characteristics and Styles of Qualitative Analysis

• Constant comparison of new to existing data aims to confirm or refute previous conclusions
• Theoretical sampling involves selecting a second sample to provide diverse viewpoints, using less restrictive selection criteria
• Template analysis develops a template for sorting data
• Editing analysis interprets text to find meaningful segments
• Immersion/crystallization employs total immersion and reflection

Qualitative Analysis Process Steps

• Comprehending involves attempting to make of the collected data
• Synthesizing uses inductive reasoning
• Theorizing identifies the most parsimonious explanation
• Recontextualizing applies the theory

Managing Qualitative Data

• Data is reviewed to identify the classification system, and develop codes and a codebook
• Data is coded, and codes evaluated to identify themes

Software and Qualitative Benefits

• Benefits include the ability to import/store data, deidentify data, streamline analysis, experiment with coding schemas and apply standardized rules
• Limitations concerns include dehumanization of the process, the sacrifice of depth for breadth, and unrealistic expectations of software capability

Enhancing Reliability and Validity

• Time spent, triangulation, and external checks increase credibility of results
• Cohen's kappa measures intercoder reliability
• Inquiry audit reviews data, procedures, and results
• Audit trail is a detailed description of decision rules used in analysis
• Thick description enhances transferability

Qualitative Report Writing

• Reports are in first person, and contain an informal writing style with many quotations
• Themes and subthemes are described
• There should be a description of overall implications for nursing practice

Qualitative Analysis Standards

• Requires transparency, linking research questions, tradition, data collection and analysis

Using Qualitative Analysis in Nursing

• Nurse researchers focus on themes, scrutinize for trustworthiness, and rate results by level of evidence

Areas for Practice Based Evidence

• Investigates Practice-based interventions, causal mechanisms, implementation strategies, approaches to adaption, "how to" guidance, contextual factors, and unanticipated effect

Qualitative Analysis Creation

• Analysis begins early by establishing a goal, organizing data and reading for tone/meaning
• Codes are created

Code Creation Guidelines

• Codes are created from documents and narrowed down until categories are created, labeled and a code book is designed

Key Concepts to Apply

• A significant amount of qualitative data must be managed
• The goal is deriving meaning from data gathered
• Although no single standard exists, some steps are common
• Automated coding systems assist with data management
• Qualitative results are reported as themes