Evaluating Research in Academic Journals – A Practical Guide to Realistic Evaluation – Eighth Edition

Evaluating Research in Academic Journals – A Practical Guide to Realistic Evaluation – Eighth Edition
Maria Tcherni-Buzzeo and Fred Pyrczak
Contents14. Evaluating Systematic Reviews and Meta-Analyses:
Towards Evidence-Based Practice 231

15. Putting It All Together 250
    Concluding Comment 256
    Appendix A1: Research Ethics: Two of the Most (In)famous Studies 257
    Appendix A2: Research Ethics: An Egregious Case that Led to
    Children’s Deaths 262
    Appendix B: Program/Policy Evaluation 265
    Appendix C: Limitations of Significance Testing 268
    Appendix D: Emerging Issues in Survey Research 273
    Appendix E: Checklist of Evaluation Questions 276
    Index 286
    Contents
    Index
    Page numbers in italics indicate a figure; page numbers in bold indicate a table
    acronym 37
    action research (or action-oriented research) 11,
    121–123; sampling (or participant inclusion) in 127,
    133–134
    aggregate units 158
    anonymity see response format
    applied research 2, 7–8, 11-12
    article: empirical article (or research report)
    12, 18; non-empirical article 13; subtitle 44–46
    attrition 107; bias (or differential attrition)
    164–165; see also meta-analysis
    auditor see qualitative research
    autoethnography see ethnography
    before-and-after-design see quasi-experiment
    big data see data
    blind (or blinded) experiment see experiment
    case study 123–124; sampling in (or selection rationale)
    128, 134
    causality 266; cause-and-effect relationships in article
    title 42–44; see also experiment
    checklist of evaluation questions 276–285
    Cochrane Library 237–238, 244
    concept(s): conceptual definition 75, 92–93; definitions
    (of key terms) 29
    confidentiality 115–116, 135–138, 146; see also
    response format
    confounding variable (or confounder) see variables
    content analysis 195
    content validity see validity
    convenience sample see sampling
    convergent design see mixed methods research
    cost–benefit analysis see evaluation research
    cost–effectiveness analysis see evaluation research
    criticism balanced 86–87; positive 85; negative 84–85
    Cronbach’s alpha see reliability
    cross-sectional research 14-15, 51
    data 96; big data 97, 274; data saturation see saturation;
    data triangulation see triangulation; dredging (or data
    mining) 271; secondary 140
    descriptive research 3–5
    descriptive validity see validity
    definition (of concept) see concept
    demand characteristic 162–163
    demographics: comparing demographics of survey
    participants and non-participants 106–107; of the
    sample 110–111, 135, 183, 213
    descriptive research 70
    descriptive statistics 177, 181
    direct quotes (in literature reviews) 80–81
    discussion 221
    double-blind experiment see experiment
    e.g. (use of in academic writing) 77
    effect(s): of treatment (expressed as difference between
    groups) 181, 269; size 231, 272; use of “effect(s)”
    in article titles 42–43; effectiveness of a program or
    policy see evaluation research
    emerging issues in survey research 273–275
    empirical research (or empirical study) 12; empirical
    article see article
    empirical validity see validity
    ethics review of research 114, 174–175; informed
    consent 115, 137, 164; (potential) harm to participants
    114; unethical research practices 257–264, 271
    ethnography (or ethnographic research) 2,
    119–120, 131; autoethnography 119–120; sampling
    (or participant selection) in 127, 131
    evaluation research 7–11, 70, 265; cost–benefit analysis
    265, 266; cost–effectiveness analysis 265, 266;
    286Index
    efficiency (of a program or policy) 266; impact
    assessment 265; implementation (of a program or
    policy) 266; process evaluation (or process analysis)
    265–266; program theory 267
    evidence-based practice (or evidence-based policy) 232,
    247–248, 265, 267
    experiment(s) 11, 43, 156; blind (or blinded) 163;
    double-blind 161–162; multiple baseline design 169;
    multiple-treatment interference 169; natural
    (or field) 167, 169–170; person effect 171; placebo
    surgeries (or sham surgeries) 161–162, 166; size of
    groups in 163; true (randomized controlled trial) 163
    experimental mortality see attrition
    explanatory research 5–8, 30, 108; design (or sequential
    explanatory design) 207–208,
    also see mixed methods research
    exploratory research 2–5, 71, 145; design (or sequential
    exploratory design) 207–208,
    also see mixed methods research
    ex post facto study see quasi-experiment
    external validity see validity
    face validity see validity
    field experiment see experiment
    field research 14–15, 119
    file-drawer problem (or publication bias) see
    meta-analysis
    flaws in research see limitations
    gaps in research see research
    generalizability (generalization) 96, 110, 112, 116, 160;
    see also validity (external validity)
    grey literature 235
    grounded theory see theory
    harm to participants see ethics review of research
    Hawthorne effect 147; see also observation
    heterogeneity (in qualitative research) 189;
    also see meta-analysis
    impact assessment see evaluation research
    implementation (of a program or policy) see evaluation
    research
    implications of research for practice 225–227, 247–248
    (in)famous (unethical) studies 257–264
    inferential statistics 177, 181
    informed consent see ethics review of research
    Institutional Review Board (IRB) see ethics review of
    research
    integrated findings see mixed methods research
    intent-to-treat (ITT) analysis 165
    internal consistency (or internal reliability) see reliability
    internal validity see validity
    interpretative validity see validity
    inter-rater (or inter-coder) reliability see reliability
    journal impact factor (journal quality) 22
    limitations (flaws): in research 26–27, 79–80, 222–223;
    of measures 24, 145, 153–54; of sampling 24–26,
    110; of significance testing 268; of systematic
    reviews and meta-analyses 246–247; unavoidable
    flaws 252
    literature review 65
    longitudinal research (or longitudinal study) 14–15, 107,
    146, 164
    mailed survey see survey response rate
    mean, misleading 179–180
    measurement 23; instruments 23; see also reliability;
    validity
    measures 140; self-reported 166; see also reliability;
    validity
    median 180
    member checking see qualitative research
    meta-analysis 9, 231; attrition bias in meta-analyses 241;
    heterogeneity of studies included in 240; publication
    bias (or “file-drawer problem”) 241; selective
    reporting bias 241
    meta-synthesis 231
    method(s) 96
    Milgram’s Experiments on Obedience
    258–261
    mixed methods research 206–207, 273; convergent
    design 207; integrated findings 214–216; sequential
    explanatory design 207, 219; sequential exploratory
    design 207
    mode 180
    multiple baseline design see experiment
    multiple-treatment interference see experiment
    natural experiment see experiment
    negative cases (exploring negative cases) 197
    non-empirical article see article
    null hypothesis 266
    observation 23; direct observation and changes in
    behavior 147; see also measurement
    online survey see survey response rate
    operationalization 141
    palpability 189, 201–203
    participants 96; recruitment (in qualitative research)
    192–193
    participation rate 103–105
    percentage 177–178
    person effect see experiment
    phenomenology (or phenomenological study) 120–121,
    sample selection in 127, 132
    pilot study 26, 125–126; sample selection 128, 135
    placebo surgeries see experiment
    287Index
    plain language summary 244
    policy evaluation 265–267
    population 96–109, 126–129, 268–269
    positionality 188, 199–201
    practical significance (as opposed to statistical
    significance) 270–271
    probability sampling (or random sampling) see sampling
    process evaluation (or process analysis) see evaluation
    research
    program evaluation 265–267
    program theory see evaluation research
    proof 31; degrees of evidence 79
    publication bias see meta-analysis
    “publish or perish” pressure 271
    purposive sample see sampling
    qualitative research 126, 188; coding of data in 195–196;
    collection of data in 142, 143–145; member checking
    197, 201; sampling in 130–131; use of quotations
    (thick descriptions) 201–202; see also content analysis
    quantitative research 177
    quasi-experiment 43, 161, 163–170, 173, 237; beforeand-after-design 169; ex post facto study 43, 167;
    single subject research (or behavior analysis) 168–169
    random: assignment 159–160; random assignment vs
    random sampling (or random selection) 159–160, 160;
    see also experiment; random sampling see sampling
    randomized controlled trial see experiment
    reflexivity 188, 197, 199–201, 211, 250–251
    reliability 144; internal consistency (measured by
    Cronbach’s alpha) 148–149; inter-rater (or inter-coder)
    147–149, 196–197, 210; split-half 149; temporal
    stability (or test–retest reliability) 150–151
    replication 31, 117; crisis 270–271
    representative sample see sampling
    research: checklist of evaluation questions 276–285;
    contradictory research findings 83–84; cross-sectional
    vs longitudinal studies 14; gaps in research literature
    87, 251–252; emerging issues in survey research 273–
    275; research article (or research report) 1
    response format (in a questionnaire) 141–142;
    anonymous responses 146; confidential responses
    146; response style bias 145
    samples/sampling: aggregate-level sampling units 41;
    biased 102; convenience 24, 102, 107–108, 125;
    in qualitative research 188–193; non-random 102;
    purposive 25, 126, 126–128; random (or probability)
    99, 268; representative 96, 109; sampling error
    268; saturation 126, 130, 191–192, 194, 197;
    simple random 102; size of 112–113,
    130–131; stratified 101–102; unbiased 99
    saturation see samples
    secondary data see data
    selection bias (or self-selection bias) 26, 104, 107
    selective reporting bias see meta-analysis
    sequential explanatory design see mixed methods research
    sequential exploratory design see mixed methods research
    sham surgeries see experiment
    significance testing see statistical significance
    simple random sampling see sampling
    single-subject research (or behavior analysis) see
    quasi-experiment
    size of groups in experiments see experiment
    size of the sample see sampling
    skewed distribution 179–180
    social desirability bias 145
    Stanford Prison Experiment (SPE) 107, 160, 257–258,
    259–261
    statistical significance 113, 181; significance testing
    268–272
    statistics: presented in a table and in text
    182–183; see also descriptive statistics
    stratified sampling see sampling
    subjects see participants
    substantive significance (as opposed to statistical
    significance) 270–271
    survey research 14; emerging issues 273–275
    survey response rate 103–104
    systematic review 231
    temporal stability (or test–retest reliability) see reliability
    theory 5-7, 30, 125, 193; developing and testing 5,
    125–126; grounded 71, 193, 194, 196, 207;
    implications of study results for 228; mention in
    abstracts 55–56; mention in introductions 69–70;
    mention in titles 41–42
    thick descriptions see qualitative research
    translational research 12
    treatment see experiment; see also variables
    triangulation 144–145, 194, 197
    true experiment see experiment
    unbiased sample see sampling
    validity 144; content 151; descriptive validity
    188–189, 197; empirical 152–153; external
    173–174; face 152; increasing validity through the
    use of multiple measures 144–145; internal 160, 174;
    interpretative validity 188–189, 197; issues unique to
    mixed methods research 215–216; theoretical validity
    188–189; with regard to self-reports of sensitive
    matters 145–146
    variable(s) 39; confounding (or confounder) 161;
    dependent, or response 156; independent, or stimulus
    (often referred to as treatment) 156
    Wakefield (Andrew) study on vaccines and autism

كلمة سر فك الضغط : books-world.net
The Unzip Password : books-world.net