Skip to main content

Advertisement

Table 1 Summary of BUILD Multi-methods Longitudinal Evaluation Design

From: A participatory approach to evaluating a national training and institutional change initiative: the BUILD longitudinal evaluation

  Data Sources Evaluation Question(s)a Sample Data Collection/ Timing (grant years) Data Analysis
Student HERI: The Freshman Survey (TFS) S:2 - Representative sample (race/ethnicity) of 500 Students per year, per institution for 10 BUILDs (n = 5000 per year, per survey), non-proportional based on major (80% Biomedical major, 20% non-Biomedical major), mix of BUILD/non-BUILD exposure (all student surveys) (ICS only for rising Juniors and Seniors). - Comparable non-BUILD sample from non-BUILD institutions for TFS and CSS (n = TBD) Fall: Y2-Y5 - Multilevel analysis - Assess degree of exposure effect of BUILD intervention
HERI: College Senior Survey (CSS) Spring: Y3-Y5
CEC BUILD Interim College Survey (ICS) Fall: Y3
CEC BUILD Student Annual Follow-up Survey (S-AFS), rising Sophomores, Juniors, Seniors Spring: Y3-Y5
Faculty HERI Faculty Survey (FAC) F:2 - 50 Faculty per institution for 10 BUILDs (n = 500 total), all from Biomedical disciplines, mix of BUILD/non-BUILD exposure - Comparable non-BUILD sample from non-BUILD institutions for FAC (n = TBD) Fall: Y3 - Multilevel analysis - Assess degree of exposure effect of BUILD intervention
CEC BUILD Faculty Annual Follow-up Survey (F-AFS) Spring: Y3-Y5
Mentee-Mentor Assessment Survey - Student/Mentee assessment of mentor (n = 1–2 mentees per BUILD faculty, per year) Spring: Y3-Y5
Cross Cutting (Student, Faculty, Institutional) Integrated Postsecondary Education System (IPEDS) Data S:2; F:2; I:1 - 10 BUILDs and comparable sample from matched non-BUILD institutions Pre-Implementation and Y1-Y5 - Descriptive analysis
Offices of Institutional Research (IR) Data S:2; F:2; I:1 - 10 BUILDs TBD with Sites Y1-Y5 - Multilevel analysis
Program Implementation Data (CEC Tracker) S:1; S:2; F:1; F:2 Ongoing Y1-Y5 - Descriptive analysis
Site Visit – Interviews S:3; F:3; I:2; I:3 - 10 BUILDs (including Pipeline & Research Partners) Y3-Y4 - 4-phase coding process - Steps to ensure credibility, trustworthiness, & inter-rater reliability
Site Visit – Observations S:3; F:3; I:2; I:3
Document Analysis S:1; F:1; I:2; I:3 - 10 BUILDs (Websites, Presentations, Reports, etc.) Ongoing Y1-Y5
  1. aEvaluation Questions at the Student (S), Faculty (F), and Institutional (I) Levels, as outlined in the body of the manuscript