- Open Access
Advancing inclusive science and systemic change: the convergence of national aims and institutional goals in implementing and assessing biomedical science training
© The Author(s). 2017
- Published: 4 December 2017
Background and purpose
National reports call for improving America’s leadership in scientific research, accelerating degree attainments, and diversifying the scientific workforce to foster innovation. However, slow progress and persistent disparities across growing U.S. populations are evident on key science workforce indicators, from degree attainment to career achievements. The purpose of this article is to provide a conceptual basis and overview of a national effort funded by the National Institutes of Health (NIH) that advances inclusive science practice and systemic change. We introduce the context, features, and rationale that drive practice and evaluation in the Diversity Program Consortium (DPC) approach, which is an experimental program to implement and evaluate evidence-based and novel practices to expand and diversify the biomedical workforce.
Despite decades of federal investment for biomedical research training, researchers identified disparate adjusted rates of R01 grant awards by scientists’ race/ethnicity. This motivated NIH to fund the DPC approach as a set of highly integrated initiatives that empower institutional change agents to create scalable, evidenced-based strategies to enhance diversity in biomedical research and health science training. Key DPC elements include: 1) A systemic approach to enhance science preparedness involving students, faculty, and institutional-capacity development; 2) Collaboration, partnerships and networks across individuals and organizations, and especially between NIH, 10 undergraduate Building Infrastructure Leading to Diversity (BUILD) sites, the National Research Mentoring Network (NRMN), and the Coordination and Evaluation Center (CEC); and 3) Increased focus within and across key career stages for expanding training and ultimately diversifying the scientific workforce. A new framework for inclusive science practices and discussion of systemic change challenges provide insights into DPC processes and activities.
Collectively, the DPC establishes a national learning collaborative to implement and evaluate multidimensional components of training and program interventions, accelerate the adoption of promising or effective practices, and disseminate lessons to the broader extramural scientific community. Linking practice with evaluation research will identify exemplars that others may adopt to advance the goals of inclusive science in promoting and sustaining innovation, accelerating equity in science careers and, ultimately, address challenging health problems in an increasingly diverse nation.
At the close of the World War II, a report to the President of the United States urged the federal government to “accept new responsibilities for promoting the flow of new scientific knowledge and the development of scientific talent in our youth…for they vitally affect our health, our jobs, and our national security” . The Research Grants Office was created at the National Institutes of Health (NIH), shortly thereafter, to facilitate federal investment in a program of extramural research grants and fellowship awards . National aims for advancing scientific research and training accelerated after the launching of Sputnik by Soviet scientists in 1957, an event widely recognized as the catalyst for expansion of the U.S. higher education system, federal investment in scientific research, and financial aid to college students to increase access to postsecondary institutions . Today, the key areas of health, jobs, and national security remain critical foci that require investment, innovation, and the development of scientific talent at all levels of training. Because the majority of scientific research training occurs in postsecondary institutions that operate rather autonomously from the U.S. government, aligning national aims and institutional goals becomes essential. This is often accomplished through competitive funding for research and postsecondary training grants that reflect national priorities.
However, significant disparities across diverse populations from undergraduate education through early career stages called for a bold new approach to training and sustaining science productivity . Persistent disparities across growing U.S. populations are evident on key science workforce indicators, from degree attainment to career achievements. For example, the National Science Foundation (NSF) reported that among doctoral degrees awarded in science, mathematics and statistics, engineering, psychology, and social sciences in 2012, women earned only 40%, blacks/African Americans 6.3%, and Hispanic/Latinos 6.5% . Despite the existence of interventions, progress towards achieving a more diverse scientific workforce has been slow relative to the rapid increases in a demographically diverse U.S. population. The U.S. is projected to become a majority minority nation, with only 44% non-Hispanic whites by 2044 . The purpose of this article is to provide an overview of the NIH-funded Diversity Program Consortium (DPC), which is a national effort to develop innovative models and institution-wide reform for biomedical training. We introduce the background, structures, key features and rationale that drive implementation and evaluation efforts to diversify U.S. biomedical science research training, as well as begin to define inclusive science and systemic change as it is reflected in practice.
Several key trends in the last decade set the stage for a national impetus for change and the development of the NIH-sponsored Diversity Program Consortium (DPC) as a renewed effort to address expansion and diversification of the biomedical research workforce. First, a succession of national reports documented the state of scientific training and research with calls to action. A widely-publicized 2007 report emphasized fading U.S. leadership in research and the production of STEM degrees, recommending increases in the percentage of all 24-year olds with first degrees in science to at least match the levels of other countries . This resulted in legislation called the America COMPETES Act that included additional funding to support all scientific research and training programs. A subsequent Congressionally-mandated report in 2011 focused on reducing the severe disparities for underrepresented groups (URGs) that compose a growing share of the American population . It recommended a comprehensive and coordinated approach, as well as financial assistance, to triple the number of URG graduates earning a first degree in natural sciences or engineering. Science advisors to the President also released a 2012 report that set new targets for a million more STEM graduates over the next decade , with a focus on aligning institutional practices that engage students to meet the national goal. Major associations also responded with their own calls for change to improve science teaching and faculty engagement with undergraduates [10, 11]. More recently, a 2016 National Academy of Sciences, Engineering and Medicine (NASEM) report documented the varied postsecondary educational pathways for diverse students and recommended “a series of interconnected evidence-based approaches to create systemic organizational change for student success” in science training . These reports recommend government and foundation investment, and more importantly, coordinated institutional action to improve practices that address instruction, research training, and career mentoring to advance excellence and equity in the scientific workforce.
A related trend is that social science research has informed many of these national reports in terms of evidence-based practice, and strengthened the rationale for the value of diversity in science training and the biomedical workforce. Research on the benefits of diversity in the last 15 years identified the cognitive and democratic benefits of education with diverse peers for college graduates [13–15]. Benefits of diverse work environments have been documented in terms of innovation, creativity, and improved performance outcomes [16–19]. Collaboration on diverse work teams also results in higher impact publications, presumably due to broader perspectives that improve research quality and diverse networks that increase citations . Social science research has also begun to unravel the accumulative processes that explain disparities perpetuated through hostile climates, implicit bias, and stereotypes, which affect decision-making and undermine talent development [21, 22]. Moreover, studies show that career advancement can be particularly stagnant for URGs without a strong network of mentors . Mentoring plays a key role in the successful academic socialization of new faculty . Social science research provides the evidence-based rationale for adopting more inclusive practices in science, increases awareness about the lack of progress, and ignites institutional reform.
A prime example of how social science research spurs reform is the result of an NIH-funded study that served as a catalyst for introspection at NIH, plans for the DPC, and continuing discussion on diversity and disparities in scientific circles [24–26]. Ginther and colleagues focused on examining whether scientists of varying races and ethnicities, with comparable research records, have similar likelihoods of having awarded NIH R01 grant applications awarded . The R01 mechanism is the oldest and primary way that NIH makes investigator-initiated grants, accounting for over half of all external research grant funding [27, 28]. Results indicate that applicants with good scores on their peer-reviewed proposals were more likely to be funded, but there were significant differences in award probability by race and ethnicity. After controlling for applicant’s educational background, country of origin, training, previous research awards, publication record, and employer characteristics, black applicants were 10.4 percentage points less likely to be awarded R01 grants compared to white applicants. Black and Hispanic investigators were also less likely to resubmit an unfunded application. The authors suggest that small differences in access to research resources and mentoring may result in a cumulative advantage for white investigators that creates large between-group differences, disadvantaging underrepresented investigators despite achievements that qualify them as principal investigators for a major research grant. This widely-publicized research served as an impetus for the formation of a working group of the Advisory Committee to the Director (ACD) at NIH, which reviewed the study and engaged in its own analysis to come up with recommendations to not only examine review processes but also level the playing field and address disparities across career stages . Changes in URG disparities in award rates would require a more systemic approach to opportunity and talent development.
In June 2012, the Working Group on Diversity in the Biomedical Research Workforce (WGDBR), through the NIH ACD, provided concrete recommendations to improve the recruitment and retention of talented individuals underrepresented in biomedical research and prepare them for successful biomedical research careers . The focus is on persons from underrepresented racial and ethnic groups, individuals with disabilities, or from disadvantaged backgrounds as identified in government statistics , and groups in the latest NSF report on Women, Minorities, and Persons with Disabilities in Science and Engineering . The Working Group provided recommendations designed to evaluate issues related to a disparity in success rates for research grant (R01) applications between White and black applicants in review processes; and ultimately broaden participation of diverse individuals in biomedical fields using more systemic strategies that are transformative and sustainable. The recommendations set an NIH-wide priority to address new comprehensive strategies that include training, mentoring, and research engagement/support for students and faculty as well as resources for improving institutional capacity.
In one of many responses to WGDBR recommendations, the NIH launched a set of experimental programs in 2014 to implement novel and innovative initiatives to engage and sustain individuals underrepresented in biomedical research training and careers, while simultaneously building infrastructure and developing faculty that would support biomedical research activities [31, 32]. The multipronged approach was intended to sustain effective innovations introduced on and across campuses. Funded at $250 million for 5 years, this trans-NIH Common Fund  program established three highly-integrated initiatives: the Building Infrastructure Leading to Diversity (BUILD) initiative (RFA-RM-13-016) , the National Research Mentoring Network (NRMN) (RFA-RM-13-017)  and the Coordination and Evaluation Center (CEC) (RFA-RM-13-015) . Collectively, the BUILD, NRMN, and CEC grantees work together with the NIH as the Diversity Program Consortium (DPC). Throughout the funding period, grantees implement local intervention strategies as well as a consortium-wide approach to assessing key Hallmarks of Success as outcomes for students, faculty and institutions. The NIH uses a cooperative agreement funding and management mechanism (U54 and UL1) that involves substantive scientific involvement of the grantee in collaboration with NIH. The DPC establishes a national learning collaborative to investigate the multidimensional components of training and program interventions, accelerate the adoption of promising or effective practices, and disseminate lessons NIH-wide and to the broader extramural community.
In addition to the BUILD, NRMN, and CEC programs, the Advisory Committee to the Director (ACD) Working Group on Diversity (WGD) was formed in response to the WGDBRW recommendations  (not shown in Fig. 1). As a permanent working group, the ACD WGD provides advice to the ACD and, in turn, the NIH Director about programs such as the DPC and “effective strategies to increase the representation of diverse individuals underrepresented nationally in biomedical research and to reduce disparities in research awards from diverse applicants underrepresented nationally in biomedical research” .
To support the design and implementation of innovative training programs and strategies to promote scientific workforce diversity, the NIH established the comprehensive BUILD initiative to transform three synergistic areas that influence student achievement: student development, faculty development, and institutional capacity-building. Ten undergraduate institutions received BUILD grants that are funded as three linked awards associated with the BUILD Cores: a UL1 Linked Specialized Center Cooperative Agreement award to support Administrative and Institutional Development Cores; an RL5 Linked Education Project award to support the Research Enrichment Core; and a TL4 Linked Training Award to support the Student Training Core (see description of components ). BUILD is designed to implement and assess novel projects to improve undergraduate recruitment and retention of student talent in biomedical fields. BUILD also provides opportunities for alterations and renovations of laboratory and training facilities, curriculum development, pilot project research funding, mentor-mentee activities, and early student engagement in hands-on-research. BUILD grantees may partner with pipeline, research-intensive, and graduate/medical academic institutions, as well as industry to strengthen research training, promote faculty collaborations, and expand the pool of students engaged in BUILD activities (see participating institutions link ). Leadership structures of the programs are variable, consisting of single and multiple principal investigators (PIs), often including a provost or administrative official as PI or multiple PIs from both the primary grantee and major partner institutions. Successful BUILD programs are expected to yield tangible advances in student, faculty, and institutional development; and through its multifaceted funding model, the program is also expected to transform and institutionalize biomedical training environments for currently enrolled students and those following them for years to come.
Recognizing that the lack of mentoring was a persistent problem at all careers stages, from undergraduate education to entry into the professions to senior leadership roles, NIH funded the development of a national network of highly motivated and skilled mentors from various disciplines to be linked with mentees across the country. NRMN is charged with developing best practices for mentoring, training opportunities, and professional development for mentees and mentors. One award went to a group of five academic institutions that coordinate activities in regional hubs across the country, collaborate to provide services, coach faculty in grantwriting and to take on effective mentoring work, introduce novel approaches (e.g. culturally aware mentorship, virtual-guided mentorship), and produce scholarship on mentoring that can be widely disseminated [39, 40]. According to the funding announcement (RFA), expectations are for NRMN to help establish Hallmarks of Success , develop standards and metrics for effective face-to-face and online mentoring, connect individuals with experienced mentors in face-to-face and through online networks, develop and test innovative mentoring strategies, engage in active outreach to those with limited access to research mentors, teach effective mentoring skills, provide professional development activities, enhance access for mentees of diverse backgrounds to learn about biomedical careers and funding opportunities, and ultimately enhance their ability to attain NIH funding (see RFA-RM-13-017) . NRMN grantees are expected to develop and implement sustainable practices that can go beyond the life of the grant. The leadership structure for NRMN works though an Administrative Core that manages and coordinates with those leading several task-specific Cores on Mentorship and Networking, Mentor Training, and Professional Development. Specifics regarding NRMN developments, programs, and events can be found at the DPC websites of NIH and NRMN.
The NIH funded a Coordination and Evaluation Center (CEC) to coordinate Consortium-wide operations and facilitate the development of consortium-wide goals, create design instruments, conduct robust evaluation of measures of success toward the individual and consortium-wide goals, and serve as a focal point for consortium communication and dissemination . The NIH recognized that, in contrast to traditional monitoring of trainee success, a more extensive evaluation would be needed to better understand the impact of the Consortium activities. Although there are other NIH-sponsored national longitudinal studies of URG science training [42, 43], on behalf of the DPC, the CEC leads a unique hypothesis driven, large-scale systemic national longitudinal evaluation of research training programs. Using both established and newly developed common measures across sites, and mixed quantitative (e.g. rigorous quasi-experimental designs) and qualitative methods, the CEC focuses on long-term outcome assessment and identifying scalable transformative approaches that have potential for a broad and sustained impact on the diversity of the biomedical research workforce. Key to the comprehensive evaluation of the impact of the DPC was the creation of agreed upon evidenced based and novel consortium-wide Hallmarks of Success, which are embedded in BUILD and NRMN logic models that align expected outcomes with practices focused on critical training and career transition points. The CEC leads the development of high-impact dissemination strategies to inform peer institutions, NIH, and other key national and international stakeholders about results from the implementation of practices at the student, faculty and institutional levels. In summary, the CEC provides coordination and technical support for Consortium-wide activities including the processes for designing and managing data collection, conducts a rigorous longitudinal evaluation across programs, and provides a framework for the assessment of factors common to interventions that lead to the success of well-trained individuals from diverse backgrounds in biomedical research, with national implications for scalability to other institutions and research training programs.
It is important to note that the NIH has long recognized an urgent and compelling need to address diversity in the biomedical research workforce. Programs under the NIH-NIGMS Division of Training, Workforce Development, and Diversity (TWD) have focused on diversity in undergraduate research training as early as 1972 . The DPC is not intended to replicate existing programs but implement new ones in the context of an array of programs that are significant investments to engage scientists using training and mentoring approaches for undergraduates, postdoctoral scholars, and faculty. These previously existing NIGMS training programs have resulted in positive outcomes for trainees and participants [45, 46]. However, a new approach to implementation and evaluation was needed because reports focused on outcomes have not identified socio-psychological, practice, or institutional factors in success that can be institutionalized and scaled for widespread dissemination. The DPC and the key features of this historic collaboration for advancing diversity in biomedical science training are described in the sections that follow.
Collaboration as the mode of operation
Various forms and levels of collaboration are essential aspects of a working consortium. The first level of collaboration within the DPC is intra-institutional; for example, within BUILD campuses institution-wide support is reflected in the involvement of multiple campus units (i.e. academic departments and varying levels of administrative authority). Most DPC grantees also have interdisciplinary initiatives involving faculty collaborators and trainees in the biosciences, engineering, public health, and the social and behavioral sciences that address research on biomedical topics, health science and disparities, and/or expertise in science education. Inter-institutional collaborations among DPC grantees involve partner institutions in the region, as well as with industry or other NIH grantee institutions, to increase research opportunities for students and faculty. While BUILD sites may be regional hubs of collaborative activity, the growing network that is NRMN is built upon multiple institutions and national involvement of scholars, coaches, mentors, and mentees. Collaboration is also inter-organizational, primarily between NIH and DPC grantees, as funding is dependent on cooperative agreements that are managed in close consultation between organizations. Collaboration between Consortium members requires regular communication between groups including monthly meetings and webinars, monthly or bi-monthly working groups that discuss evaluation, program recruitment, and communications and an annual meeting of grantee teams. Common outcomes are articulated in a consortium-wide adopted document reflecting Hallmarks of Success , data sharing agreements, and planning documents. This is in contrast to other training grants, for example, where PIs implement programs according to general RFA guidelines and reporting requirements, often working independently within an institution or with limited collaboration outside of the grantee institution. The primary exception is if the training grant calls specifically for collaboration between institution types, in which case, there can be successful outcomes across shared resources . The DPC depends on multiple types of collaboration for success and has a shared purpose, works on problems that arise and then reaches consensus to forge directions.
Evaluation is central
The DPC initiative incorporates a rigorous evaluation from planning through implementation to assess the efficacy of its programs, which addresses challenges and concerns cited about previous NIH training programs that lacked clear, measurable training goals or a rigorous evaluation . The DPC stimulates new creative approaches for training and mentoring interventions for individuals with a broad range of biomedical and behavioral research interests, and is providing data to “help shape and revise current diversity training programs and inform new ones” . That is, evaluation was made central to the DPC initiative, implemented on both a national scale coordinated by the CEC for determining what works in terms of best practices and sustainability as well as local evaluations for refining BUILD and NRMN program activities (see each article in this supplement volume for evaluation methods and innovations).
Evidence-based practice and novel approaches are implemented
National reports urging improvement in science training relied on advances in federal data collection to monitor statistics and disparities across groups [5, 48], but also depended on a growing body of social science research on science education [49, 50], and evidence-based practice in teaching . From 2004 to 2013, NIH invested in a specific R01 program to advance an understanding of the science behind training interventions that now inform many of the initiatives embedded in DPC activities (e.g. see for example Research on Interventions projects on mentoring, undergraduate research, minimizing bias and stereotype threat in STEM) . Thus, practices in science training have a more substantial research base, and DPC programs are expected to implement evidenced-based approaches as well as novel approaches that can extend the evidentiary basis for practice. All aspects of this grand experiment may serve as an impetus for reform within institutions and implementation strategies across institutions to enhance diversity in the biomedical workforce.
A systemic approach involves multiple career stages, comprehensive activities, and shared resources
Across the DPC, activities span multiple career stages. For example, NRMN facilitates access to mentoring relationships and provides coaching activities for participants across multiple career stages and disciplines [39, 40]. Instead of focusing on a single PI’s program, BUILD funding goes to specific institutions to foster institutional capacity and accelerate degree productivity among URG students. Expectations are that grantees develop comprehensive initiatives that address and assess both academic and psychosocial factors that are well-known to influence why individuals persist in science [53–56]. BUILD activities are required to develop students’ skills and learning dispositions and ultimately science identities, faculty skills in research and training, and institutional capacity for addressing biomedical research and training. The latter involves, for example, acquisition of advanced technical instruments, new positions to facilitate research training, and classroom renovations for active learning. With substantial institutional investment, senior administrative leaders are expected to support BUILD initiatives and encourage institution-wide buy-in to sustain practices. BUILD also partners with high schools or community colleges (or pipeline partners) to identify and recruit early talent that matriculates into their four-year institutions. They also link with research-intensive partners to increase research opportunities, train participants using state-of-the-art equipment, place students in graduate school, and encourage faculty research collaborations. A systemic approach is intended to recognize and strengthen diverse student pathways in biomedical fields, encourage shared resources, and increase research opportunities for talent development at all levels of biomedical training.
Before describing how diversity and equity is embedded in practice across the DPC, it is important to note that rather than fund institutions that already have significant extramural resources, the NIH targeted BUILD funding to undergraduate institutions that serve highly diverse student populations and have the capacity to accelerate the production of biomedical researchers. Specifically, institutions eligible to apply were those that received less than $7.5 million (total costs) of NIH research project grant funding annually and served an undergraduate student body where at least 25% are supported by federal grants for low-income students (i.e. Pell grants). “These requirements are based on the recognition that (1) many students from low-income backgrounds are also nationally underrepresented in biomedical research, and (2) institutional commitment to these students often comes at the expense of investments in research infrastructure” . The BUILD institutional grant encourages activities that both reinforce their commitment to student success and increases institutional capacity for research in ways that will expand opportunity for diverse faculty and students. Many of the primarily undergraduate teaching institutions benefit from improving their own research capacities, as well as partnering with research-intensive institutions, to leverage resources and opportunities for their faculty and students. A unique aspect of many BUILD institutions is the development of partnerships with many of their key pipeline institutions (e.g. community colleges, high schools) to help enhance their STEM curricula and research opportunities that will better prepare and position aspiring students for successful biomedical research careers.
This targeted institutional approach raises an important question, how does one think about diversity when a campus already has a diverse student population? Minority serving institutions (historically black institutions, Hispanic serving institutions, and tribal colleges) are often cited as the source of a large proportion of URG graduates who pursue science or advanced degrees [57, 58]. Those with a historic focus on underserved populations have an institutional ethos on the advancement of these communities. This is not the case, however, with many predominantly white institutions that have recently increased demographic diversity and must rely on interventions to help URG students navigate courses and research hurdles . Campus-based research shows disparities when educational outcomes are disaggregated by race, ethnicity, gender, low-income, or first generation status within diverse as well as in predominantly white institutions [54, 60, 61]. One can look at such numbers from a deficit perspective or begin to use an equity-mindset , focusing on equity in outcomes as a relevant accountability metric to monitor progress on demographically diverse campuses to determine if gender or racial groups, for example, have an equal probability of success in biomedical majors. Low diversity in the faculty and few women in advanced faculty ranks also remains an equity issue that affects all campuses , and much underrepresentation is a result of unequal opportunity at earlier career stages. In addition to using an equity perspective, a second way to think about diversity is to consider how training activities translate into inclusive practices that address culturally diverse and underrepresented populations.
“Diversity means an inclusive approach, both to the science itself and the make-up of the groups of people who carry out the research” . Research indicates, however, that improving compositional diversity is a necessary but not sufficient condition to achieve the benefits of diversity; and while one can have a demographically diverse science classroom, for example, it can still lack the pedagogy that engages students in ways that takes into account their social identity and lived experiences, or creates interactions among peers that result in educational benefits . The concept of inclusive science has been previously introduced in reference to classroom-level innovations, yet is specifically limited to those that are “designed to promote the inclusion of all students in science [that] range from those identified specifically as female-friendly or culturally inclusive, to those that encompass both gender and culture” . An inclusive approach to science involves a renewed appreciation for all forms of diversity, calls for understanding and respect for the contributions of people from different backgrounds, and a reconsideration of assumptions about URGs that are often the target of bias and stereotype in activities that occur both inside and outside of the classroom, as well as in professional contexts. While inclusive science is, in part, a culturally responsive approach we extend the context and dimensions of inclusive science practice using emergent practices within the DPC.
Participation of diverse researchers
A central goal of an inclusive science approach is to develop a cadre of skilled researchers and reduce disparities in representation across fields of study and career stages. A primary goal of the DPC is to develop researchers at all career stages, ranging from college entry to early career attainments, to eventually become part of the NIH-funded workforce. This is accomplished by providing opportunities to individuals who may not typically have access to research mentors or comprehensive support for success in biomedical science fields. For example, the University of Maryland, Baltimore County (UMBC) has a history of a highly successful program for high achieving URGs in the Meyerhoff Scholars program . Their BUILD initiative is focused on increasing the number of trained biomedical graduates through reform of curricula, expansion of training opportunities, and the inclusion of more students who may not apply or qualify for existing scholarship programs at admission but could benefit from talent development support in college. Xavier University of Louisiana has a history of advancing African American graduates into graduate and professional schools, with a number of successful extramurally supported programs to advance students in biomedical fields. Their BUILD initiative is focused on coordinating all programs that result in a developmental sequence of research experiences throughout the college years that includes intensive, tailored career and academic advising. A post-baccalaureate opportunity is also offered for students who missed research opportunities during their undergraduate years and/or lack research experience and preparation needed for graduate school. Both campuses, and other BUILD sites, have adopted a talent development approach and increase opportunities in biomedical fields to increase the number of graduates. At later career stages, NRMN is expanding the network of mentors and mentees engaged in biomedical research, and grantwriting coaches help to address underrepresentation in NIH-funded extramural research.
Diversity innovations in science
One way to increase the number of diverse researchers is to address topics and issues in science that involve discovery and advancing knowledge about demographically diverse communities. An inclusive science approach reflected as diversity innovations in science may involve novel curricula, new research questions that address health issues that impact specific demographic groups, new research methodologies for marginalized population research, or new knowledge resulting from interdisciplinary collaborations. For example, California State University (CSU), Long Beach promotes early and well-trained researchers to approach major health challenges in a multi-disciplinary way and incorporates this into the curriculum and research communities. University of Alaska, Fairbanks (UAF) has adopted a One Health initiative that involves a focus on the important links between the health of humans, animals, and the environment in a series of courses that also involves faculty from multiple disciplines. This thematic and problem-based approach to novel curricula is particularly relevant to rural communities, which are primarily indigenous, and can also lead to further innovations in science. These approaches not only involve a culturally responsive approach (described in a subsequent section) but also open the possibility for new knowledge that result from authentic research on relevant topics (i.e. as a result of studies using new methods and multiple disciplines to address the health of underrepresented communities). Building on URG students’ prosocial values in career goals, individuals may be attracted to and retained in science if they can understand alignment with their own values or relevance to their communities while engaged in discovery of new knowledge [54, 65, 66].
Climate for diversity
Advancing an inclusive approach to science involves improving the climate in classrooms, labs, and interactions on campus. For example, perceptions of a hostile and competitive climate are associated with lower sense of belonging and adjustment to college among aspiring biomedical scientists . The climate also affects students’ sense of belonging in STEM that, in turn, is associated with higher satisfaction and better performance . A number of efforts seek to improve the climate for diversity by educating faculty about marginalization and microaggressions, which are “brief, everyday exchanges that send denigrating messages to people of color because they belong to a racial minority group” or other non-dominant group . For example, CSU Northridge BUILD conducts faculty development using Critical Race Theory to help faculty understand racism and to recognize and diminish microaggressions in daily interactions on campus . The campus focuses on providing a counterspace for a community of equity-conscious faculty and student researchers to address challenging health disparity issues. CSU San Francisco and campus partners are committed to creating intellectually safe and affirming environments to reduce the consequences of stereotype threat (i.e., a phenomena where persons fear conforming to stereotypes about their social group, such as low expectations of academic success for women and minorities), which can affect student performance and retention [53, 71]. NRMN improves the climate in laboratories and professional environments by training faculty to adopt more effective mentoring practices, and empower mentees in various contexts.
Partnerships with diverse communities
Inclusive science may also involve direct contact with diverse communities that have been previously invisible in data or programs, or marginalized due to location and lack of opportunity. For example, Portland State University (PSU) BUILD partnerships involve students and institutions located throughout Oregon, the Pacific Rim, and Pacific Northwest, which include indigenous and underserved communities that are often isolated and have few research training opportunities. Similarly, the University of Alaska Fairbanks (UAF) is building partnerships with campuses, students, and communities in remote areas of Alaska that still rely on natural resources and subsistence economies integrated with cultural traditions. ReBUILD Detroit is composed of an integrated, cross-institutional teaching and learning community for undergraduates and faculty among higher education institutions in an urban setting moving toward economic recovery. Community partnerships on local issues and problems become the focus of classroom and research activity to improve the health of students’ home communities. Other BUILD campuses engage communities by involving educators, parents, and local community organizations to identify and support talented and aspiring biomedical scientists.
Culturally responsive practice
DPC practices are helping to define culturally responsive practices in science contexts. Culturally responsive practice is most often referred to when discussing pedagogy or “teaching practices that embrace the whole student in the learning process” and provides “insight into how college educators can create classrooms in which diversity is valued” . It is important to note that BUILD campuses demonstrate an awareness of their students’ concerns and needs as low-income, first-generation, or URGs in science. For example, University of Texas-El Paso is largely an open admissions institution located near the Mexico border. This BUILD institution has adopted an asset-based approach to science training , helping students to recognize their strengths and preparing them for engagement in authentic research and presentation opportunities as early as the first year in college. CSU Long Beach encourages faculty and staff to recognize their students’ resilience and cultural strengths, and emphasizes a growth mindset. Culturally responsive practice can also be reflected in mentoring contexts. To address this NRMN has created a module on culturally aware mentoring to encourage critical self-reflection and help mentors recognize their own and mentee’s cultural identity and worldviews as it may affect interactions in research training .
Integrated science identities
A related element are programs that provide opportunities and role models that encourage the integration of participants’ identities as aspiring scientists and researchers with their social identities that constitute authentic selves. Research on women of color in STEM has led to the development of a science identity model (composed of competence, performance and recognition from significant others) that is shaped by race and gender . DPC participants are engaged in evaluation regarding science identity and how it may vary across time or career stage in programs. For example, Morgan State University’s ASCEND program emphasizes student ownership of knowledge production, sense of belonging, and the development of students’ science identity within the context of a historically black university. Opportunities to integrate identities and interact with diverse researchers as role models can also be found in activities such as the meeting of the Society for the Advancement of Chicanos and Native Americans in Science (SACNAS), the Annual Biomedical Research Conference for Minority Students (ABRCMS) and local science professional clubs. These activities allow participants to demonstrate competence, practice performance, and obtain recognition from peers and faculty. The DPC has held program-wide meetings that coincide with the SACNAS annual meeting of students, postdoctoral scholars, and faculty to celebrate the scientific accomplishments of diverse researchers.
Many of the practices in the Inclusive Science model cannot be enacted without an institutional opportunity to experiment, willingness to engage in systemic and institutional change, and broad institutional buy-in. Much will be learned about how to enact institutional change from DPC grantees, collaborating partners, and working groups. It has required energetic champions to move the vision of change forward on all sites, relationship-building, and some amount of risk-taking to attempt a new approach because it involves a critique of how typical ways of teaching, mentoring, and research may no longer be suitable for the twenty-first century needs . Changing the norms of an institution can be hard work if one is confronted with daily forms of resistance from other faculty, specific departments, or institutional leaders who lack awareness or may passively oppose inclusive or evidence-based practice. Each change agent involved in DPC activities has had to develop allies, advocate for change, recruit support from colleagues, and strategize to make initiatives successful. They developed innovations alongside traditional teaching and training practices, navigated institutional policies, and have recruited individuals that are critical to the success of initiatives. Extramural support has helped facilitate and increase attention to change efforts but, without a growing number of participants willing to contribute on each site, the investment does not guarantee buy-in from all segments of campus or partner institutions needed for sustainability.
Moving from a focus on a single intervention biomedical training approach that served small numbers of scholars to a whole trainee, whole institution approach has been a significant scale-up challenge locally and nationally in the DPC. Many of the BUILD sites are committed to providing broad access and serving students with a variety of needs. It has been a challenge to simultaneously take on intensive student development interventions and new faculty development interventions, for example. Institutions have multiple aims and campus cultures (e.g. academic culture, business culture of the administration, student cultures) and methods to advance their teaching, research, and service missions. The realities of high teaching loads and raising research capacity at BUILD campuses has created new tensions that must be resolved and may begin to reshape institutional identity. Valuing skilled mentorship and training is integral to individual success and program outcomes, for example. These new duties that come about as a result of the DPC have had implications for faculty workload and rewards. Faculty and staff contributions to national goals to enhance the diversity of the biomedical workforce should be reported and valued as evidence of impact in tenure, promotion, and merit reviews.
Many DPC grantees have been used to working independently as PIs of research and training grants. While a few had established working collaborative relationships before the DPC, the multiple levels of collaboration described as a key feature of the Consortium has been time-consuming, involving multiple institutions, people, and organizations. In a way, it has worked according to the same principles as described in the literature regarding how diversity works [14, 17], requiring each party to step out of their comfort zones, learn from each other, and work in interdisciplinary groups and diverse teams. It has required mutual respect for expertise and willingness to hear each point of view to focus on improving an implementation strategy or evaluation procedure with quality results. In the end, an enormous amount of respect and appreciation has resulted from understanding the level of commitment and hard work that each party has devoted to the success of local initiatives and to the DPC as a whole. Old networks have been expanded and new networks have been established as a result of DPC collaborative activity.
Although DPC grantees have been engaged in inter-institutional partnerships, changes in terms of mentorship training and program philosophies or approaches are not always part of the repertoire of training strategies at many partner institutions, especially at research-intensive universities. Finding ways to diffuse novel approaches and philosophies to more scientists and institutions, especially those dependent on extramural NIH funding, will be critical to creating wide-spread and lasting progress in enhancing diversity in the biomedical workforce. While NRMN has been making headway at research-intensive institutions to improve mentorship, DPC trainees will likely enter graduate school and work in environments driven by old training models and philosophies. The extent to which DPC initiatives take hold in other institutions is still unknown, but the hope is that trainees emerging from the experience are likely to pay it forward in terms of leadership, innovations in science, and awareness— serving as the next generation champions of excellence in research and inclusive science. The nations’ health, jobs, and biomedical-related security priorities will depend on their success.
We thank Pamela Thornton for assisting in several ways, namely providing a detailed account on the origins of the DPC, clarifying the role and scope of varying NIH committees, and offering helpful comments on other DPC-related matters. We also thank Steve Wallace for assisting in the external review process for this article and offering helpful comments. Finally, we thank the external reviewers for making suggestions that improved the manuscript and its appeal to a variety of audiences.
Work reported in this publication was supported by the National Institutes of Health Common Fund and Office of Scientific Workforce Diversity (USA). Publication of this article was funded by the CEC awards U54GM119024 and U54GM119024–03:S1 administered by the National Institute of General Medical Sciences (NIGMS). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Availability of data and materials
Not a data-based article.
About this supplement
This article has been published as part of BMC Proceedings Volume 11 Supplement 12, 2017: The Diversity Program Consortium: Innovating Educational Practice and Evaluation Along the Biomedical Research Pathways. The full contents of the supplement are available online at https://bmcproc.biomedcentral.com/articles/supplements/volume-11-supplement-12.
SH conceptualized the article and wrote the introduction, inclusive science framework, systemic change and larger context of the DPC. DW-L assisted with conceptualization, reviewed the literature, and wrote several sections in reference to key resources. KN wrote the section on the CEC, descriptions of the DPC, added statistics, and confirmed program information. All authors reviewed and edited the manuscript. All authors read and approved the final manuscript.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Bush V. Science, the endless frontier: a report to the president by the director of the Office of Scientific research and development. Washington, DC: United States Government Printing Office; 1945.Google Scholar
- NIH Chronology of Events. https://www.nih.gov/about-nih/what-we-do/nih-almanac/chronology-events. Accessed 27 June 2017.
- Mumper M, Gladieux LE, King JE, Corrigan ME. The federal government and higher education. In: Bastedo M, Altbach PG, Gumport PJ, editors. American higher education in the 21st century: social, political, and economic challenges. 3rd ed. Baltimore: Johns Hopkins University; 2011. p. 113–38.Google Scholar
- National Institutes of Health. Draft report of the advisory committee to the director working group on diversity in the biomedical research workforce. Bethesda: The National Institutes of Health; 2012. https://acd.od.nih.gov/documents/reports/DiversityBiomedicalResearchWorkforceReport.pdf. Accessed 27 June 2017.
- National Science Foundation, National Center for Science and Engineering Statistics. Women, minorities, and persons with disabilities in science and engineering. Arlington: Special Report NSF 17–310; 2017. www.nsf.gov/statistics/wmpd/. Accessed 27 June 2017.
- Colby SL, Ortman JM. Projections of size and composition: 2014–2060. Current Population Reports, P25–1143. Washington, DC: United States Census Bureau; 2015.Google Scholar
- Committee on Science, Engineering, and Public Policy. Rising above the gathering storm: energizing and employing America for a brighter economic future. Washington, DC: The National Academies Press; 2007.Google Scholar
- National Academy of Sciences. Expanding underrepresented minority participation: America’s science and technology talent at the crossroads. Washington, DC: The National Academies Press; 2011.Google Scholar
- President’s Council of Advisors on Science and Technology (PCAST). Engage to excel: producing one million additional college graduates with degrees in science, technology, engineering, and mathematics. Washington, DC: The White House; 2012.Google Scholar
- Brewer CA, Smith D. Vision and change in undergraduate biology education: a call to action. Washington, DC: AAAS; 2011.Google Scholar
- Association of American Universities. Five-year initiative for improving undergraduate STEM education: discussion draft. Washington, DC: AAU; 2011.Google Scholar
- The National Academies of Science, Engineering, and Medicine (NASEM). Barriers and opportunities for 2-year and 4-year STEM degrees: systemic change to support students’ diverse pathways. Washington, DC: National Academies of Press; 2017.Google Scholar
- Bowman NA. College diversity experiences and cognitive development: a meta-analysis. Rev of Educational Research. 2010;80:4–33.View ArticleGoogle Scholar
- Gurin P, Dey EL, Hurtado S, Gurin G. Diversity and higher education: theory and impact on educational outcomes. Harvard Educational Rev. 2002;72:330–66.View ArticleGoogle Scholar
- Hurtado S, Alvarez, CL, Guillermo-Wann C, Cuellar, M, Arellano L. A model for diverse learning environments: The scholarship on creating and assessing conditions for student success. In: Smart JC, Paulsen MB, editors. Higher education: Handbook of theory and research. 2012. p. 41–122.Google Scholar
- Page SE. The difference: how the power of diversity creates better groups, firms, schools, and societies. Princeton: Princeton University Press; 2007.Google Scholar
- Phillips KW. How diversity makes us smarter. Sci Am. 2014;311:43–7.View ArticleGoogle Scholar
- Kochan T, Bezrukova K, Ely R, Jackson S, Joshi A, Jehn K, Leonard J, Levine D, Thomas D. The effects of diversity on business performance: report of the diversity research network. Hum Resour Manag. 2003;42:3–21.View ArticleGoogle Scholar
- European Commission. The costs and benefits of diversity. Kent: Centre for Strategy and Evaluation Services; 2003.Google Scholar
- Freeman RB, Huang W. Collaborating with people like me: ethnic co-authorship within the US. Cambridge: The National Bureau of Economic Research; 2014.Google Scholar
- Dovidio JF, Fiske ST. Under the radar: how unexamined biases in decision-making processes in clinical interactions can contribute to health care disparities. Am J Public Health. 2012;102:945–52.View ArticlePubMedPubMed CentralGoogle Scholar
- Shapiro JR, Williams AM. The role of stereotype threats in undermining girls’ and women’s performance and interest in STEM fields. Sex Roles. 2012;66:175.View ArticleGoogle Scholar
- Schrodt P, Cawyer CS, Sanders R. An examination of academic mentoring and new faculty members’ satisfaction with socialization and tenure and promotion processes. Commun Educ. 2003;52:17–29.View ArticleGoogle Scholar
- Editorial Board. Diversity challenge. Nature. 2014;513:279.Google Scholar
- Hayden EC. Racial bias continues to haunt NIH grants. Nature. 2015;527:286–7.Google Scholar
- Reardon S. NIH to probe racial disparity in grant awards. Nature. 2014;512:243.View ArticlePubMedGoogle Scholar
- Ginther DK, Shaffer WT, Schnell J, Masimore B, Liu F, Hakk LL, Kington R. Race, ethnicity, and NIH research awards. Science. 2011;333:1015–9.View ArticlePubMedPubMed CentralGoogle Scholar
- National Institutes of Health Data Book - Research and Training Grants: Competing Applications by Mechanism and Selected Activity Codes. https://report.nih.gov/NIHDatabook/Charts/Default.aspx?showm=Y&chartId=200&catId=2. Accessed 14 July 2017.
- Tabak LA, Collins FS. Weaving a richer tapestry in biomedical science. Science. 2011;333:940–1.View ArticlePubMedPubMed CentralGoogle Scholar
- National Science Foundation website for Statistics on Minorities in Science and Engineering. http://www.nsf.gov/statistics/showpub.cfm?TopID=2&SubID=27. Accessed 27 June 2017.
- Building Infrastructure Leading to Diversity (BUILD) RFA. https://grants.nih.gov/grants/guide/rfa-files/RFA-RM-13-016.html. Accessed 27 June 2017.
- Valantine HA, Lund PK, Gammie AE. From the NIH: a systems approach to increasing the diversity of the biomedical research workforce. CBE Life Sci Educ. 2016;15:1–6.View ArticleGoogle Scholar
- NIH Common Fund. https://commonfund.nih.gov/. Accessed 27 June 2017.
- National Research Mentoring Network (NRMN) RFA. https://grants.nih.gov/grants/guide/rfa-files/RFA-RM-13-017.html. Accessed 27 June 2017.
- Coordination and Evaluation Center (CEC) RFA. https://grants.nih.gov/grants/guide/rfa-files/RFA-RM-13-015.html. Accessed 27 June 2017.
- Enhancing the Diversity of the NIH-Funded Workforce, Working Group – Consortium Members. https://commonfund.nih.gov/diversity/members. Accessed 18 Sept 2017.
- ACD Working Group on Diversity – Mission and Charge. https://acd.od.nih.gov/working-groups/wgd.html. Accessed 18 Sept 2017.
- Building Infrastructure Leading to Diversity (BUILD) Participating Institutions. https://www.nigms.nih.gov/training/dpc/Pages/partinst-BUILD.aspx. Accessed 27 June 2017.
- Sorkness Ca, Pfund C, Ofili EO, Okuyemi KS, Vishwanatha JK. A new approach to mentoring for research careers: the National Research Mentoring Network. BMC Proc. 2017;11(Suppl 12). [S14 this supplement].Google Scholar
- Jones HP, McGee R, Weber-Main AM, Buchwald DS, Manson SM, Vishwanatha JK, Okuyemi KS. Enhancing research careers: an example of a US national diversity-focused, grant-writing training and coaching experiment. BMC Proc. 2017;11(Suppl 12). [S15 this supplement].Google Scholar
- McCreath HE, Norris KC, Calderόn NE, Purnell DL, Maccalla NMG, Seeman TE. Evaluating efforts to diversify the biomedical workforce: the role and function of the Coordination and Evaluation Center of the Diversity Program Consortium. BMC Proc. 2017;11(Suppl 12). [S2 this supplement].Google Scholar
- Schulz PW, Hernandez PR, Woodcock A, Estrada M, Chance RC, Aguilar M, Serpe FT. Patching the pipeline: reducing educational disparities in the sciences through minority training programs. Educ Eval and Pol Analysis. 2011;33:95–114.Google Scholar
- Chang M, Sharkness J, Hurtado S, Newman C. What matters in college for retaining aspiring scientists and engineers from underrepresented racial groups. J Res Sci Teach. 2014;51:555–80.Google Scholar
- National Institute of General Medical Sciences – Training, Workforce Development, & Diversity. https://www.nigms.nih.gov/Training/Pages/default.aspx. Accessed 27 June 2017.
- Hall AK, Miklos A, Oh A, Gailliard SD. Educational Outcomes from the Maximizing Access to Research Careers Undergraduate Student Training in Academic Research (MARC U-STAR) Program. https://www.nigms.nih.gov/News/reports/Documents/MARC-paper031416.pdf. Accessed 28 June 2017.
- Faupel-Badger J, Miklos A. Institutional Research and Academic Career Development Awards (IRACDA) (K12) Outcomes Assessment. https://www.nigms.nih.gov/News/reports/Documents/IRACDA-outcomes-report.pdf. Accessed 28 June 2017.
- National Research Council. Assessment of NIH minority research and training programs: phase 3. Washington, DC: The National Academies Press; 2005.Google Scholar
- National Science Board Science and Engineering Indicators 2014. https://www.nsf.gov/statistics/seind14/. Accessed 27 June 2017.
- Higher Education Research Institute. Degrees of Success: Bachelor’s Degree Completion Rates Among Initial STEM Majors. Los Angeles: HERI Report Brief; 2010. https://heri.ucla.edu/nih/downloads/2010-Degrees-of-Success.pdf. Accessed 28 June 2017.
- Estrada M. Ingredients for improving the culture of STEM degree attainment with co-curricular supports for underrepresented minority students. Washington, DC: The National Academies Press; 2014.Google Scholar
- National Research Council. Discipline-based education research: understanding and improving learning in undergraduate science and engineering. Washington, DC: The National Academies Press; 2012.Google Scholar
- Understanding Interventions Article Database. https://uiai.uccs.edu/article-database.php. Accessed 7 Nov 2017.
- Chang MJ, Eagan MK, Lin MH, Hurtado S. Considering the impact of racial stigmas and science identity: persistence among biomedical and behavioral science aspirants. J High Educ. 2011;82:564–96.View ArticleGoogle Scholar
- Harackiewicz JM, Canning EA, Tibbetts Y, Giffen CJ, Blair SS, Rouse DI, Hyde JS. Closing the social class achievement gap for first-generation students in undergraduate biology. J Educ Psychol. 2014;106:375–89.View ArticlePubMedPubMed CentralGoogle Scholar
- Chemers MM, Zurgriggen EL, Syed M, Goza BK, Bearman S. The role of efficacy and identity in science career commitment among underrepresented minority students. J Soc Issues. 2011;67:469–91.View ArticleGoogle Scholar
- Graham MJ, Frederick J, Byars-Winston A, Hunter A-B. Handelsman, J: increasing persistence of college students in STEM. Science. 2013;341:1455–6.View ArticlePubMedGoogle Scholar
- Palmer RT, Maramba DC, Gasman M. Fostering success of ethnic minorities in STEM: the role of minority serving institutions. New York: Routledge; 2013.Google Scholar
- Stage FK, Hubbard SM. Undergraduate institutions that foster women and minority scientists. J Women Minorities Sci Eng. 2009;15:77–91.View ArticleGoogle Scholar
- Hurtado S, Cabrera NL, Lin MH, Arellano L, Espinosa L. Diversifying science: underrepresented student experiences in structured research programs. Res High Educ. 2009;50:189–214.View ArticlePubMedPubMed CentralGoogle Scholar
- Bensimon EM, Hao L, Bustillos LT. Measuring the state of equity in public higher education. In: Gandara P, Orfield G, Horn C, editors. Expanding opportunity in higher education leveraging promise. Albany: State University of New York Press; 2006.Google Scholar
- Hurtado S, Sork VL. Enhancing student success and building inclusive classrooms at UCLA: Report to the Executive Vice Chancellor and Provost. https://evc.ucla.edu/reports. Accessed 28 June 2017.
- Turner CSV, Myers SL. Faculty of color in academe: bittersweet success. Needham Heights: Allyn & Bacon; 2000.Google Scholar
- Bianchini JA, Whitney DJ, Breton TD, Hilton-Brown BA. Toward inclusive science education: university scientists views of students, instructional practices, and the nature of science. Sci Educ. 2000;86:42–78.View ArticleGoogle Scholar
- Hrabowki FA, Henderson PH. How undergraduate programs can boost minority success in medical school. Association of American Medical Colleges (AAMC); 2017. https://news.aamc.org/diversity/article/how-undergraduate-programs-boost-minority-success/. Accessed 27 June 2017.
- Bonous-Hammarth M. Pathways to success: affirming opportunities for science, mathematics, and engineering majors. J Neg Educ. 2000;69:2–111.Google Scholar
- Tibbetts Y, Harackiewicz JM, Canning EA, Boston JS, Priniski SJ, Hyde JS. Affirming independence: exploring mechanisms underlying a values affirmation intervention for first-generation students. J Pers Soc Psychol. 2016;110:635–59.View ArticlePubMedPubMed CentralGoogle Scholar
- Hurtado S, Han JC, Sáenz VB, Espinosa L, Cabrera N, Cerna O. Predicting transition and adjustment to college: biomedical and behavioral science aspirants’ and minority students’ first year of college. Res High Educ. 2007;48:841–87.View ArticleGoogle Scholar
- Strayhorn TL. College students’ sense of belonging: a key to educational success for all students. New York and London: Routledge; 2012.Google Scholar
- Sue DW, Bucceri J, Lin AI, Nadal KL, Torino GC. Racial microaggressions and the Asian American experience. Cult Divers Ethn Minor Psychol. 2007;13:72–81.View ArticleGoogle Scholar
- Saetermoe CL, Chavira G, Khachikian C, Boyns D, Cabello B. Critical race theory as a bridge in science training: The California State University, Northridge BUILD PODER program. BMC Proceedings. 2017;11(Suppl 12). [S4 this supplement].Google Scholar
- Steele CM. A threat in the air. How stereotypes shape intellectual identity and performance. Am Psychol. 1997;52:613–29.View ArticlePubMedGoogle Scholar
- Tuitt F. Afterword: realizing a more inclusive pedagogy. In: Howell A, Tuitt F, editors. Race and higher education: rethinking pedagogy in diverse classrooms. Cambridge: Harvard Educ Rev; 2003. p. 243-68.Google Scholar
- Johnson J, Bozeman B. Perspective: adopting an asset bundles model to support and advance minority students’ careers in academic medicine and the scientific pipeline. Acad Med. 2012;87:1488–95.View ArticlePubMedPubMed CentralGoogle Scholar
- Carlone HB, Johnson A. Understanding the science experiences of successful women of color: science identity as an analytic lens. J Res Sci Teach. 2007;44:1187–218.View ArticleGoogle Scholar
- Wieman C. Improving how universities teach science: lessons from the science education initiative. Cambridge: Harvard University Press; 2017.View ArticleGoogle Scholar