Data, Insights, and Solutions to Aid NSF ATE and 2-Year Colleges in Creating and Sustaining “Grant Active” Cultures

Introduction

Advanced Technological Education (ATE) is a program managed by the Division of Undergraduate Education (DUE) within the Directorate for STEM Education (EDU) of the National Science Foundation (NSF). “With a focus on two-year Institutions of Higher Education (IHEs), the ATE program supports the education of technicians for the high-technology fields that drive our nation’s economy. The program involves partnerships between academic institutions (grades 7-12, IHEs), industry, and economic development agencies to promote improvement in the education of science and engineering technicians at the undergraduate and secondary institution school levels [1].” ATE celebrated 30 years as a program in 2023. It is highly esteemed by its community of Principal Investigators (PIs) and is recognized by its broad community of practice as a huge success and important to the nation.  

“We like to think ATE is the gold standard for funding opportunities in the federal government,” said James L. Moore III, assistant director for the STEM Education Directorate at NSF. “Since its inception, ATE has been a model of how impactful partnerships can be in America, particularly when it comes to preparing a skilled technical workforce not only for today but for tomorrow,” he said. “The program has transformed institutions and communities and helped students see themselves in well-paying STEM careers. It continues to play a significant role in developing the workforce,” Moore said, adding that he is looking forward to “even bigger investments in community colleges.”’  Echoing these statements, American Association of Community Colleges (AACC) President and CEO Walter Bumphus stated, “It seems like we have hundreds of partnerships at AACC, but we don’t have a partnership any more significant than with ATE and NSF,” he said [2].

“Since 1994, NSF has invested $1.5 billion—$1,455,091,553, to be exact—in ATE grants. Five hundred and five community and technical colleges have received most of those funds. However, four-year colleges and other organizations that partner with two-year colleges are also eligible for ATE grants.” This equates to funding roughly half of the approximately 1,100 community and technical colleges [2]. 

Yet despite ATE’s successes, the number of proposals to the ATE program, and in fact, the EDU directorate and the entire NSF agency, have declined over the past several years [3, 4, 5]. NSF’s official workbook, “NSF By the Numbers: Providing Statistical and Funding Information,” shows that total proposals evaluated by NSF dropped from 48,197 in the fiscal year 2014 to 38,341 in 2023, a decrease of 20.44%. For proposals evaluated by NSF, specifically from two-year institutions to the Directorate of STEM Education (EDU), the number dropped from 279 to 188, a decrease of 32.62%. For proposals evaluated by NSF specifically from 2-year and Baccalaureate granting institutions (which categorically best represent community and technical colleges while excluding universities awarding Masters and Ph.D. degrees) to EDU, the number dropped from 505 to 322, a decrease of 35.6% [6].

The reasons for this decline vary but include factors such as a complex application process, lack of faculty and staff capacity and expertise due to other priorities, and burnout [7]. Additional barriers to proposal development include the lack of an institutional culture and commitment that values or supports grant work, lack of experience or knowledge about the proposal development process, and limited technical expertise to support proposal development [7]. Finally, while faculty in 2YCs benefit from developing grant proposals, the role of these faculty is typically different from that of four-year institution faculty, focused more on teaching than research [4, 8]. Having noted these and other obstacles to 2YCs that could be overcome by providing external support to 2YC faculty and administrators, ATE began supporting mentoring and capacity-building projects to help colleges pursue and sustain grant development efforts and ultimately achieve the NSF ATE mission of assisting 2-year IHEs to improve the education of technicians in science and engineering.

However, while effective, institutional investments from NSF and individual 2YCs still suffer from inefficiencies; for example, not all mentoring efforts result in grant submissions or sustainability. Part of the inefficiencies is that the variances explaining grant funding and outcomes across the profile of 2YCs are not yet understood, nor are AI tools combining big data and localized data being widely used. Thus, individual 2YCs and Principal Investigators (PIs) having the greatest likelihood of success with grants are not being intentionally identified through data-driven processes at the onset of NSF mentoring projects; the effecting result is that NSF and 2YCs are not making the most well-informed and judicious decisions regarding where and how to invest the time, effort, and resources towards professional development and institutional capacity building.

This article focuses on data, insights, and solutions for improving 2-year colleges’ and mentorship programs’ efforts to build grant capacity and sustainability. It is based on the authors’ involvement in two ATE-supported projects:

1. Project Vision, an ATE mentoring project, was established in May 2020 under DUE # 2018198, “Broadening Institutional Participation in the NSF Advanced Technological Education Program.” Ben Reid and Shalee Hodgson are the independent evaluation team; Kevin Cooper is the PI; Rassoul Dastmozd and David Brown are the co-PIs. Project Vision helps colleges discover and match innovative ideas with NSF funding opportunities and is led by a seasoned team of NSF ATE experts, former faculty and senior college administrators, and former NSF program officers. Project Vision’s goal is to provide 2YCs with expertise to generate ideas and subsequently support capacity building at each college so that these colleges can regularly submit proposals when appropriate to DUE.
2. Grant Insights, an ATE-applied research project, was established in October 2022 under DUE # 2202169, “Grant Insights for Research and Development (GIRD): Using Big Data Centered Mixed Methods to Explain Variances in Grant Funding and Outcomes at Two-Year Colleges.”  Reid is the PI, Dastmozd is a co-PI, and Hodgson and Cooper are on the Advisory Board. The research objective is to explain variances in grant funding and outcomes amongst similar 2YCs. Mixed methods research design is being conducted combining four types of data: (1) algorithm-derived meta-data on 2YC characteristics and performance, (2) public and campus institutional data, (3) surveys of college and program faculty and administrators, and (4) in-depth interviews with college and program faculty and administrators. Grant Insights employs this innovative research design approach to classify colleges based on campus characteristics and draw comparisons between their grant infrastructures, allowing the researchers to identify best practices employed by colleges that are proficient in securing external funding that can be adopted by colleges that are not yet as proficient. The goal is for NSF ATE and 2YCs to use these project results to determine impactful practices that can guide institutional investments. 

This article communicates:

• Project Vision’s survey results: Understanding Proposal Submissions from Two-Year Colleges (2YCs). The purpose of the nationally distributed survey was to understand proposal submission trends, motivators, and inhibitors at 2YCs from the perspective of faculty, administrators, and presidents. It received 238 responses from 134 unique 2YCs and highlighted the variances in grant funding and the effects of incentives, support, and changes to senior administrations upon 2YC grantsmanship and proposal submissions.
• Project Vision’s root-causes analysis: Addressing the Key Factors to Being Grant Active.
• Grant Insights’ innovative quantitative research methods, which use big data algorithms, cluster analyses, and decision-support systems commonly used in the financial and healthcare sectors, are applied in this project to 2YCs.
• Project Vision’s use of Grant Insights’ computational analytic tools to identify appropriate prospective mentee 2YCs more efficiently.
• Grant Insights and Project Vision studies are underway to identify, respectively, the characteristics and factors that differentiate colleges with varying levels of external funding and the characteristics of colleges where mentorship efforts have led to relatively low and high success rates. The intended purpose of these studies is to inform NSF and IHE institutional investments with a high return on investment and broaden participation.

Methods

This section addresses three methods for their corresponding results in the following section:

1. Project Vision’s survey results: Understanding Proposal Submissions from Two-Year Colleges.  
2. Project Vision’s root-causes analysis: Addressing the Key Factors to Being Grant Active.
3. Grant Insights’ quantitative research methods using big data algorithms, cluster analyses, and decision-support systems.

Project Vision’s Survey Results: Understanding Proposal Submissions from Two-Year Colleges

The first project activity of Project Vision’s PI and evaluator teams was to collaboratively develop (using multiple rounds of iterations) and distribute a survey to assess the landscape at the nation’s two-year colleges regarding issues connected to pursuing NSF grant support. These survey results (See Results section) helped inform Project Vision’s mentorship efforts [9].

• Survey Title: Understanding Proposal Submissions from Two-Year Colleges.  
• Purpose of the Survey: To understand proposal submission trends, motivators, and inhibitors at 2YCs from the perspective of faculty, administrators, and presidents.
• The survey was disseminated through multiple channels, including the Project Vision PI and evaluator teams, ATE lead program officer, ATE Central, AACC, Rural Community College Alliance (RCCA), CREATE National Energy Center, Regional Center for Nuclear Education and Training (RCNET), Center for Laser and Fiber Optic Technical Education (LASER-TEC), and National Center for Systems Securing and Information Assurance (CSSIA). Two hundred thirty-eight responses were received from 134 unique 2YCs, and highlighted the variances in grant funding and the effects of incentives, support, and changes to senior administrations upon 2YC grantsmanship and proposal submissions.

Root-Cause Analysis by Project Vision: Addressing the Key Factors to Being Grant Active

Following Project Vision’s pilot year and the first months working with Cohort 2, the Project Vision leadership team and external evaluator took a step back to assess the learnings to date. Three broad evaluation questions measured outcomes and addressed opportunities: performance versus plan, consequences of Project Vision mentee institutions, and consequences of Project Vision on NSF DUE. The data sources included three mentorship surveys, pre, mid, and post; notes from open office hours, which were held monthly for mentee colleges and subject matter experts (SME); notes from topic-specific calls that were based on common cohort foci areas; a professional development webinar series; project documentation from PIs, Co-PIs, SMEs and the project manager; mentee colleges’ verbal and forwarded Proposal Panel Reviews; and interviews of Project Vision PIs and SMEs. After analyzing all these data, two insights to improve operations and underlying factors to sustained grant activity were realized: 1) there were differences internally (PIs, SMEs, evaluator) and within the community of practice (other mentor organizations and 2YC stakeholders) on the ways different people speak of and measure the factors that lead to grant success and sustainability (e.g., characteristics, strengths, limitations, impediments) and, 2) only a few factors are significant in determining whether a college will be a repeat submitter of grant proposals.

So, the team sought to identify the key factors and create a common language set and quantifiable reporting system, which could be used as a regression analysis model to measure how the factors helped or hindered the achievement of a college’s grant activity results. This model formed the basis for the next two pieces phases to identify the root causes behind grant proposal activity at 2YCs:

• The evaluator conducted in-depth interviews with the three PI/Co-PIs and eight SMEs (range: 8 to 47 minutes; mean: 32 minutes). Collectively, these individuals have more than 200 years of experience in higher education and NSF award roles. The interviews were coded for common themes and presented back to the group of participants for discussion, clarifying modifications, and consensus. Derived from the responses common themes were three key factors and seven primary sub-variables most determinant of whether a 2YC is likely to become and sustain being “grant active.”
• This framework formed the post-mentorship report out survey, which is now integrated from multiple perspectives: mentee faculty, mentee administrators, Project Vision PI/Co-PIs, and Project Vision SMEs. 

Grant Insights’ Quantitative Research Methods Using Big Data Algorithms, Cluster Analyses, and Decision-Support Systems

Large industries use cluster algorithms to identify and categorize organizations that are similar in style. For example, mutual funds create diverse portfolios by using algorithms to place companies into different clusters and invest in companies in each of those clusters. Co-PI Dr. Baechle developed the Portfolio Mapper as a research tool to illustrate non-intuitive relationships among stock trades; a case in point, his analyses of Fortune 500 companies reveal that Tesla trades more like a software company than a car company due to Tesla’s strengths in automation, sensory, and battery technologies. Another car company, like Ford Motor Company, could refer to this analysis and assess whether strategically adjusting its organizational culture and operations may cause it to function and be valued more like Tesla.

As a preliminary exercise, we created a beta algorithm starting with 2,330 variables of real data points from the US Department of Education (USDOE) College Scorecard and NSF. Figure 1 clusters colleges with ATE funding (red) and colleges without ATE funding (blue) based on hundreds of factors. We can compare colleges to peer institutions on hundreds of factors and common factors between grant-proficient colleges and how less proficient colleges differ. The major strength of this project is the application of big data analyses common in high-tech industries with limited application in higher education.

Results and Discussion

This section addresses the corresponding results from the previous section, along with discussing why, how, and the results of Project Vision’s use of Grant Insights’ analytic tools:

1. Project Vision’s survey results: Understanding Proposal Submissions from Two-Year Colleges (Figure 2 and Table 1).  
2. Project Vision’s root-causes analysis: Addressing the Key Factors to Being Grant Active.
3. Project Vision’s use of Grant Insights’ computational analytic tools to identify appropriate prospective mentee 2YCs more efficiently.

Figure 2 provides a breakdown of Project Vision’s 238 survey responses from 130+ unique community colleges. This includes the types of respondents by job title and primary role, as well as a profile of responding colleges that included the identifying categories of rural location, minority-serving institution, less than 5,000 FTE students, 5,000-10,000 FTE students, a president that had announced their departure, and a president that had been at the college less than two years [9].

Table 1. Project Vision survey results: Four key insights

Key Insights	Details
1) Lack of INCENTIVES is the greatest limiting factor for faculty to develop grant proposals.	69% (n=163) disagree that their college is structured to incentivize faculty to pursue grant funding. 82% (n=194) need more incentives for developing proposals and implementing an award. 83% would be motivated [41% definitely, 27% probably, 15% would consider] to submit a grant proposal for a stipend or release time for their development effort(s). 88% would be motivated [41% definitely, 29% probably, 18% would consider] to submit a grant proposal for a stipend or release time for the grant management responsibilities.
Suggestion: As a requirement in the NSF-ATE solicitation, state that proposals must describe how faculty are to be motivated to complete faculty adoption of the work.
2) Faculty need more SUPPORT in developing proposals and managing awards.	71% (n=168) need more support writing grant proposals. 74% need more institutional support in managing the grant award. 69% need more assistance formulating an innovative idea to match grant funding opportunities. 72% need more assistance identifying and bringing together partners in a grant proposal.
Suggestion: Continue scaling independent mentoring organizations and encourage peer-to-peer mentoring among already funded ATE grant recipients. Additionally, promote peer mentoring at ATE PI and HI-TEC Conference sessions.
3) Effect of changes to senior administration on proposal submissions.	Highly dependent on prior experience with grants of the incoming president (qualitative interviews) Disconnect between faculty and administration perspectives on the effect (administration thought the number of proposals increased while faculty thought the opposite)
Suggestion: Explicitly emphasize professional development, encouragement, and dissemination of the value of ATE grant funding to the College Board of Trustees, so it is a key criterion in selecting a new president.
4) Key themes from qualitative interviews of presidents that promote grant-seeking efforts.	Major themes emerged: (a) support for grant-seeking efforts from presidents, (b) alignment with the institution’s strategic priorities, (c) having an infrastructure in place (human, talent, capacity), and (d) sustainability efforts after the grant project is completed at the recipient institutions.

Table 1 illustrates the three key takeaways from the Project Vision Survey, along with a fourth takeaway from qualitative interviews conducted by the project team coinciding with the survey conducted by the independent evaluation team, and suggested actions to support colleges becoming and sustaining “grant active” status. Details summarizing survey responses are also provided, giving insight into how and why the key takeaways were developed [9]. This analysis highlights that while existing mentoring projects support grant work in 2YCs, gaps still exist to contend with in strengthening the success of 2YCs in proposal development.

Project Vision’s Root-Causes Analysis: Addressing the Key Factors to Being Grant Active

The data analysis and interview coded themes produced three key factors and seven primary sub-variables most determinant of whether a 2YC is likely to become and sustain being “grant active.”  Below are the key factors and variables using plain language from the interviews:    

1.  Value Proposition Buy-in of Grants, NSF, ATE, and Industry-partnerships
   1. Administrators – understand the benefits and the costs and agree to make it happen.
   2. Faculty – understand the benefits and the costs and agree to at least “jump in the pool”; thereupon, their overall experience will largely determine if they will “stay in the pool” and integrate grants and research in their academic careers complementing teaching.
2. Faculty / Principal Investigator User Experience Design
   1. Compensation models – equitable course release time or additional contract for grant work.
   2. Support structures – specifically, human resources; “Academic Support Staff” or time from executives to operationally and fiscally manage and report on the grant.
   3. Encouraging environment – Administrator/Leadership attitude and actions that communicate to faculty that grants are seen as positive and valuable.
3. Depth of Team in Capacity and Expertise
   1. Sufficient number of faculty members and disciplines.
   2. Driven grants department or equivalent and administrators providing grant expertise.

*Additionally, recognized as a key factor but ultimately outside the sphere of direct influence of 2YCs was the size and type of local and regional industry employers.

Project Vision Cohort 1: Post-mentorship Survey Results

After the interviews of the three PI/Co-PIs and eight SMEs were coded and produced the above framework of Key Factors to Being Grant Active, they were converted into survey questions and administered to Project Vision’s first cohort in a Post-Mentorship Survey with the following results (Tables 2-4) [10].

Table 2. Do your college and colleagues buy into the value proposition of “grants,” NSF, ATE, and industry partnerships?^a

Primary sub-variables	Somewhat/ mostly disagree	Slightly disagree	Neither agree nor disagree	Slightly agree	Somewhat/ mostly agree	Completely agree
Administrators are active in and advocate for grant funding and industry partnerships.		7%			20%	33%
Faculty are active in and advocate for grant funding and industry partnerships.	13%	27%	13%	20%	20%	7%

^aOverall rating, no=1, yes=14

Table 3. In your lived experience and observations, is the faculty with grant development and PI roles a positive experience?^b

Primary sub-variables	Completely Disagree	Somewhat/ mostly disagree	Slightly disagree	Neither agree nor disagree	Slightly agree	Somewhat/ mostly agree	Completely agree
There are equitable compensation models at my college.		13%	7%	27%	20%	7%	27%
There are adequate support structures in place at my college.	7%	13%	13%	13%	13%	13%	27%
There is a grant-encouraging environment at my college.		7%		13%	40%	13%	27%

^bOverall rating, no=2, yes=13

Table 4. Does the college have “depth of team” in capacity and expertise?^c

Primary sub-variables	Completely Disagree	Somewhat/ mostly disagree	Slightly disagree	Neither agree nor disagree	Slightly agree	Somewhat/ mostly agree	Completely agree
The college has sufficient numbers and expertise of faculty members and program disciplines.		13%	7%	7%	27%	33%	13%
The college has sufficient numbers and expertise in grants department and administrators.	13%		7%	20%	27%	33%

^cOverall rating, no=7, yes=8

The Project Vision Cohort 1 Post-mentorship Survey was administered to gain feedback directly from the faculty and administrators most engaged in the Project Vision mentoring regarding their professional gains and experiences, the usefulness of Project Vision’s offerings, and their broad observations and opinions on where their college stands in terms of capacity and how to engage and sustain grant activity. Twenty-five participants received the survey, and 15 responded (60% response rate), including eight faculty and seven administrators from eight two-year colleges [10].

Tables 2-4 provide detailed survey results, illustrating the three key factors and seven primary sub-variables determining whether a 2YC is likely to become and sustain being “grant active.” The responses were used as a forecasting and regression model for each mentee college but were reported in aggregate for confidentiality purposes. The range of responses correlates to the differences between colleges and disconnects between faculty and administration [10].

The significance of the Project Vision mentored colleges was further compared and correlated with the previous national survey results on these topics. For example, when asked about their overall “grant experience” (a key indicator of whether faculty and administrators will continually seek to incorporate PI roles into their primary job responsibilities), the respondents to the national survey (n=226) answered 53% Positive, 43% Mixed, and 4% Negative [9]; whereas Project Vision’s first cohort respondents (n=15) replied 93% Positive, 7% Mixed [10].

Project Vision’s Use of Grant Insights’ Computational Analytic Tools to Identify Appropriate Prospective Mentee 2YCs More Efficiently

Project Vision began using the Grant Insights analytic tools in its third year to recruit for its fourth cohort to identify appropriate prospective mentee 2YCs and PIs more efficiently. To recruit Project Vision’s first three cohorts, it took personalized outreach messages to an average of 74 2YCs to recruit 13 2YCs for a single mentee cohort. For its fourth cohort, Project Vision targeted 26 2YCs using the Grant Insights tools and recruited 18. By finding and assessing 2YCs that are statistically most similar to one another (not just geographically or by a subjective set of benchmarks), Project Vision was able to identify colleges “statistically nearby” 2YCs that have high-performing track records with ATE funding yet do not have ATE themselves. This particular method and tool helped with more accurate and efficient identification of appropriately ready 2YCs, explaining the value proposition and contextualizing the pitch of why that new 2YC should strongly consider pursuing NSF ATE funding. Using traditional methods, the 3-year average was one recruit for every 5.7 outreaches to a prospective 2YC; using the Grant Insights tool, that number was reduced to 1 recruit for every 1.4 outreach, an accuracy, and human resource efficiency savings of 400%.

Because four cohorts is not a statistically significant sample size, we are continuing to track inputs and outcomes to assess whether and how these analytic tools and processes evolve over time from a promising practice to a best practice, along with understanding how others use them in their particular use cases.

These methods, processes, and results have been shared and discussed with the NSF ATE community through two different conferences to date:

1. High Impact Technology Exchange Conference (HI-TEC), July 26, 2023 in Atlanta, GA [11]
   • Session title: Using AI and Machine Learning to Broaden Participation in the ATE Community
   • Abstract: Employ big data sets and tools and AI (Artificial Intelligence) and ML (Machine Learning) techniques to be better equipped to harness the data revolution. Learn how to add big data sets and use tools of dimensional reduction, cluster algorithms, and spectral analysis in your specific ATE work for the aim of increasing participation (e.g., STEM enrollment, completion, diversity metrics, funding, and usage of a technology) in your respective projects.
   • Presenters: Ben Reid, Director and Principal Investigator, Grant Insights for Research and Development (GIRD), Impact Allies, Vero Beach, FL; Kevin Cooper, Director and Principal Investigator, Project Vision, Indian River State College, Fort Pierce, FL
2. ATE Principal Investigators Conference, October 25, 2023, in Washington, D.C. [12]
   • Workshop title: Using AI to Broaden Participation in ATE + How to Conduct Applied Research
   • Abstract: Two distinct foci applicable across STEM disciplines are addressed in this hands-on workshop: (1) how to employ big data sets and Artificial Intelligence, and (2) how to design and conduct applied research. Through the lens of an applied research project that uses big data with localized data to understand grant ecosystems, participants will learn how to use these skill sets in their specific ATE work for the purpose of increasing participation (e.g., STEM enrollment, completion, diversity metrics, funding, usage of a technology) in their respective fields.
   • Presenters: Christopher Baechle, Co-PI, Impact Allies, FL; Rassoul Dastmozd, Co-PI, Project Vision and GIRD, MN; Ben Reid, Director and PI, Impact Allies, FL; Will Tyson, Co-PI, University of South Florida, FL

Conclusion

For 2YCs in the United States, a culture that supports institutions in being “grant active” can be very valuable in supporting faculty professional development, innovation, and new program development. Both Project Vision and Grant Insights, through their respective foci, have made significant contributions to the knowledge base of how to contribute to the NSF ATE mission of assisting 2-year IHEs to improve the education of technicians in science and engineering. Yet, much remains to be accomplished through mixed methods research, evaluation of practice, development of tools, replication of promising practices by independent researchers, and refinement by testing the resulting methods, tools, and best practices. Project Vision will continue to measure and evaluate its work with mentee colleges, and Grant Insights will be producing findings reports this year from its separate mixed methods applied research and computational analytics. Grant Insights and Project Vision studies underway identify, respectively, what characteristics and factors differentiate colleges with varying levels of external funding and the characteristics of colleges where mentorship efforts have led to relatively low and high success rates. The intended purpose of these studies is to inform NSF and IHEs which institutional investments have high and low returns on investment and broaden participation in the NSF ATE mission.

Acknowledgments

This work was supported by the National Science Foundation (NSF) under awards 2018198 and 2202169.

Disclosures

The authors declare no conflicts of interest.