The Impact of Combining Coaching and Mentoring Skills for Successful ATE Grant Proposal Development

Introduction

Across careers and workspaces, there is a need for the enhancement of communication skills [1,2]. This need is critical at any level, not exclusively for those in managerial positions. One source for developing crucial listening and questioning communication skills is through coaching courses sanctioned by the International Coaching Federation (ICF) [3] and taught by credentialed instructors. This article focuses on only one of the FORCCE-ATE project objectives − mentor training. FORCCE-ATE is based on prior research findings from several peer-reviewed articles. The collection of these articles all concluded that acquiring a specific subset of coaching skills would enhance educators’ and mentors’ questioning and listening skills [1,2,4-7]. These studies were used in the development of the mentor training that is described throughout this article. In the FORCCE-ATE project, the mentor-coaches applied their new coaching skills when working with their mentee teams to submit competitive NSF ATE proposals. An iterative approach was used to improve the mentor-coach training across project years and, subsequently, the success of ATE grant proposal development among project participants.

This NSF research project sought out established mentors with grant writing knowledge and then enhanced their skill sets by adding coaching skills. As clarified by researcher Larisa Pfeiffer, “Mentoring is learning how someone more successful succeeds; coaching is learning how you will succeed” [7]. Because the coaching skills training would be delivered to adult learners, the research project is aligned with researcher Malcolm Knowles’ work on his adult learner-centered Andragony model [8]. This NSF research project began with a foundational pilot year where the methodology utilized only mentoring skills. The progression from mentoring alone to the combined use of mentoring and coaching skills is documented in this paper.

Methods

Participants and Qualitative Data Collection

This research focused on collecting qualitative data to assess the effects of providing coaching skills training to mentors. Participants were all college faculty who had experience as principal investigators on NSF ATE projects. These individuals were tasked with working with other community college teams whose goal was to create and submit an NSF ATE proposal. To gather data for drawing comparisons and conclusions, the following measures were used:

• Pre and post-surveys targeting mentor-coaches
• Pre and post-surveys targeting mentee team participants
• Professional ICF-certified coach focus groups

While the qualitative data collection methods stayed the same year over year, the content of the mentor training was enhanced based on the formative assessment conducted annually by the project’s external evaluator. The specific training enhancements for each year are described below.

Pilot Year Methodology and Implementation (2020)

Mentor Selections

In January 2020, the leadership team of a pilot NSF project invited six individuals to serve as mentors to one or two college teams participating in a National Science Foundation (NSF) Advanced Technological Education-funded program to strengthen grant proposals. The requirement for selecting this cadre of mentors was previous leadership experience with NSF grants and familiarity with the NSF grant submission process. Additionally, all mentors had competencies in information technology (IT) or cybersecurity, as defined by the Association for Computing Machinery (ACM):

“A computing-based discipline involving technology, people, information, and processes to enable assured operations in the context of adversaries. It involves the creation, operation, analysis, and testing of secure computer systems. It is an interdisciplinary course of study, including aspects of law, policy, human factors, ethics, and risk management” [9].

These skill sets qualified the selected faculty to mentor other faculty in specific STEM disciplines who needed to learn the grant development process. As a result of the global pandemic of March 2020, the leadership team swiftly pivoted from planning an in-person grant development workshop to a virtual one using Canvas Free-for-Teachers’ version and Zoom technologies.

Mentee Colleges

To be considered for the grant development training, college teams comprised of cybersecurity or IT faculty were required to submit an application that included an institutional support letter. Twelve (12) college teams were selected from the applicant pool.

Following the selection, the mentors received assignments that paired them with one or two college teams consisting of at least two faculty members and a grant writer. These mentors were tasked with advising and guiding the mentee teams through the entire grant development cycle, culminating in the grant submission. Over the next five months (May – September), mentors spent up to 35 hours with their respective teams on the initial idea development for the grant, a three-day virtual workshop, and the final review of the grant application before the October 2020 submission deadline.

Year 1 Methodology and Implementation (2021 – 2022)

The decision to add coaching skills to the mentor development resulted in the development of six (6) one-hour coaching modules, supplementary audio and video materials, and a component consisting of 4.5 hours of practicum (also called triads). The mentor-coach training was executed via online platforms (e.g., Zoom). This combination of coaching skills training plus practicum sessions remains a differentiator between this project and other similar Advanced Technological Education (ATE) mentoring initiatives. In mid-October 2021, the FORCCE-ATE project team received their funding notice, and delivery of the progression of the mentoring-to-mentor-coaching project began.

This knowledge and skills progression from Educator to Mentor-Coach involved the introduction of instructional components strategically woven into the training:

• In-depth exploration of the NSF grant submission process plus access to related resources
• Coaching knowledge and skills instruction coupled with application through practicum sessions
• Monthly mentor-coach check-in sessions while working with mentee teams
• Additional support resources in between meetings

The FORCCE-ATE coaching model is similar to the model used in two student-centered coaching research studies at the University of North Carolina, Chapel Hill (UNC). Volunteer academic personnel at the UNC at Chapel Hill were also being trained in a similar coaching methodology and set of skills for their role as academic coaches [10,11]. Results from that study showed that the recipients of the academic coaching “who participated in both the in-person and online academic coaching conditions had significant increases in their metacognitive skills.” While the UNC study targeted students as the recipients of the coaching, in both the FORCCE-ATE and UNC studies, the (coaching) service was delivered by college faculty that had been given basic training in coaching. The main difference within the FORCCE-ATE project is that faculty were the recipients of the coaching while students were the recipients of coaching in the UNC studies. In both cases, faculty served as coaches.

As with the pilot, potential mentors applied for the FORCCE-ATE program in the late fall of 2021. Of the six (6) selected, only one had participated in the pilot year. Additionally, three faculty from the NSF Mentor-Connect project [12] were invited to participate in the coaching training and practicum sessions, increasing the number attending training sessions to nine.

Mentor-Coach Course Rollout

Prior to the launch of the coach training, the course content, which focused mainly on deep listening skills and powerful question creation, was reviewed by three ICF-certified coaches.

The syllabus included the following content and materials:

• Discussion of ethics in coaching
• Coverage of different coaching methods used internationally
• Differentiation among the roles of counseling, mentoring, consulting, and coaching
• Development of powerful questions and deep listening skills
• Discussion on the importance of action steps taken by clients (those being coached)
• Audio sound bites from coaching sessions
• Video clips by internationally recognized coaches
• Exercises focused on session content
• Live application of coaching knowledge in the form of practicum sessions (i.e., triads)

The workbook, audio files, presentation slide decks, and all supporting materials were ported to the CANVAS LMS online platform so that mentor-coach trainees would have 24/7 access.

Mentor-Coach Triads (Practicum Sessions)

Before the course launch, the next step was to recruit ICF-certified coaches to oversee the practicum sessions (triads). Like other skill practice environments, triads are intentional sessions where the roles of coach, client, and observer are assigned to the participants. Each participant practices newly acquired coaching knowledge across the three rotations, as shown in Figure 2 below. In this case, triads were limited to 20-minute practicing coaching sessions, with an additional 5-10 minutes of recap and evaluation by the participants. After evaluation, the participants rotate into the next assigned role.

Between December 2021 and February 2022, the new mentor cohort, plus three “observers” from the NSF-funded Mentor-Connect project, took part in the inaugural delivery of the course and triads. By obtaining this subset of coaching skills, these mentors received the title of “mentor-coaches.”

While the coaching modules provided supporting educational theory and methods, the customized triads became the highlight of the training for the participants. For each of the three 90-minute practicum sessions, participants attempted to utilize their new coaching skills, observe others, and receive input from the ICF-certified coach who was present. So that the participants could reap the benefits from interacting with each of the ICF coaches, the coaches rotated weekly among the three triad groups of mentor-coaches. While other reviewed studies provided similar classroom work on coaching, the FORCCE-ATE project is currently the only one that combines knowledge acquisition with multiple ICF-monitored practicum sessions, coupled with ongoing meetings with mentee teams where the mentor-coach can continue to hone their coaching skills.

Skills Application with Mentees

In mid-April of 2022, ten (10) mentee college teams were selected for participation in a new cohort of the FORCCE-ATE grant development initiative and were subsequently matched with a mentor-coach. Once the mentor-coaches received their college team assignment(s), they began their mentor-coaching journey, ending when an NSF ATE proposal was submitted or at the October 2022 grant submission deadline.

Summer Workshop with Mentee Colleges

As the COVID-19 pandemic subsided, the project reverted to offering an in-person workshop, as originally planned. While interaction with mentee teams utilized online communication platforms (e.g., Zoom, Microsoft Teams), both mentor-coaches and the mentee teams attended the in-person Grant Development Workshop hosted at Prince George’s Community College in Largo, MD (PGCC) in June 2022. Along with the workshop’s content, this face-to-face meeting helped build trust relationships among mentee teams and mentor-coaches.

Year Two Methodology and Implementation (2022 – 2023)

The process for selecting the mentor-coach cohort and the mentee teams for year two changed little from the previous year. Four (4) of the 2022 mentor-coaches were invited back for year 2 of the grant. Four (4) additional mentors were added through the application process, bringing the total to eight mentor-coaches. With assistance from the Community College Presidents’ Initiative in STEM (CCPI-STEM) ATE project [13] plus several other academic organizations, the number of applicants for the mentee teams exceeded that of the previous year, and the team profiles also appeared stronger. In March 2022, 15 new college teams and one (1) returning team (not funded in year 1) were selected for the year-2 mentee cohort. As in the previous year, these teams were paired with a mentor-coach for their April-to-October journey of crafting a competitive proposal for submission to NSF ATE.

Mentor-Coach Training

Since half the mentor-coaches had already received the coaching skills training the previous year, they were only required to attend newly added training modules on team coaching and how to begin with your mentee team. Returning mentor-coaches were also required to participate in the virtual orientation meeting and all three triads.

The Fundamentals of Coaching workbook developed for year one received substantial upgrades, including resequencing of materials, additional content for modules 2 and 4, additions to the appendices, and two new modules, as called for in the year one evaluation (“Team Coaching” and “How to Begin”). The final course module, “How to Begin,” introduced suggestions on content for the first four meetings with their mentee teams. This module addressed the discomfort mentioned by several mentor-coaches who felt unsure how to structure their initial meeting with their mentee teams. The module’s content melded mentoring activities and coaching questions that could be used to build team cohesion and focus. The workbook’s title changed in year two to Coaching Fundamentals for Mentors.

In addition to the workbook changes, the training materials were enhanced by adding videos. The major points from the content of each of the modules were identified and repurposed in a 4- to 5-minute video, which included audio commentary. The video served as a prerequisite to the actual class. These brief videos were integrated into the Canvas LMS, flagged as mandatory, and linked to homework questions to be answered after viewing the video content.

Mentor-Coach Triads (Practicum Sessions)

The triads were again staffed by three ICF coaches: two returnees and one new coach. The only change in the triad delivery format was to honor the request by the ICF coaches to stay with one assigned triad group throughout the three sessions. They felt they could better evaluate participant growth and skill development if they observed the same mentor-coaches over the three triad sessions. What was added to the triads was a small block of time at the end of the session where the mentor-coach trainer and ICF coaches could privately discuss what they observed.

The goal of providing coaching competencies to the mentor-coaches was to increase their effectiveness during their interactions with their mentee teams. This training began at the end of January and ran through the beginning of March 2023. The three triad/practicum sessions were again required for the entire cohort. In mid-April, the college teams applying for the mentee cohort were selected. Mentor-coaches received their mentee team assignments and were expected to establish a schedule of ongoing meeting dates with their teams.

Summer Workshop with Mentee Colleges

As in year one, all mentor-coaches and at least two representatives from their mentee team attended the in-person Grants Development Workshop in June at PGCC in Largo, MD, facilitated by the FORCCE-ATE leadership team. Different in year two was the requirement to have each mentee team’s Academic Dean or other college administrator attend the workshop. Their presence increased the college’s overall understanding of the NSF grant process. Additionally, they identified what was at stake for their institution and how an administrator could play a significant role in supporting the mentee team. Additional post-workshop mentor-coaching was provided until the October 6, 2023 ATE submission deadline.

Results and Discussion

In the 2020 pilot year (global pandemic), the training focused on the grant development process and mentoring techniques. Coaching skills were added to the mentor training during the first year of the FORCCE-ATE project. Additional elements were added to the mentor-coach training in subsequent years. The additional training elements are listed in Table 1.

Table 1. Elements Added to the Mentor-Coach Training Each Year

Year	Mentor-Coach Training Element
Pilot Year (2020)	Mentoring Techniques
FORCCE Year 1 (2021-22)	Active Listening and Powerful Questioning
FORCCE Year 2 (2022-23)	Team Coaching and Starting with Your Team
FORCCE Year 3 (2023-24)	Assistive Technology

Pilot Year

Over the next five months (May – September), mentors spent up to 35 hours with their respective mentee teams developing the initial idea for the grant, attending a three-day virtual workshop, and reviewing the final grant application. All work had to be completed before the October 2020 submission deadline. In this pilot year of the grant-writing project, 11 of the 12 college teams submitted a project proposal to NSF. Subsequently, nine of the eleven colleges received funding for an 82% success rate compared to the average ATE successful funding rate of 23%.

Not only did this cohort’s funding percentage exceed expectations, but the comments from the mentee teams reflected their satisfaction with their new knowledge set. The mentoring was shown to affect the production of competitive grant proposals positively. The research question then became the following: what other ways could the entire grant development process and learning experience be improved? Click the interactive map link for additional background information on the participating colleges. See Figure 3 below.

Link to active map: https://www.google.com/maps/d/u/0/edit?mid=19kfcCXv2igz6AlgcUW8EKrO82nqXfTo&usp=sharing

Lessons Learned from Pilot Study

In the 2020 Pilot delivery analysis, the leadership team identified the following areas needing improvement. Filling the role of mentor coordinator earlier in the process was required to give sufficient time for creating the educational plan and training materials. The management resolution was to add a Co-PI to oversee the mentor training in the follow-on ATE grant proposal. The leadership team identified a missed opportunity to assist college teams to better prepare for the in-person grant development workshop. Also, a requirement for mentee teams to complete a pre-workshop questionnaire was added to the mentoring process. The observation that most mentee teams struggled with preparing NSF budgets and associated justification resulted in the addition of budget reviews by a leadership team member.

Related Research Studies

In 2020, two separate studies were conducted at the University of North Carolina (UNC), Chapel Hill, NC, which provided a two-day training course in coaching skills to UNC faculty and permanent staff [10, 11]. UNC’s research showed positive behavioral effects accruing to attendees from in-person and virtual classes in coaching skills that utilized the coaching skills in interactions with students and peers. A second study by M. Erikson at Providence College, RI, trained graduate and undergraduate students in basic coaching skills, which they applied in peer-coaching situations throughout one semester. Their research found that “peer coaching positively affected students’ personal and leadership development.” However, having a well-trained coach to deliver the coaching educational aspects was crucial. Subsequent literature searches identified research on using coaching with teams [14] with implementing a “brain-based” approach to coaching [15], and a new frontier linking neuroscience and coaching [16, 17]. While research into coaching effectiveness is increasing, the number of studies utilizing a combination of mentoring and coaching with faculty subjects remains few.

NSF Project after the Pilot Year (2020): FORCCE-ATE

A new ATE project was proposed for the next grant submission cycle based on the lessons learned during the pilot year and related research findings previously described. Several added elements differentiated the proposed project from the pilot project. ICF-certified coach training research [18] showed that the mentors benefited from coaching skills, especially during the idea development phase of the grant creation process. In mentor preparation, a unique educational component added basic coaching skills to the existing mentor skills. Consequently, a co-principal investigator with formal training in coaching was selected to handle the mentor-cohort training. The working hypothesis was that mentoring, augmented by coaching skills, would enhance creativity, team focus, and buy-in and increase the overall impact of the mentor-coach role. With more engaged and focused grant writing teams, the effect should also be seen in an increase in the number of successfully funded grant proposals to the NSF ATE program solicitation. NSF funded the submission for the 2021-2024 cycle, and the FORtifying Cybersecurity and Computing Education through ATE grants (FORCCE-ATE) project (DUE #2055250) became a reality.

Year One (2021-2022)

Results

As with the previous year, the mentoring-coaching with the mentee cohort ended in October 2022 at the ATE submission deadline. Of the 10 college teams participating in the first cohort of FORCCE-ATE, four (4) submitted an NSF ATE grant proposal. Out of those submissions, two (2) proposals were subsequently funded. According to NSF, the number of awards trended downward soon after the long-term and societal effects of the 2020 global pandemic. Even with this downward trend, the FORCCE-ATE funding rate was 50%, as compared with the overall NSF funding rate of approximately 20% in the years immediately following the pandemic [19].

Discussion

The evaluation plan for determining the effects of providing coach skills training gathered data from the ICF coaches and the mentor-coach cohort. The ICF coaches were interviewed by the project evaluator in one-on-one settings. The coaches all agreed that the elements within the training had given the students the knowledge to listen deeply and ask probing, open-ended questions. Over the three practicum sessions, they reported seeing growth in applying skills. One recommendation was to augment the training with an ongoing practice plan to help cement the skills presented in the class training. Also recommended was adding the viewing of one complete coaching session before the first triads to give the students a better understanding of the dynamics of a real-life coaching session. Assessment of their coaching skills training and practice sessions by the mentor-coaches was done via both surveys and interviews by the project evaluator. The first survey was released after the completion of the training and triads. This survey was designed as a self-assessment focusing on perceived gains in knowledge and skills, relevance of training to the mentor-coaching assignment, and generally applicable, observed benefits to the mentor-coaches.

The evaluator’s report noted that “participants overwhelmingly enjoyed the training and found the triads to be a particularly valuable learning experience.” The survey also showed that participants felt the need to listen more intently. On a more granular level, 75% of the respondents felt that their ability to evaluate an individual’s statements and behaviors had been increased.

One consistent theme was that while the mentor-coaches strongly endorsed the coaching skills training, they found it challenging to know when to coach versus mentor. They also identified a need for specific training on how to coach teams.

In terms of the biggest “take-aways,” statements included the following:

“Coaching is well-suited for use during ideation (i.e., idea creation).”

“The value of using coaching skills became clear.”

“Most useful in the learning was how to optimize the client’s experience with powerful questions.”

When asked about how they envision applying their newly acquired coaching skills, answers revolved around the power of listening and asking open-ended questions. Examples:

“I loved the idea of not ‘fixing’ the client’s issues but rather questioning to help them find solutions.”

“I hope to have improved interactions with peers and students.”

To evaluate the mentor-coach process, participants were asked to respond to questions about their perception of the training sessions’ content and delivery, their learning gains, and their preparedness to serve as a mentor-coach. In Figure 4, while the original scale ranged from “strongly agree” to “strongly disagree,” there were zero responses to the strongly disagree column. This figure condenses the categories in line with participants’ responses.

Figure 5 expands upon the evaluation report findings from Year 1 to include mentor-coach perceptions.

While it would be helpful to compare the results from the 2020 proposal submissions with those from 2022, the noticeably different characteristics of the applicant teams precluded that comparison. Some teams were directed to participate by their upper management, whether they were interested or not. Others had no institutional capacity or support, lacking grants offices or anyone with local NSF grant development experience. While all the mentee teams received grant training at the June workshop and all were assigned competent mentor-coaches there were other factors in play, including two years of the societal effects of the global pandemic that could not be mitigated by the expertise and efforts of the FORCCE-ATE leadership or mentor-coach staff.

Year Two (2022-2023)

Results

At the close of the October 2023 NSF grant filing period, 12 of the participating 16 teams reported submitting their grant proposals. At the time of publication, six mentee colleges received an ATE award. The four (4) remaining mentee teams chose to wait until the 2024-25 grant cycle to submit. See Figure 6 for additional background information on the participating colleges. Click the link to access the interactive map.

Link to active map: https://www.google.com/maps/d/u/0/edit?mid=19kfcCXv2igz6AlgcUW8EKrO82nqXfTo&usp=sharing

Discussion

The assessments of coach training, triads, and instructional materials were again evaluated via two survey instruments and a series of stakeholder interviews. In the post-training survey, mentor-coaches again reported that they valued the training in deep listening and asking powerful questions and were looking forward to putting these skills into practice with their mentee teams. They felt they now understood the difference between mentoring, which is more prescriptive, and coaching, which supports solution-based, higher-level thinking among those being coached.

In the survey administered in late August 2023, after four months of working with their mentee teams, the respondents felt they had more of a practical understanding of utilizing mentoring and coaching skills during the grant development process. Their self-assessment also revealed that all but one thought they had made gains in their coaching skills, including how to facilitate actions that would advance the grant development process. The following are individual comments from the mentor-coaches.

• On training: “I learned new ways to get to the root of the question that the client wants to discuss.”
• Biggest takeaway: “The importance of careful listening as a coach.”
• On future application: “I see that coaching works in teaching, especially when students ask questions about lessons they’ve learned in a course.”

As part of the year two project evaluation, participants were asked to respond to a series of questions about their individual learning gains from the training sessions and triads. It should be noted that for Figures 7 and 8, the same respondent chose “strongly disagree” for every survey item even though s/he provided positive comments for the open-ended questions. This respondent likely misread the direction of the Likert-style scale.

Although the study is not complete and the collection of additional quantitative evidence on the effectiveness of providing educator-mentors with coaching skills continues, this observation stands out:

Teaching basic coaching skills of listening and asking thought-provoking questions empowers the recipient (mentors) of the coaching skills to confidently use these skills at the appropriate times in their career and work environment. Their coaching drives their clients (mentee teams) to examine, synthesize, hypothesize, and validate their current understandings while moving toward their expressed goals.

While the collection of quantitative data, such as submission and funded grants, tells one story, the effect that coaching had on the mentee teams is more difficult to assess. When queried, the mentor-coaches concurred that their mentee teams appreciated and responded positively to coaching questions.

“I used coaching to encourage my teams when working on their goals and when they needed to find information that had already been shared in training.”

Self-reflections by the mentor-coaches consistently stated that adding coaching competencies strengthened their mentoring effectiveness and enhanced their communication skills in their professional and private lives. The PI of the Mentor-Connect project, who attended the coaching skills training, reported:

“What an eye-opener and enriching experience! It was extremely helpful to us to understand the difference between mentoring and coaching and to learn what is and isn’t good coaching – not to mention how hard it is to get it right!”

Conclusion and Future Directions

In summary, augmenting the skill set of faculty mentors with fundamental coaching skills for working with individuals and teams resulted in improved communication among the mentors and mentee teams. The coaching skills of deep listening and asking powerful questions served to motivate the college teams to develop a competitive ATE grant proposal. The combination of mentoring and coaching skills also empowered the mentor-coaches to use these newly acquired skills professionally and personally. The FORCCE-ATE research conclusions were consistent with the results of other studies cited in this paper. These results confirmed that the combination of coaching and mentoring skills is superior to relying solely on mentoring. Combining coaching with mentoring can be considered a viable, promising practice. In particular, the use of probing questions by the mentor-coaches was the impetus for high-level and creative thinking on the part of the mentee teams. An unanticipated but significant impact on the ATE community will be the expansion of the national pool of mentors equipped with coaching competencies.

Funded for three years, the FORCCE-ATE project will conclude in Fall 2024 with the October ATE submission deadline. Because of the long lead time needed for the NSF review and selection process can take up to nine months post-submission for notification letters to be sent to the awardee colleges. In the meantime, the submission rate of the mentee colleges provides valuable and impressive comparison data for three years. The final FORCCE-ATE funding rate can only be calculated once the last mentee team is officially notified in 2025. The future summative findings of the NSF FORCCE-ATE project will encompass four years of quantitative and qualitative data, including the pilot year. At the conclusion of the FORCCE-ATE initiative, overall project findings, including submission and funding rates, will be made available for the benefit of the broader ATE community.

Acknowledgements. This article reflects the planning and execution of the NSF FORCCE-ATE project, award #205525. We want to acknowledge the following individuals and groups who were instrumental in developing this research paper. First, we recognize our J ATE coach, Dr. Thomas Stoebe, who shared his experience crafting publication-worthy research papers. Next, we owe a debt of gratitude to our project evaluator, Dr. Blake Urbach, who designed the data collection instruments and produced the tailored dashboards that appear in Figures 4, 5, 7, and 8. Next, we are grateful to our Co-PI, Dr. Vera Zdravkovich, and all the FORCCE-ATE mentor-coaches for coming along this incredible journey of discovery with us over the years. We acknowledge the design skills of the Nelly Group that translated textual descriptions into visually appealing graphics for figures 1 and 2. Last but not least, we thank Elaine Craft of Mentor-Connect for participating in and supporting our mentor-coach training and for sharing her significant experience in mentor training for the ATE grant development process.

Disclosures. The authors declare no conflicts of interest.