Strategically Aligning Curriculum with Future Employer Needs Using the Skill Standards Process

Introduction

From 2019-2024, the “IT Skill Standards 2020 and Beyond” (ITSS) National Science Foundation (NSF) Advanced Technological Education (ATE) project used the Business and Industry Leadership Team (BILT) Model – developed by the National Convergence Technology Center, which was funded 2012-2023 by a National Science Foundation ATE grant (DUE1700530) – to create and update future-focused, employer-led skill standards for the six most critical IT (information technology) job clusters. Prior to the ITSS grant, this kind of detailed job skill identification and evaluation work had not been undertaken at a national level since 2003, when the NSF funded a skill standards project for the National Workforce Center for Emerging Technologies (NWCET) in Bellevue, Washington. This skill standards work is essential because it comes at a time when IT is transforming the business landscape with technologies like AI, 5G, and cloud computing, yet companies are struggling to find qualified workers to support this seismic change as well as standard openings. See Table 1 for details on employers and meetings convened by ITSS for this project. Industry employers were recruited as either “thought leaders” (C-suite executives whose jobs demand innovation and forward-thinking) or “job cluster SMEs” (experts in the day-to-day technical needs of a specific IT discipline). In a few cases, an employer qualified for both categories and attended both sets of meetings. 

Table 1. ITSS meeting details – participants and hours spent 

Meeting participants	Number	Total meetings	Total hours
Thought leaders	142	8	15
Job cluster meeting SMEs	187	45	123 hours

Most of the material developed by NWCET in 2003 – before Facebook, AWS, and the iPhone – is now hopelessly out of date. In a recent Robert Half survey, 90% of technology managers in the U.S. with hiring responsibilities reported experiencing challenges finding skilled candidates [1]. Likewise, of the ten industries cited by Business News Daily in December 2023 as being “most affected” by a skills gap, five are in IT – cybersecurity, machine learning, cloud migration, big data, and computer technology [2]. Contrast those skills gap challenges with Manpower Group’s IT World of Work 2023 Outlook, which noted that one of the key trends driving IT innovation was that “digital transformation is accelerating in every industry” [3]. With such high demand for qualified workers in a climate of rapidly evolving technologies, educators must have as many tools as possible, like the ITSS-created skill standards, to keep the curriculum aligned with the ongoing needs of the workforce. 

Methods

The first step in the ITSS process was to identify the most critical IT-related job clusters on a national level. The ITSS leadership team convened 142 expert “thought leaders” that included CTOs, CIOs, and futurists responsible for keeping their companies in business. These thought leaders met across four meetings (eight hours total) in early 2019 to discuss and develop a list of IT job clusters with significant critical present and future needs of talent. The ITSS grant staff provided thought leaders with metrics on the number of open jobs in 14 clusters, including the number of jobs that remained open six months after they were posted. Ultimately, through a follow-up Delphi set of surveys, ITSS gained consensus across the various meetings. The thought leaders ultimately identified six job clusters: data analytics, data management, infrastructure management, software development, technical support, and technical project management. Additionally, the thought leader group asked to reconvene later to discuss additional job clusters, but in late 2020, they decided that emerging technologies like AI were not yet sufficiently mature enough to warrant creating skill standards and that they had no other job clusters to add. The thought leaders agreed to stick to the original six job clusters; instead of pursuing new clusters, the ITSS team was tasked by the thought leaders with using the remainder of the grant funding period to go back and update the original six job clusters due to the rapidly changing nature of those clusters. 

The next step in the ITSS project was to convene employer subject matter expert (SME) meetings for each job cluster to develop the specific skill standard materials. From 2019-2024, 187 IT experts, including first-line hiring managers, experienced technicians, and high-level technical executives from both small businesses and large corporations, collaborated for well over 130 total hours at job cluster meetings to prioritize and verify knowledge, skills, abilities, and tasks (KSATs) they predicted they would want well-qualified job candidates to possess, usually with two-three employer meetings per cluster. This prioritization, a key feature of the BILT Model, was facilitated by Dr. Beheler through an electronic voting system whereby employers ranked each KSAT item on a 1-4 scale with “1” representing an unneeded KSAT item that can be cut from a program; “4” representing an essential KSAT item that must be included; “3” representing a KSAT item that really should be included; and “2” representing a KSAT item that would be a nice thing to include, but not mandatory. Employers typically spent 20-25 minutes voting, allowing the rest of the 2-hour meeting for a facilitated discussion of the voting results. These discussions allowed for adjustment and editing of the KSAT item statements as desired by the employer subject matter experts. Further, at the end of the facilitated discussion, the facilitator asked each employer group to identify items missing from the pro forma KSAT list. Then, after the results of a given cluster’s employer meetings, the ITSS leadership team synthesized the results into one voting form, including all the votes and discussion results. Note that the KSATs were always developed as “future-facing;” employers were asked to determine what an entry-level worker would need to know 12-36 months in the future to give the educational institutions reaction time to develop/modify curriculum and get it approved. As mentioned, these steps were conducted twice: the original prioritization and then the updated “revision” prioritization. 

Building off those employer-verified KSATs, the ITSS leadership created a list of key performance indicators (KPIs) for the highly prioritized tasks resulting from the meeting. A KPI statement or group of statements tells the educators and the employer how good is good enough performance for a given task in the KSATs. At separate “verification meetings,” employer SMEs voted on and discussed these KPIs in a manner similar to the voting and discussion used in the original votes on the KSATs. Further, they also ranked employability skills (ES), often known as “soft skills,” as those that support technical “hard” skills. Select educators then provided the final element in the skill standard product by creating student learning outcomes (SLOs) aligned with the KSAs. The SLOs were meant for educators to create competency lists to guide implementing the prioritized KSATs into the curriculum. Each of these four “products” (KSATs, KPIs, ES, SLOs) created for each of the six job clusters was packaged together and disseminated on the ITSS website. Additionally, these product packets were also submitted to the Texas Skill Standards System in 2021 for initial approval and dissemination across the state, and the updated set of skill standards information by-product was submitted to this same Texas Skill Standards System in the spring of 2024. This material is still available [4]. 

One quick clarification: thought leaders initially identified cybersecurity as an in-demand IT job cluster, but with so many other resources available to provide educators with cybersecurity job skills, ITSS ultimately worked with employer SMEs to create a single shorter set of cybersecurity “knowledge areas” (the “K” in KSAT) skill set. Notably, the NICE/NIST lists of needed cybersecurity knowledge and skills are updated on an ongoing basis through federal funding. There was no need for the ITSS project to duplicate efforts. 

As mentioned, the innovative cornerstone of the ITSS work is the process of employer vote and discussion of individual KSATs, which comes from the BILT Model. The BILT is the “engine” that powers the ITSS process. The BILT Model addresses what educator Leah Grant calls “a disconnect between the skills students learn in school and the skills employers seek ”[5], which can be a problem with a traditional business advisory council approach where employers might meet less frequently and perhaps focus more on faculty interests rather than discussing in sufficient detail what employers need new graduates to know. 

The BILT Model, which features seven essential elements, upends that format [6]. BILTs offer a structured, repeatable, and efficient process that energizes the relationships between educators and employers to better align curriculum with workforce needs. The secret of the BILT Model is putting employers in a co-leadership role by convening frequent meetings, preferably quarterly, and using annual job skill votes as a forum for employers to explain in detail what they need in entry-level workers, recommendations faculty commit to following if they possibly can. As employers recognize that their time and expertise are valued and that their suggestions are being implemented, they become more committed to the faculty and the students. That commitment eventually extends beyond meetings; as employers become more engaged, they typically begin to participate in mock interviews, capstone classes, job fairs, internships, and guest speakers. Best of all, the educator-employer relationships open the door to graduate hires. As Harvard Business Publishing’s Kenneth R. Lutchen explains, “Businesses benefit from a pipeline of well-prepared personnel, while higher ed institutions gain enviable reputations for providing them” [7]. To date, over 150 organizations and projects across 35 states have adopted the BILT model, including a successful implementation by Central Carolina Community College’s Bioprocess Manufacturing program [8]. The ITSS project has been called a “BILT-plus” approach that supplements the BILT Model’s verified KSATs with other resource products (KPIs, ES, and SLOs). Among the colleges that have successfully implemented the BILT model as a result of working with ITSS are Forsyth Technical Community College (North Carolina), Palm Beach State College (Florida), and Snow College (Utah). Half of the 150 colleges mentioned above that have successfully implemented the BILT were trained by a separate NSF-funded project, Pathways to Innovation, that uses an extensive cohort mentoring model to guide educators step-by-step through the implementation process [9]. 

Alongside disseminating the job cluster skill standard product packets online and at educational conferences, the ITSS project also spent the final year of its funding period hosting multi-day interactive workshops called “ITSS Summits.” Summit attendees learned not only how to find and use the ITSS job cluster product packets but also how to use the ITSS process to develop technical skill standards for any discipline. Attendees – who came from multiple disciplines, not just IT – were encouraged to attend Summits in teams that included at least one faculty and one administrator and were required to develop a customized “action plan” identifying three or more goals the team intended to pursue based on Summit presentations to strengthen their own programs’ relationships with employers and better align curriculum to workforce needs. The Summits walked attendees through the entire skill standard development process, from recruiting employers to hosting the KSAT vote meetings. Several hands-on group activities were included in each Summit, so attendees left with more than lecture knowledge. Overall, six ITSS Summit workshops were held (five in person and one online) for over 100 attendees from 44 schools and organizations in 21 states. 

Results and Discussion

Each Summit attendee was asked to complete two surveys: one survey at the conclusion of the Summit and then a longitudinal survey many weeks later to measure long-term impact and success. See Figure 1 below. 

Survey data immediately following the Summit workshops showed high levels of satisfaction and optimism among attendees. The same questions were posed to five of the six Summit cohorts (the December 2023 online Summit was part of a larger program, and subsequently, those attendees were asked a different set of questions). Fifty-six attendees responded to the survey, representing 62% of the 90 total attendees. When asked if they would “recommend the ITSS Summit to a colleague,” 68% responded “definitely,” with another 30% responding “probably.” See Table 2 below for select responses regarding the content of the Summit. 

Table 2. ITSS Summit survey responses regarding workshop content 

Summit feedback	% who “strongly agreed” or “agreed” (n=56)
The ITSS Summit training, materials and resources are relevant for my work.	96%
The ITSS Summit helped me reflect on how I work with employers.	96%
I could explain to a colleague why a BILT is important.	96%
I understand the difference between a Business Advisory Council and a BILT.	96%

Across the six Summits, attendees developed a total of 54 action plans itemizing 163 customized goals to pursue at their home institution. These 163 goals can be divided into six broad categories. See Table 3. 

Table 3. ITSS Summit “action plan” item categories 

Action plan category	Number and % of action plans (n=163)
Recruit or engage employers to join the BILT or attend a BILT meeting	43 (26.4%)
Prepare for a BILT meeting (includes developing pro forma job skills lists for employer votes)	33 (20.2%)
Host a BILT meeting	32 (19.6%)
Educate administrators and faculty about the BILT	30 (18.4%)
Conduct post-BILT meeting activities (includes cross-referencing BILT feedback to curriculum)	12 (7.4%)
Other	13 (8.0%)

Remember that one of the more challenging parts of standing up a BILT is often dealing with people, whether recruiting busy employers to join the effort or convincing skeptical educator colleagues to embrace a new approach. Almost half of the goals above (44.8%) focused on those elements. 

From October 2023 to July 2024, thirty attendees (representing 27% of the total number of 111 Summit enrollments) responded to a longitudinal survey to share what they’d done since attending the workshop. For most Summit cohorts, this longitudinal survey was a six-month check-in (e.g., April 2023 attendees were surveyed in October 2023), but for the last two Summits (March and April 2024), because of time constraints, this longitudinal survey was a four-month check-in. Twelve attendees (40%) reported making a “great deal” or “moderate” progress in implementing their action plan goals. Of those 12, 10 believed it was “very likely” that they would achieve those goals in the next six months. 

See Table 4 below for responses to the question, “Which of the following have you done since the Summit?” (For comparison purposes, responses to a 12-month longitudinal survey sent only to the April 2023 cohort are also included. Five April 2023 attendees responded.) 

Table 4. ITSS Summit longitudinal impact survey responses 

Action since Summit	Six-month (n=30)	12-month (n=5)
Reviewed the materials that were shared at the Summit	25 (83.3%)	5 (100%)
Shared Summit resources with someone else	18 (60%)	3 (60%)
Met with my employer partners	17 (56.7%)	5 (100%)
Recommended my college use the IT Skills Standards to review our curriculum	15 (50%)	2 (40%)

Note the responses related to meetings with employers, which suggests an implementation attempt beyond internal meetings and hypothetical discussions with school colleagues. Indeed, seventeen six-month longitudinal respondents (56.7%) reported making changes to their employer council since attending the Summit and learning about the benefits of implementing a BILT. Specific changes reported by those attendees include: 

• “We had our first Foundational BILT Meeting in August. We are scheduled to hold our first KSA meeting in December.” 
• “The decision to replace the Advisory Board format for all programs with the BILT process.” 
• “In our advisory committees, we focused on gaining more insight and perspective from our industry partners rather than just reporting out what we are doing.” 
• “I am working on converting existing advisory boards to BILT teams by getting them more involved in the KSA identification and helping them see “WIIFM” [what’s in it for me?]. By answering WIIFM, I am getting more interested employers.” 
• “We have a KSA review scheduled for August and the faculty cross referencing to courses scheduled after.” 

All five 12-month longitudinal respondents reported making changes to their employer council. Specific changes include: 

• “Added more members.” 
• “We now have a fully functioning BILT model in one of our programs. We are looking to launch 1-2 more this next academic year.” 
• “Moved our machine tool program to a BILT-like model.” 
• “We implemented our first BILT advisory board, including our KSA meeting. This implementation has increased interest among our other programs to do the same.” 

These longitudinal surveys provide insight into what Summit attendees did after returning to their home schools: attendees were reviewing the ITSS material, sharing ITSS content with colleagues, engaging with employers, and adjusting the format of their employer meetings to align with BILT principles. 

A more detailed look at the lasting impact of Summit participation can be found by examining the outcomes of a unique mentoring program. In Spring 2024, ten teams from the 2023 Summit cohort opted into a mentoring program with ITSS grant staff. These teams regularly met individually with their mentors from February 2024 to June 2024. Mentors provided individual support and guidance in implementing the Summit team’s action plan items. 

After the mentoring period in July 2024, an impact survey was sent to all ten teams (14 attendees). Nine responded. All nine (100%) called their attendance at the ITSS Summit “very valuable.” Below are select responses to the question, “What impact do you think this work will have on your students or school?”

• “I think this work will increase business partner support, which will help our students have better employment outcomes.” 
• “As our advisory boards shift toward the BILT model, our students will benefit from up-to-the-minute KSAs to make them the most agile and employable folks on the market.” 
• “It will have a significant impact as we get more industry engaged through the BILT model, the pathway for students will be stronger and more accessible with better outcomes for job opportunities.” 
• “Students are getting more opportunities to engage directly with employers.” 
• “It will have a positive impact; they will get the knowledge, skills, and abilities that satisfy industry requirements.” 

As of June 2024, ITSS staff mentors reported that of the 31 goals of the ten mentored schools, 19 (61%) were successfully completed with another 10 (32%) still in progress. One school is no longer pursuing two goals because of institutional decisions. See Table 5 for more specifics.

Table 5. Status of ITSS Summit mentored action plan goals as of June 2024 

Action plan goal	Number and % of total goals (n=31)	# completed
Recruit or engage employers to join the BILT or attend a BILT meeting	9 (29.0%)	6 completed
Prepare for a BILT meeting (includes developing pro forma job skills lists for employer votes)	4 (12.9%)	2 completed
Host a BILT meeting	6 (19.4%)	4 completed
Educate administrators and faculty about the BILT	5 (16.1%)	4 completed
Conduct post-BILT meeting activities (includes cross-referencing BILT feedback to curriculum)	3 (9.6%)	2 completed
Other (increase program visibility via recruiting events; implement new cert program; implement internship program; revise existing program)	4 (12.9%)	1 completed
Totals	31	19 completed

Further details regarding the lasting impact of ITSS Summit training were gathered in the Spring of 2024 when the ITSS project grant published a summary booklet showcasing grant accomplishments. Below are a few testimonials from Summit attendees featured in that book. 

• “This is a model I think all community colleges would benefit from because it really brings the employers in and lets them see what an associate’s degree graduate can do for their company.” 
• “Our IT program had four AAS tracks and one certificate but they all suffered from low enrollments. After adopting the ITSS process and using their skill standard materials, our BILT worked with us to completely change the program. We created a new AAS degree and three new certificates that launched in Fall 2024.” 
• “We’re working to make these employer meetings relevant and industry-led. We’re not going to just have the faculty report out.” 
• “I was already convinced that the BILT Model worked. I had already done some of the training. But when my faculty got fired up and then they took it home and fired up others, getting them to buy in on the ITSS process—that was better than me trying to do it myself as the Dean.” 
• “Reaction from the businesses was very positive. In fact, this year, when we’ve gone out to visit different industries, and we’ve been to several different industries a couple of times by this point, the employers are asking “So when is this meeting?” They want to make sure they’re there. They’re excited because they want to have a say in what we need.” 

One final qualitative data point came via the 2024 HITEC conference, where an “alumni” workshop was convened to allow Summit attendees from the 2023 workshops to share successes and challenges. As part of its workshop report-out, one school (Palm Beach) noted that “We’ve put the BILT into the strategic plan for the college.” 

Summit attendees found value in participating in the 1:1 mentoring. Eight respondents (89%) reported being “very satisfied” with the mentoring; seven (78%) further called the value of their mentor meeting “very valuable.” When asked about the specific impact of the mentoring, only one respondent said “a great deal of success” would have been possible without the mentor’s support. Five others (56%) estimated they’d only have had a “moderate” amount of success. The mentees’ belief that the mentors provided important assistance backs up the ITSS project’s theory that as valuable as the ITSS Summit training was, BILT implementation works better with mentoring support to provide guidance and troubleshoot challenges. 

Conslusion

The “IT Skill Standards 2020 and Beyond” grant sunset on August 31, 2024, but two publications are available for dissemination as legacy documents [10]: an “Impacts” booklet showcasing the project’s successes with metrics, photos, and testimonial quotes, and a “Resources” booklet providing detailed step-by-step directions to implement the ITSS process and copies of the final set of skill standard products for all six IT job clusters (plus the Cybersecurity skill set). The ITSS process and the BILT Model upon which that ITSS process was built provide an actionable blueprint for educators to successfully engage with industry to maximize their time together, deepen the relationship between schools and businesses, and develop detailed, valuable job skills data to make sure students learn what industry needs them to know. Indeed, the new National IT Innovation Center (NITIC), funded by a $7.5 million, five-year National Science Foundation Advanced Technological Education (ATE) grant based on Columbus State Community College in Ohio, will continue to leverage the BILT model and use expert industry feedback to illuminate IT industry trends and in-demand job skills for technical colleges across the country so they can better prepare the IT professionals of tomorrow. NITIC will further build on the success of the ITSS Summits and offer a variation at the High Impact Technology Exchange Conference (HI-TEC) in July 2025. This workshop will allow educators a chance to learn BILT model processes and best practices for implementing a BILT. NITIC also has a detailed “toolkit” that was originally published by the National Convergence Technology and provides step-by-step instructions for developing a BILT [11]. 

Acknowledgements. This work was supported by the National Science Foundation (NSF) under awards DUE ATE #1838535 and #2300188. 

Disclosures. The authors declare no conflicts of interest.