Colleges Must Move Beyond AI Literacy Pilots

Universities are investing in AI literacy programs, but a professor argues they lack evidence of real-world effectiveness. Ngoc Cindy Pham, writing for The Hechinger Report, states that higher education has an "AI pilot program problem" where success is often declared prematurely based on limited data.

The Problem with Current Programs

Colleges frequently measure program success with short-term metrics like workshop attendance, certificates, or student satisfaction. Pham contends these are insufficient. The core issue is that these initiatives are rarely evaluated based on whether students can actually apply AI skills to solve workplace problems after graduation. Also, it is unclear if these programs give students from different types of institutions a fair shot at competing for jobs.

Knowing how to use an AI tool is not the same as applying it responsibly. A student might prompt a chatbot but not know how to verify its output, protect private data, or decide when the tool is inappropriate. "Those abilities develop through repeated practice in real contexts," Pham writes. She calls for colleges to adopt shared measures of true AI workforce readiness.

Evidence from a Cross-Campus Initiative

Pham founded an initiative to study the challenge, bringing together undergraduates from CUNY and graduate engineering students from NYU Tandon for a series of hands-on workshops with industry professionals. A preliminary study found that the amount of participation, and not the institution, was the clearest predictor of applied AI use.

The data showed a direct correlation between workshop attendance and practical application.

The study did not detect a performance difference between the two institutional groups on this measure. This suggests program design and the volume of meaningful practice may be more critical than a student's starting point. For example, a student's progress should be measured by their ability to create useful work, which is a key part of workforce stats.

A Three-Part Path Forward

Pham proposes a three-part approach for universities to improve their AI training efforts. First, colleges must evaluate programs by measuring changes in student judgment, problem-solving, professional communication, and ability to create useful work with AI. They should also track tangible outcomes like internships or job placements, which could be reflected in graduate standings.

Second, universities must study programs across different institution types. Findings from a well-resourced university cannot be assumed to apply to a public or less-resourced college. Cross-institutional comparison is essential.

Third, employers need to be integral partners in program design, not only occasional guests. They can help define real problems, review student work, and clarify which abilities matter on the job. Students need contact with professionals who model responsible AI use within an actual organization.

Connecting to Workforce Policy

Federal workforce policy is beginning to emphasize measurable outcomes. In July, the U.S. Department of Labor awarded nearly $162 million to expand Registered Apprenticeship programs, using incentives tied to hiring and retaining apprentices. One grant of $40 million went to Jobs for the Future to support apprenticeships in infrastructure roles supporting AI and other industries.

While colleges cannot replicate the apprenticeship model directly, they can aim for its core principles: training that leads to demonstrated skills, meaningful employer involvement, and measurable results. Future grants for university AI programs should, Pham argues, require shared measures, cross-campus comparisons, and follow-up tracking of graduates. Without such evidence, AI workforce readiness will remain only a collection of promising campus stories.