Scaling Course Development with an AI Automation with Human-in-the-Loop System: Contractor Perceptions of Workflow, Scalability, and Quality

Abstract

Recent advances in generative artificial intelligence (AI) have opened new possibilities for accelerating course production in higher education, yet they have also raised concerns about instructional coherence, alignment, and ethical oversight when content is generated at scale. These tensions are especially consequential in high-stakes fields such as nursing, where poorly sequenced or misaligned learning materials can affect clinical readiness and patient safety. As institutions face mounting pressure to expand online offerings quickly, there is a growing need for development models that preserve pedagogical rigor while leveraging automation for efficiency. The rapid growth of artificial intelligence (AI) in higher education has intensified the need to strike a balance between scalability and pedagogical rigor. This convergent mixed-methods study examined how an AI-enabled automation and human-in-the-loop (HITL) system, grounded in the Learner Journey Framework (LJF), supported large-scale course development in a nursing curriculum redevelopment initiative. Quantitative survey data from eight instructional design contractors assessed workflow efficiency, mapping clarity, scaffolding, and perceived rigor, while qualitative interviews explored their lived experiences with the AI-assisted process. Across nineteen courses comprising more than 22,000 instructional assets, participants reported high consensus that the workflow was efficient (Mean [SD] = 4.81 [0.29]), clearly structured (4.78 [0.34]), and pedagogically sound (4.70 [0.33]). Thematic analysis revealed five overarching themes: distributed expertise and structural scaffolding, dialogic human–AI interaction, objective-driven prompting, scalability through structured collaboration, and scaffolding as a learner-pathway mechanism. Together, these findings demonstrate that when automation is disciplined by instructional theory, scalability and rigor can coexist. The LJF’s objective-based structure enabled AI agents and human reviewers to co-produce coherent, ethically transparent, and inclusive learning materials at a rapid pace. This study contributes empirical evidence that pedagogically governed AI systems can sustain instructional alignment and ethical oversight at scale, reframing automation as an extension of pedagogy rather than a threat to it.

Keywords: Instructional design, artificial intelligence, human-in-the-loop, scalability, course development, nursing education, Learner Journey Framework, pedagogical alignment, ethics, mixed-methods research

DOI: 10.7176/JEP/16-13-11

Publication date: December 30th 2025