10/20/2026
How ScribeAR, a local caption service, developed from an innovative research project into classroom technology at the Grainger College of Engineering.
ScribeAR: Making Accessibility Look Like Magic
"It's like magic!"
ScribeAR is an innovative, open-source live transcription technology designed to make learning more accessible.
Don't miss the double meaning in this exclamation from Melissa Whitt, senior eLearning Specialist at Engineering IT. At first, it's easy to understand her amazement as she watches ScribeAR in action—spoken words appearing almost instantly as captions on multiple displays during a demonstration in 2406 Siebel Center for Computer Science. What is less obvious is the equally remarkable collaboration behind the scenes: faculty innovators, undergraduate researchers, and Engineering IT professionals bringing their expertise together to make this technology available in classrooms.
ScribeAR is an innovative, open-source live transcription technology designed to make learning more accessible. What began as a collaboration between a physics researcher seeking a better way to follow classroom instruction and a computer science professor passionate about accessible technology has grown into something much larger. Through years of student research and development, experimentation, and collaboration with Engineering IT, ScribeAR has evolved from a promising prototype into a technology being tested and supported across multiple instructional spaces. And the people involved are already imagining what comes next.
The Challenge: Captions That Work for Learning
Captions are everywhere, but simply displaying spoken words doesn't always make information accessible. In classrooms, students must simultaneously follow instructors' gestures, demonstrations, diagrams, and explanations—often while navigating specialized terminology that automated transcription struggles to recognize. For students who are Deaf or hard of hearing, these challenges can create significant barriers. For others, captions can support focus, comprehension, and language learning. Recognizing these different needs, Colin Lualdi and Professor Lawrence Angrave began exploring a more flexible, privacy-conscious approach to captioning that could improve the learning experience for everyone.
Read more: The Accessibility Challenge That Inspired ScribeAR
Colin Lualdi, a physics doctoral student, now a Ph.D. alumnus, who is Deaf, understood the limitations of traditional captioning firsthand. Following a physics lecture often meant shifting his attention between an interpreter or captions and an instructor's gestures, equations, or demonstrations. When a professor pointed to a formula and said "this" or "that," looking away at the wrong moment could mean missing the context necessary to understand the explanation. Colin began experimenting with augmented reality (AR) glasses to keep captions within his field of vision while watching the instructor.
Colin connected with Lawrence Angrave, a Teaching Professor in the Siebel School of Computing and Data Science, whose interests in accessibility, computer science, and instructional technology made him an enthusiastic collaborator. Together, they explored how captioning could be more responsive to the realities of learning. Their goal wasn't simply to transcribe speech, but to improve how students experience and use captions, including reducing delays, accommodating different interfaces, and addressing errors involving specialized academic language.
Their work also emphasized privacy and universal access. ScribeAR's locally hosted transcription approach helps keep classroom audio from being sent to external services. Captions can be delivered to classroom displays, personal devices, and AR glasses, allowing students to choose what works for them without having to identify themselves as needing an accommodation. The result is an approach to accessibility that recognizes a simple truth: when learning tools are flexible and widely available, everyone has the opportunity to benefit.
Innovative Iterations: Cookies, Coding, and a Community of Researchers
Turning that vision into working technology became an ongoing experiment fueled by undergraduate researchers and Angrave's weekly "Cookies and Coding" gatherings. Students brought interests in artificial intelligence, natural language processing, embedded systems, and interface design, developing successive versions of ScribeAR while learning from one another. Along the way, they earned research recognition, shared demonstrations at Engineering Open House, and discovered how meaningful it could be to build technology that helps others. Testing ScribeAR in real classes in the Grainger College of Engineering’s Test Kitchen demonstrated its potential—and raised the next challenge: how could an experimental system become reliable, sustainable, and available in more classrooms?
Read more: From Cookies and Coding to Classroom Innovation
Angrave's informal "Cookies and Coding" gatherings created an inviting entry point into undergraduate research. Students could ask questions, explore technical challenges, and contribute to projects with real-world applications. Alumni Himnish Jain and Eisha Peyyeti, both Illinois 2025 graduates, described finding not only opportunities to develop advanced technical skills, but also a community united by the desire to improve accessibility. Himnish explored natural language processing and AI technologies, while Eisha brought interests in embedded systems and the challenges of delivering useful technology through accessible interfaces.
The students' work extended well beyond writing code. Eisha recalls vividly how she helped introduce ScribeAR at Engineering Open House (EOH) to visitors of all ages, inviting them to imagine how live captions might be useful in their own lives. Student researchers also earned recognition at the University's Undergraduate Research Symposium, including Honorable Mention in Oral Presentations in Honorable Mention Oral Presentations in 2024 and Outstanding Performance & Exhibit in 2025. Both alumni described how the experience influenced their perspectives on accessibility, collaboration, and the responsibilities of technology professionals - lessons they have carried into their careers.
Successive student teams experimented with transcription models, captioning interfaces, and ways to improve accuracy and responsiveness. During Fall 2025 and Spring 2026, ScribeAR was used in actual classes in the Test Kitchen, providing an opportunity to observe how it performed over an entire semester. Those experiences demonstrated the technology's promise but also revealed the practical challenges of broader deployment: capturing dependable audio, managing classroom devices, supporting multiple simultaneous sessions, and keeping experimental software reliable for instructors and students.
Photo Credit: Eisha Peyyeti
Collaborative Solutions: Bringing the Pieces Together
That's where Engineering IT's collaborative expertise became essential. Audio-Visual Services, Instructional Services, infrastructure specialists, and project coordinators joined Angrave and the research team to solve challenges that crossed traditional technical boundaries. Together, they developed practical approaches for integrating classroom microphones, displays, servers, and transcription software while establishing the documentation, monitoring, and processes necessary for long-term support. By balancing ongoing innovation with stable classroom operations—and making thoughtful use of existing resources - the team transformed individual technical contributions into a coordinated solution with the potential to reach more learners.
Read more: The Engineering IT Collaboration Behind the Magic
The collaboration began organically as Engineering IT staff learned about Angrave's work and identified opportunities to help. Nick Day and the Audio-Visual Services team focused on capturing clear classroom audio and delivering captions through existing displays and controls. Using technologies such as Dante audio networking, they found ways to connect classroom microphones to the transcription system. Different instructional spaces required different approaches, from dedicated monitors and computers to iPads and portable equipment. Rather than designing a costly, one-size-fits-all installation, the team looked for practical solutions that took advantage of existing technology. Working with Jeffry Royce, Assistant Director of Infrastructure and Facility Operations at the Siebel School of Computing and Data Science, allowed for a variety of deployments of ScribeAR into multiple rooms, specifically Siebel Center 0216, 2406 and 4124, on top of the original deployment in the Test Kitchen in the Digital Computer Lab (DCL).
Behind the scenes, Jason Colwell, Luke Scharf, and James Han worked to make the system sustainable. Their efforts included managing transcription servers, deploying and registering kiosk devices, monitoring system health, documenting administrative procedures, and establishing workflows for software updates. Rob Bennett contributed instructional technology expertise, testing, and feedback, helping connect the experimental system with the realities of classroom support. Melissa Whitt helped coordinate communications with instructors, while Travis Ashmore and Elaine Robbins helped organize the cross-functional effort. Regular meetings and shared communication gave the contributors opportunities to solve problems together and learn from one another.
The team's enthusiasm became one of the project's defining strengths. Luke described their technical problem-solving sessions as a "nerd party," where colleagues eagerly shared expertise and tackled challenges together. Each specialty brought an important piece of the solution, but making those pieces work together was the real achievement. The collaboration also established reusable approaches to deployment and support that could benefit future projects. Just as importantly, it demonstrated how Engineering IT can help faculty innovators move promising ideas beyond experimentation and toward sustainable services that support teaching and learning.
Photo Credit: Nick Day
"Let's Do This Again"
ScribeAR's story is still unfolding. Researchers continue exploring improvements in transcription accuracy, language translation, and personalized captioning, while Engineering IT considers how to support additional classrooms and future iterations. But the project's success already extends beyond the technology itself. It has expanded opportunities for accessible learning, helped undergraduate researchers develop technical skills and a sense of purpose, and demonstrated what can happen when people across specialties unite around a meaningful challenge. For the Engineering IT team, the experience has inspired enthusiasm for future collaborations. As Luke put it, "Let's do this again."
Acknowledgments:
Lawrence Angrave, Colin Lualdi, Eisha Peyyeti, Himnish Jain, Jeffry Royce, Travis Ashmore, Rob Bennett, Jason Colwell, James Han, Elaine Robbins, Luke Scharf, and Melissa Whitt
Disclosure: Generative artificial intelligence (GenAI) was used to assist with drafting and editing portions of this article. All facts, content, and final editorial decisions were reviewed and approved by project members and Engineering IT staff.