The College of Engineering and School of Medicine will take part in a $20 million, five-year National Science Foundation-funded project to harness artificial intelligence (AI) technologies to help children with speech and language challenges.

Computer Science & Engineering Associate Professor David Feil-Seifer is a co-principal investigator on the project, designed to address a nationwide shortage of speech-language pathologists and provide services to children ages 3 to 10 years old who are at increased risk of falling behind in their academic and socio-emotional development.

In addition to Feil-Seifer, School of Medicine Associate Professor and clinical educator in speech-language pathology Abbie Olszewski is a senior personnel member on the project.

Grant will establish new AI institute

The project, led by the University at Buffalo and announced Jan. 9, will establish a national institute — the AI Institute for Exceptional Education — to develop artificial intelligence systems that identify and assist young children with speech and/or language processing challenges. Nine academic institutions will take part: in addition to the University of Nevada, Reno and the University at Buffalo, those are the University of Illinois Urbana-Champaign; Stanford University; the University of Washington; Cornell University; the University of Texas at El Paso; Penn State University; and the University of Oregon.

The University expects to see about $1.25 million in funding from the project, split about 60-40 between the College of Engineering and the School of Medicine, according to Feil-Seifer.

“Information and communication technologies are a focus for the College of Engineering,” Dean Erick Jones said. “Designing the information and communications technologies that will drive the future are a priority, and we are excited to be working with top universities around the country on this project.”

The new AI Institute for Exceptional Education is the 19th AI institute established by the National Science Foundation.

“AI institutes are very prestigious,” Computer Science & Engineering Chair Eelke Folmer said. “This AI institute specifically seems to try to develop an automated, AI-powered solution to help children with speech problems.

“Dave has built an impressive record of doing (AI) research with high societal impact,” Folmer continued, “whether it is developing robots to help kids with autism or using robots in K-12 to get more kids interested in STEM education. Given Dave’s strong track record in this area, I am not surprised he was asked to participate in the $20 million award.”

Using artificial intelligence to assist children with speech and language impairments

The AI Institute for Exceptional Education will focus on serving the millions of children nationwide who, under the Individuals with Disabilities Education Act, require speech and language services. 

School of Medicine Associate Professor Abbie Olzewski is a senior personnel member on the project. 
It is estimated that more than 3.4 million children receive speech and language services under the Individualized with Disabilities Education Act, yet there are less than 61,000 speech-language pathologists to serve them. Due to national speech-language pathologist shortages and late or unmet services during COVID-19 pandemic, these children risk falling behind in their academic and socioemotional development.

The AI Institute for Exceptional Education will address that shortfall with two advanced AI solutions:

The AI Screener will improve early identification of potential speech and/or language impairments and disorders and the AI Orchestrator is an app that will assist speech-language pathologists to provide tailored evidence-based interventions to students.

Feil-Seifer, who works on robotics and robot-human interaction, will be involved both solutions. He will be leading the “embodiment” portion of the project: basically, designing a robot that will interact with young children who have been identified for speech and language assistance.

“When the time comes to give this thing a body, we’re going to be the ones figuring that out,” Feil-Seifer said.

This will be a new technology, Feil-Seifer added, requiring researchers to figure out everything from what sensors will need to be on the robot to how it looks.

“Once we have the physical design picked out, a lot of software technology goes in to making that design work,” Feil-Seifer said. “Our group will develop the software enabling the robot to interact with the environment.”

He added that the technology is not meant to replace existing speech and language practitioners, but to develop technologies that can help speech-language pathologists better serve their clients.

Northern Nevada school districts might be involved in testing, privacy guidelines in place

It may be a while before speech-language pathologists have this technology at their disposal, but northern Nevada schools may see it earlier rather than later, as Feil-Seifer and Olszewski expect some testing to happen locally, with participants’ knowledge.

Olszewski, in the Department of Speech Pathology and Audiology, is an associate professor who also provides clinical education in the University’s Speech and Hearing Clinic. She also is the director of the Language and Literacy Development Center, a specialty clinic within the UNR Speech and Hearing Clinic. Her role in the grant taps her experience in clinical education and 25 years as a speech-language pathologist in educational settings and clinical practice.

“We have the potential to go into the local schools and hope to partner with Washoe County School District,” Olszewski said. “Ultimately, our hope is to serve all Nevada school districts, urban and rural. Our goal is to include children of different cultural and linguistic backgrounds.

“There is a statewide shortage of speech-language pathologists,” Olszewski added, “so this could help to identify children with speech and/or language related concerns earlier and to identify these children we have missed due to insensitive tests.”

The AI Screener will listen to and observe children in the classroom, collecting samples of children’s speech, facial expressions, gestures and other data. It will create weekly summaries of these interactions that help teachers monitor their students’ speech- and language-processing abilities and, if needed, suggest a formal evaluation with a speech-language pathologist.

This is critical because, typically, the earlier speech and language concerns are addressed, the greater the likelihood children will excel academically and socio-emotionally.

The AI Orchestrator is an app that will help speech-language pathologists, most of whom have caseloads so large that they are forced to provide group-based interventions for children instead of individualized care, according to the University at Buffalo. The app addresses this by recommending personalized content tailored to students’ needs. It continues to monitor students’ progress and adjusts lesson plans to ensure that the interventions are working.

“We also hope to have children in speech and language services for shorter time spans due to the improved intervention services with the AI Orchestrator,” Olszewski said. “In other words, the AI Institute for Exceptional Education will leverage artificial intelligence to scale up the services provided by speech-language pathologists.”

The research team will ensure that data collected by the AI Screener and the AI Orchestrator adheres to ethics and privacy guidelines. Information about these plans will be shared with participants. University at Buffalo estimates that the team will deploy prototypes of each system to roughly 80 classrooms, reaching 480 kindergarteners around the country.