Turning artificial intelligence research into real-world tools isn’t always straightforward, despite the remarkable progress in recent years ranging from self-driving cars to large language models. Systems that perform well in controlled lab settings often fall short when deployed in homes, in hospitals or on city streets.

Vision-based systems, for example, hold promise for improving safety, monitoring health and aiding rehabilitation. For example, a camera-based system could help a clinician assess a patient’s gait or rehabilitation progress.

In the future, an assistive robot in an older adult’s home might need to recognize when the person is struggling to stand or reaching for an object. But a major challenge is ensuring these robots perform reliably beyond the lab, particularly in messy, unpredictable or low-visibility conditions.

As a computer scientist who studies AI systems designed to understand human behavior – especially through visual cues like body movements and interactions with objects – I see three main challenges to taking AI from the lab to the real world: generalization, human behavior and resources.

Challenge 1: The generalization gap

Most AI vision systems are trained on well-lit, high-quality images, often from curated sources like motion capture studios or daytime recordings. But when these systems are deployed in real-world settings, such as in a dimly lit home, hospital room or nighttime street, they often struggle. This is not because the technology is flawed, but because the data used to train it cannot fully represent the enormous variability of the real world.

Take human pose estimation, for example. This task involves detecting key points on a person’s body, such as joints, to understand how they’re moving. The result resembles a digital skeleton that represents the positions of a person’s arms, legs and other body parts. Pose estimation has been studied for health and rehabilitation applications, including measuring a person’s gait, monitoring rehabilitation exercises and assessing movement patterns associated with the risk of falling. In well-lit........

© The Conversation