RatioLogo
Back

The Challenge of Eating Gesture Recognition

In a bustling hospital or an aging parent’s kitchen, the difference between a patient staying healthy and one succumbing to malnutrition or obesity often hangs on what they ingest—and how often. For years, scientists have tried to automate "eating gesture recognition" using wrist-worn sensors, but most studies relied on tiny groups of 25 people or fewer.

The Question of Complexity

What if the "grammar" of how we eat is actually more complex than the grammar of language itself?

A new large-scale validation study suggests that previous laboratory "proofs-of-concept" have radically underestimated the messiness of human motion. By deploying sensors in an instrumented cafeteria, researchers found that the path from a fork to a mouth is far more variable than the deliberate, clean motions of sign language.

The Large-Scale Validation

The study transitioned away from small lab samples to a diverse population of 269 subjects, revealing a sharp reality check for the field.

Key Findings of the New Data

  • Scale of Data: Researchers analyzed a massive dataset of 51,614 total gestures, including 18,462 bites and 2,182 drinks.
  • Model Complexity: While sign language recognition typically requires only 3 or 4 hidden states in a mathematical model, identifying a single eating gesture requires 13 states and 5 Gaussians to reach an accuracy plateau.
  • Training Requirements: The point of diminishing returns for model training doesn't happen until a system has seen 500 training samples per gesture type, a significant jump from the 65 samples required for basic viability.

The researchers noted, "The findings provide evidence that the size of a data set typically used to demonstrate laboratory proofs-of-concept may not be sufficiently large enough to capture all the motion variability."

Performance of the Models

Accuracy and Predictability

  • Best-Performing Model: The most advanced model, the HMM-1, achieved an overall accuracy of 89.5% by looking at the gesture immediately preceding the current motion.
  • Limits of Context: Trying to map longer sequences (HMM-2 through HMM-6) failed to improve results.
  • Human Motion Patterns: It turns out that humans are predictable in short bursts—getting a drink is recognized with 96.1% accuracy—but the broader "logic" of a meal is largely non-systematic.

Significant Remaining Hurdles

Major Challenges for the Field

  • Study Limitation: The research only captured one meal per person. We still don't fully understand how an individual’s movements change as they tire or change environments.
  • Annotation Burden: The 1,000+ man-hours required to annotate the video data suggests that while the technology is fast, the human labor required to train it remains a massive hurdle.

Reference: The Impact of Quantity of Training Data on Recognition of Eating Gestures; Authors: Yiru Shen, Eric Muth, and Adam Hoover; Source: arXiv:1812.04513v1 [cs.LG] / IEEE Journal of Biomedical and Health Informatics.