The High-Stakes Problem of Tracking Nutrition in Long-Term Care

The Flaw in Visual Estimates

In many facilities, a resident's nutritional health is tracked by a staff member glancing at a meal tray. These manual assessments are notoriously unreliable, with error rates as high as 66%. This is especially critical given that roughly 50% of residents in these settings are already malnourished or at risk.

The Cost of a Missed Bite

The stakes are immense but invisible. When a caregiver fails to notice that a resident ate their potatoes but left their protein untouched, a vital window for early clinical intervention closes silently.

Affordable, Powerful Hardware

The system utilizes an Intel RealSense RGB-D camera—a piece of hardware costing less than $200 USD. This camera captures both color and depth data to reconstruct the 3D volume of a meal, creating a volumetric map that translates physics into pharmacology.

Exceptional Accuracy in Measurement

The results of technical validation show remarkable precision:

• Caloric Intake: Estimation showed a correlation of r² = 0.94 compared to gold-standard weighed records, with a mean bias of only -2.7 kcal.
• Macro-Nutrient Tracking: Accuracy was even higher for critical nutrients:
  • Protein intake tracking achieved an r² of 0.96.
  • Fiber tracking reached 0.95.

Handling Real-World Complexity

This granularity is rare. AFINI-T was tested against 104 food classes, including challenging "visually discordant" modified texture foods (MTF) like purées. While standard AI fails on indistinguishable mash, AFINI-T maintained an overall top-1 classification accuracy of 88.9%.

A Multi-Stage Computational Process

The system's power comes from a specialized pipeline:

1. A deep convolutional neural network separates food from the plate.
2. A convolutional autoencoder identifies what is being eaten.
3. The identified food is linked to a nutritional lookup table.

This process allows the system to monitor 13 different macro- and micronutrients simultaneously.

The Occlusion Conundrum

The technology faces a physical challenge. Because the camera looks down from above, it struggles with "air pockets" in foods like stacked toast or loose salads, which can lead to volumetric overestimation.

The Modified Texture Food Challenge

While regular foods achieved a 93.9% accuracy rate, the MTF accuracy dipped to 73.7% due to the inherent visual similarity of different puréed meals.

Remaining Manual Step

Currently, the system still requires a human to hand-label one reference image per meal to set the daily menu.