Skip to main content
Back to timeline
arXivSource publication:

One scoop reads granular properties: Latent Filter++ takes raw force–torque signals and estimates grain size and bulk density even on unseen materials

Synopsis

When a robot scoops granular material, the force–torque signal mixes the bulk load with higher-frequency grain-scale contacts, and Latent Filter++ combines a learnable frequency–temporal filter bank with a causal latent dynamical model to infer material properties online from that raw signal. The earlier latent-filter approach relied on heuristic preprocessing to extract spatial and frequency components, which introduced non-causal dependencies and left generalization to materials outside the training distribution unexamined. With materials held out, the learned representation estimates both grain size and bulk density better than direct supervised regression, and a few new scoop samples suffice to calibrate the property readout.

Source-provided article image: Interactive Perception of Granular Materials via Robotic Scooping and Force-Torque Sensing
arXiv

Interpretation

A robot scooping granular material can infer grain size and bulk density online from wrist force–torque readings alone, without first hand-decomposing the signal. The earlier latent-filter approach relied on heuristic preprocessing to extract spatial and frequency components, which introduced non-causal dependencies that limited direct online deployment, and it did not explicitly evaluate generalization to materials outside the training distribution. Before the scoop ends, Latent Filter++ estimates both grain size and bulk density with lower error than the prior approach, and the comparison is made on the same observation buffers.

The model treats a scoop as a latent state that evolves over time, with one part fixed by the current observation and another part representing physical properties that only accumulate evidence over time. The prior method fed the filtered state to an LSTM, merging the measurement and transition distributions and discarding their disagreement. Latent Filter++ adds an innovation pathway that passes the gap between prediction and observation, together with the action, into the LSTM, and masks a fraction of observations during training so the latent state must rely on predicted dynamics; test-time inference uses unmasked observations.

The learned latent representation transfers to materials never seen in training, and a few new scoop samples suffice to calibrate the property readout. A directly supervised two-layer CNN plus LSTM has lower in-distribution grain-size error, but once materials are withheld the proposed model is better on both grain size and bulk density. With five materials held out, the proposed model's grain-size and bulk-density errors are both lower than the supervised model's; in the leave-one-out setting, grain-size error drops markedly as anchors increase from a single sample to forty.

Perspective

The demonstrated setting is a 7-DoF Franka Emika Panda holding a spoon, with a 6-axis ATI force–torque sensor between the end-effector and the spoon, scooping 26 everyday granular materials; the downstream test uses only the 12 of those materials that actually spilled within the robot's kinematic limits.

Bulk density is estimated more reliably than grain size, whose predictions show higher variance; angle of repose is poorly estimated from the scooping interaction, indicating that this property is not sufficiently observable from the available interaction signals. The learned representation does not reliably extrapolate beyond the range represented in training, with the largest leave-one-out errors on powdered sugar, large macaroni, perlite and potting gravel; few-shot adaptation still requires manually selected anchor interactions, because latent prediction variance alone is insufficient to trigger adaptation automatically.