Different paths, same preferences: TRAIL-IRL separates reward from randomness, sorting stork migration into four preference classes and three temperature levels
Synopsis
Unlabeled trajectory clustering has often conflated differing preferences with differing randomness in action choice, because multi-reward inverse reinforcement learning assumes a shared level of action randomness. TRAIL-IRL crosses reward classes with a shared set of temperature levels, so action regularity can vary within a preference group, and it establishes identifiability conditions in a known finite-state environment. In simulations it improves predictive fit, reward recovery and trajectory classification over common-temperature mixtures with the same component count; on 140 white-stork bird-year trajectories, BIC selects four reward classes and three temperature levels.
Interpretation
Different paths need not imply different preferences: individuals may simply choose actions with different degrees of randomness. TRAIL-IRL assigns each unlabeled trajectory both a preference-class label and a level of action regularity, treating the two sources of variation separately. Earlier multi-reward inverse reinforcement learning fixed a single decision-precision parameter shared by all individuals, so randomness differences could be misread as preference heterogeneity; methods that let decision precision vary across demonstrators required known demonstrator identities. With centered and normalized rewards in a known finite-state environment, sufficiently long trajectory distributions determine rewards, temperatures, mixing weights and class counts up to relabeling; under the same conditions, common-temperature reward mixtures with the same component count cannot reproduce a crossed mixture with multiple temperatures. In simulations the crossed mixture improves predictive fit, reward recovery and trajectory classification over common-temperature mixtures with the same component count.
The model pairs every reward with the same set of temperature levels, so one preference group can contain several levels of action regularity and different preference groups can share a level. Temperature in an entropy-regularized decision model controls how concentrated action choices are, with higher temperature meaning more dispersed actions; the crossed structure shares this parameter across reward classes instead of giving each reward class its own. Estimation uses a generalized expectation-maximization algorithm that updates weights, rewards and temperatures in turn, with class counts chosen by the Bayesian information criterion; on 140 bird-year trajectories from 79 white storks, that criterion favors four reward classes and three temperature levels over common-temperature candidates.
In the white-stork autumn migration data, younger birds have higher posterior membership probabilities for a southwestward migration class, while older birds have higher probabilities for a northern retention class. Within reward classes, higher fitted temperatures are associated with less directionally consistent movement and with positions farther ahead along the migration route. These age and position associations come from regressions of posterior membership probabilities and front scores that adjust for calendar year, sex and other covariates; the source relates them to developmental plasticity and social information use as consistent explanations rather than causal findings.
Perspective
The demonstrated scope is identifiability in a known finite-state environment, parameter recovery and model comparison in grid simulations, and clustering plus association analyses on 140 white-stork bird-year trajectories; the stork analysis maps continuous positions onto a grid and assumes rewards and temperatures stay fixed within each trajectory while individuals share one transition mechanism.
The identifiability results rely on centered and normalized rewards, strong connectivity of the state graph and related conditions, and hold only up to relabeling; the age and position associations in the stork analysis are statistical associations from regressions, which the source interprets through developmental plasticity and social information use without establishing causation. The model also assumes rewards and temperatures are fixed within a trajectory, with weather, habitat and individual interactions left to future work.
