Under prism shift, the arm stops learning from scratch: parallel pretrained policies adapt in about 12% of a naive network's median reach time
Synopsis
When prism glasses shift the visual field, canonical neural networks previously had to relearn an action policy from scratch, with error falling only as 1/t. The authors let a network hold several policy matrices pretrained for different target positions in parallel and select among them through a mixture weight λ that acts like an attentional switch. In a 10×10 grid arm-reaching task, this transfer learner cut reach duration within about 5 trials of prism exposure and showed an after-effect once the prism was removed; on the first trial after the shift, its median reach duration was about 12% of the naive learner's.
Interpretation
After a prism shift, the network does not relearn the sensorimotor mapping; it selects one of its existing policies and reweights them, so reach time falls back within a few trials. Earlier canonical neural networks had to learn a generative model for a new environment from scratch, with generalization error decaying slowly at 1/t order, which sits poorly with human prism adaptation completed in roughly 10 trials. In a three-phase simulation on a 10×10 grid with 200 trials per run, the transfer learner's reach duration dropped after about 5 trials of prism exposure and converged around 10, and an after-effect appeared after prism removal, whereas naive learners that learn the policy matrix directly failed to reproduce the after-effect.
The implementation combines several pretrained policy matrices at the output layer through mixture weights that sensory evidence drives toward the most plausible one. A naive learner updates a single policy matrix V, while the transfer learner uses λ to select and combine pretrained V_k, equivalent to modulating a mixture of policy matrices in a partially observable Markov decision process. Each policy matrix was pretrained for one target position, with 100 matrices covering the 10×10 grid; in simulation λ shifted toward the policy for the new target position during prism exposure and returned to the baseline policy afterward.
This discrete selection yields exponential-order convergence, faster than the power-law order of continuous parameter relearning. The naive learner's policy-matrix posterior follows a Dirichlet distribution and its generalization error decays as 1/t, whereas the transfer learner's mixture weights converge exponentially with the evidence gap. In a simplified simulation with random-walk actions, the transfer learner's mixture-weight generalization error converged in about 2400 steps and appeared as a straight line on a semi-log plot, while the naive learner needed longer to align with the 1/t line on a log-log plot.
Perspective
The demonstrated scope is an arm-reaching task on a 10×10 discrete grid with simulated prism shift, where states and actions are discrete; the authors note that whether a mixture model extended to continuous environments still shows exponential decay remains to be evaluated.
The authors state that the architecture is more memory-consuming than naive networks, so its advantage should be understood as holding under sufficient memory resources; adaptation fails when the prism shift exceeds the 4-grid Manhattan distance covered by pretrained policies; adaptation speed falls sensitively as the risk value Γ is attenuated, while after-effects diminish only when the risk contrast is low enough.
