Quantum repeater protocols stop splitting the network into routes: reinforcement learning lifts end-to-end entanglement rate above routed swap-ASAP on intersecting networks
Synopsis
On non-linear quantum networks, entanglement routing splits the network into disjoint paths and runs chain protocols on each in parallel, shrinking the search space. This work instead uses reinforcement learning to search the larger non-routing protocol space directly, and finds end-to-end logarithmic-negativity rates above routed swap-ASAP on an eight-station intersecting network and other small networks, even in the parameter regime where the latter is widely regarded as state of the art. The authors argue routing should be discarded in favor of non-routing methods, and give a network score tied to the size of the improvement.
Interpretation
Quantum repeater protocols no longer have to allocate entanglement along disjoint paths first; they can be searched directly in the full non-routing protocol space that includes intersecting paths. The prior approach sliced a non-linear network into disjoint paths and ran chain protocols on each in parallel, namely entanglement routing, to shrink the search space. On an eight-station intersecting network whose target nodes have degree four and two, with generation and swap success probabilities between 0.7 and 1 and infinite coherence time, the trained policy reaches a higher end-to-end logarithmic-negativity rate than routed swap-ASAP, a gap large in units of their standard deviations.
The search is carried out by a deep reinforcement learning agent that maps network state to the next repeater operation to perform. The algorithm is Reinforce, a policy-gradient method suited to continuous states and discrete actions; the agent is a multilayer neural network with ReLU activations whose output passes through a softmax to become action probabilities. Training uses 400 batches of 100 episodes of 20 time steps each with the ADAM optimizer, and performance is measured over 200 simulations after training, with logarithmic-negativity rate as both reward and metric.
To keep the action space growing polynomially rather than exponentially with network size, the agent picks one elementary operation at a time and caches them into composite operations. Modeling all parallelizable operations directly would make the last layer's neuron count grow exponentially with network size, so action serialization keeps only elementary generations or swaps. The state encoding concatenates the Bell-edge adjacency matrix with an action-cache indicator vector and grows polynomially with network size, and action masking sets the probability of physically impossible or certainly failing actions to zero.
Perspective
The demonstrated scope is the authors' network model: individual quantum memories as graph nodes, faulty entanglement generation and swapping, depolarizing decoherence and inhomogeneous parameters, trained and evaluated on an eight-station intersecting network, a length-5 chain, the kite network and square lattices among other small networks. The intended reader is a researcher in quantum networks or reinforcement learning; the authors propose extending the model with distillation, more nuanced classical communication delays, and distributed agents replacing the single agent.
The authors state the conclusion does not hold for all networks: in chains with very high coherence times, Reinforce and routed swap-ASAP perform the same. The explanation of the improvement rests on an accumulated non-commuting score defined as the sum of scores of discarded paths, and its correlation with the improvement comes from the several tested networks rather than a proof. The parameters take generation and swap success probabilities from 0.7 to 1, initial Werner parameter 1 and infinite coherence time, chosen so that swap-ASAP stays close to optimal on each route.
