Skip to main content
Search

# Fisher Information Matrix

6 related articles

Subscribe to updates
  1. arXiv

    Near-field E-MIMO beamforming now predicts only stream powers: 1.11 ms direct inference, with feasibility reaching 100% on the retained test set after verification and repair

    Beamforming for near-field extreme massive MIMO integrated sensing and communication usually requires repeatedly solving nonconvex constrained optimization, and online latency grows with array size. PATBO fixes the communication and sensing beam directions geometrically and lets a Transformer predict only the normalized stream powers, trained with Cramér–Rao bound (CRB) sensitivity weighting. On a retained test set with 1024 antennas, 4 users and 2 targets, direct inference reaches 19.54% joint feasibility, physics-constrained power repair raises it to 87.79%, and optimizer fallback for the remainder brings it to 100%, at 1.11 ms per sample.

  2. arXiv

    CANS gives every feed-forward neuron its own scaling coefficient, steering safety, toxicity and hallucination while holding general capability

    Behavior control in large language models often costs general capability; CANS instead assigns each feed-forward network neuron a scaling coefficient that accounts both for its contribution to the target behavior and for its disruption of general outputs. Target behavior is proxied by the log-probability of target tokens in model responses, capability preservation by output-distribution KL divergence on general-purpose text, and local approximations of the two yield a closed-form solution with no iterative training and no added inference-time computation. On Qwen3-4B and Llama models, CANS matches or beats SFT, CAA and neuron-level baselines on safety, toxicity and hallucination control while better preserving MMLU, BBH and IFEval performance.

  3. arXiv

    EWC stops only guarding old weights: FS-EWC shapes the solution first, then protects it, beating EWC across LoRA benchmarks

    In replay-free continual learning, EWC penalizes changes to parameters deemed important using Fisher information estimated at the retained solution, but that local estimate does not capture sensitivity after the parameters move. FS-EWC uses Fisher information earlier, shaping the current task's solution by minimizing the worst-case Fisher trace over a neighborhood, and builds the retained anchor by averaging checkpoints along the optimization trajectory. Across vision and language benchmarks, FS-EWC generally improves on the corresponding EWC baseline under both full-parameter fine-tuning and LoRA.

  4. Hasegawa and Ohzeki compare Ising and QUBO encodings under fixed model, sampler, and step size, finding QUBO has lower Fisher spectral entropy and slower SGD convergence while natural gradient descent makes the encodings converge alike

    Under a controlled protocol that fixes the Boltzmann machine model, simulated-annealing sampler, and learning-rate design, the study compares Ising ({−1,+1}) and QUBO ({0,1}) variable encodings, exploits the identity that the Fisher information matrix equals the covariance of sufficient statistics to visualize empirical moments, and finds that QUBO induces larger cross terms between first- and second-order statistics, creating more small-eigenvalue directions and lowering spectral entropy, which explains slower convergence under stochastic gradient descent, whereas natural gradient descent, which rescales updates by the Fisher information matrix metric, achieves similar convergence across encodings due to reparameterization invariance.

  5. arXiv

    1% of Tokens Can Match Full Distillation: IER Re-ranks Sparse Supervision by Gradient-Estimation Reliability

    The work reframes token selection in on-policy distillation (OPD) as a gradient-estimation reliability problem at a fixed prefix, decomposes the one-sample reverse-KL gradient into signal and sampling noise in information geometry, proposes the information-efficiency ratio (IER) as a signal-to-noise ratio under the optimal scalar baseline, and approximates it on a top-K candidate set for ranking; on mathematical and medical reasoning, IER alone approaches or exceeds full OPD at token budgets of 0.1%–1%, and combined with existing usefulness scores (IER-OR, IER-AND) it matches or exceeds full OPD in multiple settings.

  6. arXiv

    Random matrix theory yields a closed-form noise-error bound for reconstructive spectrometers and predicts a 17.9 µm optimal cavity in on-chip FDTD

    Using Fisher information and random matrix theory, the authors derive a closed-form expression for the noise-induced error of chaotic/diffusive reconstructive spectrometers, linking the variance bound σ²_ε Tr[(AᵀA)⁺] to the spectral correlation length Γ_corr, mean transmittance T₀, and the numbers of frequency channels N and measurement channels M, establish conditions for super-resolution, and validate the theory with a random matrix model and full-wave FDTD simulations, where the predicted optimal on-chip cavity size is about 17.9 µm.

You’re all caught up