Skip to main content
Search

University of Hong Kong

34 related articles

Subscribe to updates
  1. arXiv

    MoE serving re-splits expert replicas every decode step: Relief's integer relaxation nears the per-step optimum, and an on-die engine lifts effective utilization to 99.5%

    In expert-parallel MoE serving, each layer finishes only when its most loaded GPU does, and routing shifts at every decode step. Relief replaces the per-step linear program with an integer bidirectional pairwise relaxation that halves load gaps along replica edges and nears the per-step optimum in a few passes. On decode routing from three open MoE families, scored as effective utilization relative to that optimum, the deployed static scheme reaches 66% to 72%, the GPU kernel 93%, and the on-die engine 99.5%, with the largest recovery over static placement at 37.3 percentage points.

  2. arXiv

    Cross-Layer TopK lets the model split its sparsity budget: lower reconstruction error than per-layer TopK at equal activations, and 2.9× faster training steps

    Cross-layer transcoders (CLTs) decompose a large language model's layer activations into interpretable features, but layers differ widely in how hard they are to reconstruct, so giving every layer a fixed number of active latents is a poor fit. Cross-Layer TopK, from ByteDance Seed and collaborators, ranks the latents of all layers together and keeps a fixed total of K per token, letting the model decide how many each layer gets. On Qwen3-8B-Base, at the same 360 activations per token as per-layer TopK, validation FVU falls from 27.64% to 27.07%, and pairing it with sparse decoding cuts training step time to about one third.

  3. arXiv

    Pension accounts can't short or borrow: a reparameterized neural network cuts the duality gap to 2.87%

    A defined-contribution pension member invests in cash, two nominal bonds, an inflation-linked bond and a stock index in a two-factor market, and may also buy term life insurance before retirement, with trading restricted to a non-empty closed convex constraint set. The approach builds a family of unconstrained artificial markets, computes dual upper bounds with a reparameterized neural network, and recovers feasible lower bounds via nested Monte Carlo and a Hamiltonian quadratic program. Relative duality gaps under two risk-aversion settings are below the SAMS and deep pricing kernel benchmarks, one of them at 2.87%.

  4. arXiv

    Long-horizon agents should bank credit per turn: MPO splits turn-level advantage from token refinement, lifting search pass@1 by up to 4.33 points

    Long-horizon LLM agents interact with their environment through complete turns but generate each action token by token, and existing reinforcement learning either gives a whole trajectory one shared advantage or treats every token as an action. MPO splits credit assignment into two scales: turn-level generalized advantage estimation carries credit across interactions, a mean-preserving local correction refines it within each turn, and the policy update uses complete-turn importance ratios. Across search, tool-use and automation tasks, MPO reports the highest score on all nine metrics at both Qwen3-4B-Thinking and Qwen3-30B-A3B-Instruct scales, including a 4.33-point BrowseComp-Plus pass@1 gain over the strongest non-MPO baseline at 4B.

  5. arXiv

    HKU-led team releases Compass 1 navigation model: trained on monocular RGB video, it reaches 79.4% open-world success against 17.8% for the strongest baseline

    A group from the University of Hong Kong, Shanghai Innovation Institute, Archon Robotics and other institutions released Compass 1, an embodied navigation foundation model trained on monocular RGB video that adds sparse keyframes and an explicit history map to a unified understanding-and-generation backbone, letting an agent backtrack through nearly two minutes of visual history. Across zero-shot real-world trials in four scenarios (office, terrace, complex outdoor and pedestrian-dense street) and eight tasks, Compass 1 reached 79.4% overall success versus 17.8% for the strongest baseline, AwareVLN; on persistent history backtracking the baseline scored 0% and Compass 1 scored 58%.

  6. arXiv

    Attention now picks its own parameters per token: NESTOR nests expert routing inside column routing and lowers pretraining loss at equal compute

    Mixture-of-experts transformers apply conditional computation only to the MLP, so every token still passes through all attention parameters. NESTOR extends the same routing idea to attention: each token is sent to a few columns, each column has its own attention and experts, and the two levels of sparsity nest hierarchically. In language-model pretraining from 100M to 1B active parameters, the MoE transformer needs 1.13–1.29 times the compute and the dense transformer 3.1–4.1 times the compute to reach the same held-out loss, and this factor shows no downward trend as scale grows.

  7. bioRxiv

    BLOOM turns sparse developmental atlases into a queryable virtual embryo, topping gene and cell correlation on all three mouse atlases

    BLOOM frames transcriptomic virtual-embryo modeling as conditional generation: given an observed window of cell populations, a requested developmental stage and cell-state information, it generates the expression distribution at that stage. Pseudo-trajectories link independently sampled populations, while developmental age, cell-identity hierarchy, anatomical context, gene programmes and transcription-factor–target networks condition both a Gaussian base and a flow-matching generator. Across prenatal, visual-cortex and multi-organ endothelial mouse single-cell atlases, BLOOM ranks first for stage-averaged gene and cell correlations against 22 time-series forecasting baselines.

  8. medRxiv

    Colorectal cancer recurrence after surgery gets a spatial host-microbiome readout: AUC 0.81 versus 0.45 for standard pathology

    Whether colorectal cancer recurs after resection has been judged mainly from routine pathology such as lymphovascular invasion, perineural invasion, tumor budding and Ki-67. AlphaFISH pairs a sequencing-free nanoprobe biochip with a transformer model to read 41 host transcripts and 51 bacterial genera across 149 colorectal biopsies, and in 28 patients followed for one year after surgery it predicted recurrence or metastasis with an AUC of 0.81, against 0.45 for the combined routine clinical markers. The result comes from a preprint that has not been peer reviewed, on a limited sample.

  9. Agent training stops grading only the ending: STEP-VERIFIER scores each step from prefix splits in logged runs

    Tool-using language-model agents often need dozens of interactions before a single success-or-failure label arrives, leaving no way to tell which step went wrong. STEP-VERIFIER turns a success-labeled corpus of logged interactions into action-level comparisons: the difference in average final return between two continuations that share a common prefix labels which of two actions was better, and a Bradley-Terry ranking head converts those pairs into calibrated per-step scores. On Llama-3.1-8B-Instruct and Mistral-7B-Instruct backbones, it beats outcome-only GRPO on the average metric in all six benchmark-backbone combinations across GAIA, ToolBench and AgentBoard, and its per-action scores agree with human judgments on 81.5% of pairs.

  10. Meitei Mayek handwriting recognition stops slicing words into characters: whole-word reading cuts synthetic test word errors to 1.57%

    Handwritten Meitei Mayek has been recognised one character at a time, yet its most frequent confusions depend on where a character sits in a word. Training words are composed from isolated character images and read whole by a convolutional encoder, a bidirectional LSTM and CTC, giving 0.24% character and 1.57% word error on 5,000 synthetic test words; on 100 unseen real words written on a tablet by one native writer, character error is 2.96% and word error 16.0%.