Skip to main content
Search

# SynMOE

1 related articles

Subscribe to updates
  1. arXiv

    SynMOE routes syntax labels into expert gating: LoRA and IA3 gain on code tasks while training about 1.90% of parameters

    Parameter-efficient fine-tuning usually trails full fine-tuning on code tasks, partly because it relies on low-rank or sparsity assumptions without explicitly modeling code structure. SynMOE first extracts a syntax label for each leaf node of the abstract syntax tree, aligns those labels to the code language model's input tokens, and lets the labels take part in the routing decision of a mixture-of-experts gate. Across code search, code clone detection and line-level code completion with five code models, SynMOE improves both LoRA and IA3, with a maximum gain of 8.78%, and matches or exceeds full fine-tuning while training about 1.90% and 0.41% of parameters.

You’re all caught up