Skip to main content
Search

Connor Makowski

1 related articles

Subscribe to updates
  1. arXiv

    Functionalizer splits casing, diacritics and repetition into reversible opcodes, cutting vocabulary needs by up to 19.7% under corpus exhaustion and lifting a 98M-parameter GPT-2's Python syntax validity from 7.70% to 9.12%

    The work presents the Functionalizer, a lossless pre-tokenizer framework that factors orthographic variation such as casing, diacritics and character repetition into a compositional opcode/operand prefix stream encoded in the Unicode Private Use Area, achieving complete corpus coverage with up to 19.7% fewer required vocabulary slots and, on 98M-parameter GPT-2 models, raising Python syntax validity from 7.70% to 9.12% while reducing duplicate n-gram repetition on FineWeb-Edu prose from 66.0% to 55.8%.

You’re all caught up