LLaMA3-8B cut to 4.5B parameters: SlimFM moves LLM semantics into online CTR serving at about 16 ms
Synopsis
Adding large-language-model semantics to click-through-rate prediction previously meant caching LLM outputs offline and serving only a small CTR model online. SlimFM prunes LLaMA3-8B to a 4.5B-parameter backbone with dependency-aware structured pruning, fine-tunes it with LoRA for field-level semantic embeddings, and injects those embeddings into a conventional CTR model's feature interactions. Across four datasets and six CTR backbones it retains over 99% of the uncompressed model's accuracy, with single-sample inference latency of 15.98–17.58 ms.
Interpretation
Large-language-model semantics can now be inferred online alongside the click-through-rate model instead of being precomputed and cached. Earlier LLM-enhanced CTR methods precomputed and cached LLM outputs offline and served only the CTR model online, so cached outputs had to be regenerated whenever the item or feature catalog changed. On four public CTR datasets and six CTR backbones, SlimFM's single-sample inference latency is 15.98–17.58 ms, versus 54.62–55.08 ms for KAR and 46.46–47.53 ms for LLM-CF, and the gap holds across all six backbones.
The compressed model still produces discriminative field-level semantic embeddings for the CTR model's feature interactions. Pruning groups structurally coupled parameters and estimates each group's importance from a Taylor expansion of the next-token prediction loss, removing groups until a target sparsity is reached. On the FM backbone, the 8B model reaches a mean AUC of 0.7346 and the 4.5B model reaches 0.7329, a recovery rate of 99.77%; pruning to 6B gives 99.88% and pruning to 2B drops to 99.62%.
Field-level semantics are trained with contrastive learning and reweight the CTR model's pairwise feature interactions through learnable weights. Fine-tuning updates only the LoRA adapters on the attention projection matrices, while the pruned backbone stays frozen. Training builds prompts for a field-identification task that assigns feature values to candidate field names and optimizes an InfoNCE-style contrastive loss; at inference the pruned backbone is deployed directly without pre-caching.
Perspective
Evaluation covers four public CTR datasets (Amazon Gift Cards, Amazon Video Games, Amazon Digital Music, MovieLens-1M) and six CTR backbones, with only one LLM family, LLaMA3-8B, on the LLM side; latency is single-sample inference averaged over 50 runs with 10 warmup runs.
Pruning to 2B lowers the recovery rate further to 99.62%, with the decline most visible on Gift Cards and Video Games; generalization to other backbones, domains and LLM architectures remains to be verified.
