Skip to main content
Search

# Biomedical Information Extraction

2 related articles

Subscribe to updates
  1. arXiv

    OncoNoteBERT trains oncology-specific encoders on 290,000 UK outpatient notes: continued pretraining cuts perplexity to 2.10, while a from-scratch tokenizer shortens sequences by about 23.6%

    Using 290,026 outpatient oncology notes from 21,564 patients treated for lung and head-and-neck cancer at a UK specialist oncology centre, the study compared RadBERT and PathologyBERT with two local strategies: OncoNote-RadBERT, produced by continued masked language model pretraining, reached a validation perplexity of 2.10, while OncoNoteBERT, trained from scratch with an oncology-specific WordPiece tokenizer, reached 2.83 but the most efficient tokenisation (subword fertility 1.224, normalised sequence length 0.7636, about 23.6% fewer tokens) and returned clinically acceptable predictions for 12 of 13 masked-token probes versus 7 of 13 for OncoNote-RadBERT; both local models represented the institutional placeholder [REMOVED] as a single learnable token.

  2. medRxiv

    An LLM-Enabled Pipeline for Natural History Study Information Extraction in Rare Disease Research

    This study built a proof-of-concept information extraction pipeline using three open-source LLMs (Athena-v3-AWQ, Gemma3-27B, and Llama-3.1-70B-Instruct) to extract 11 natural history study characteristics from PubMed abstracts labeled "2" in the CZI DRSM corpus (148 gold-standard and 3,547 full-corpus abstracts), finding that all three models exceeded 99% processing success, that Gemma was best overall on expert rating (68.0% of outputs rated "good") and full-corpus runtime (~16 minutes), that Llama scored higher on automated Token F1 (0.874 vs. 0.723), and that Athena performed worst largely because it copied source text verbatim rather than synthesizing it.

You’re all caught up