Skip to main content
Search

# Four-level executable validation protocol

1 related articles

Subscribe to updates
  1. arXiv

    HVAC digital twins pass the run test but not the physics test: seven of eight LLMs build and converge, yet behavioral equivalence spans 50.0% to 98.6%

    For HVAC digital-twin topology specifications that large language models write from plain-language descriptions, a four-level executable validation protocol moves the judgment past parsing, building and convergence to whether the converged state matches a hidden reference model. Across eight LLMs and fourteen topologies, seven models built in 98.6% to 100% of trials and converged in at least 80.0%, but behavioral equivalence reached only 50.0% to 98.6%; about half of the failed trials did converge, to a steady state different from the reference, mostly in desiccant rotor systems.

You’re all caught up