Skip to main content
Back to timeline
arXivSource publication:

CAD generators get the volume nearly right, but DrawScore catches the small manufacturing errors on drawing curves

Synopsis

Automatic CAD generation is usually scored with volumetric IoU, Chamfer Distance and surface F-score, which barely penalize local errors occupying a small fraction of a part's volume or surface, such as a misplaced hole or a slot on the wrong face. DrawScore projects the candidate and the ground truth into three orthographic views with visible and hidden lines and applies a saturating penalty along the projected curves, so a local error's weight decays only linearly with its size. Among 1,188 outputs from six agent configurations, 190 of the 1,007 with IoU at least 0.95 (18.9%) score below 0.95 on DrawScore, while only 10 of the 827 with DrawScore at least 0.95 (1.2%) fall below 0.95 on IoU.

Source-provided article image: DrawScore: Evaluating Generated CAD Models against Engineering Drawings
arXiv

Interpretation

Geometric similarity scoring for generated CAD models changes what is compared: not solid volume or surface, but projected curves on engineering drawings. Volumetric IoU, Chamfer Distance, surface F-score and Edge Chamfer Distance measure differences over volume, surface or sharp edges, responding strongly to wrong overall size or shape but giving little weight to a missing or misplaced small hole or slot. On 1,188 valid outputs from six agent configurations, DrawScore ranges from 0.4218 to 1.0000 among the 1,007 outputs with IoU at least 0.95, and 190 of them (18.9%) score below 0.95; among the 827 outputs with DrawScore at least 0.95, only 10 (1.2%) have IoU below 0.95.

The method projects both models into three orthographic views that draw visible and hidden lines, then compares whether the two sets of curves coincide. DrawScore uses the CAD kernel's hidden-line removal to project along three orthogonal directions, measures each ground-truth curve point's distance to candidate curves of the same visibility, and applies a Gaussian saturating penalty that approaches 1 once the deviation exceeds the positional tolerance. Each view's recall and precision are computed by arc length and combined as a harmonic mean, and the final score is the lowest across the three views; on 53 controlled comparisons built from 35 ground-truth models, the fitted decay orders of all five metrics differ from the integration-dimension prediction by less than 0.2.

The same score serves as a training reward and an inference-time candidate selector without 3D ground truth. Training Qwen3.5-4B with DrawScore as a reinforcement learning reward raises the share of valid outputs from 75.2% to 89.5% and improves the other three metrics over the SFT model, whereas the Edge Chamfer Distance reward raises validity but lowers DrawScore below SFT. On 20 valid Public T1 cases from BenchCAD, Qwen3.8-max with the codex framework generates 8 candidates per case and DrawScore selects the best against the input drawing; relative to the single-generation baseline, its gains on IoU, CD and surface F-score reach 4.7%, 17.2% and 13.0% of the Oracle's gains respectively.

Perspective

The demonstrated scope is comparison of projected curves across three orthographic views with visible and hidden lines, with controlled experiments on 53 comparisons built from 35 ground-truth models, agent-output experiments on 1,188 valid outputs from six configurations, and training and selection experiments on Ortho2CAD, Fusion360 and BenchCAD Public T1 cases respectively.

Three orthographic views do not uniquely determine geometry for a few parts, where stepped bores and internal cavities appear only as overlapping hidden lines; inference-time selection assumes the input drawing is already vectorized, and raster drawings would need vectorization first; DrawScore takes longer to compute than other metrics because it runs hidden-line removal on each model along multiple viewing directions.