Frozen Utonia reads building function from facade point clouds at 0.732 mAP, while networks trained from scratch fail to beat hand-crafted descriptors
Synopsis
Semantic labels say what a building element is, not what it does: two similar-looking windows on one facade may be operable or fixed. Across 5,172 elements in 22 mobile-LiDAR buildings, each element carries 15 functional attributes, and whole buildings are held out to compare hand-crafted descriptors, networks trained from scratch and frozen self-supervised point transformers, all read by the same linear layer. Frozen Utonia reaches 0.732 mAP on functionally ambiguous elements against 0.610 for the descriptors, and a random-weight control of the same architecture scores 0.652, which the authors use to attribute the gain to pretraining.
Interpretation
The function of facade elements in building point clouds can be read directly from frozen self-supervised point representations with a single linear layer, without training a network from scratch on a handful of labelled buildings. Existing urban and facade point-cloud benchmarks mainly annotate semantic classes for segmentation and level-of-detail reconstruction; to the authors' knowledge no network is designed specifically for instance-level functional attribute prediction from 3D facade point clouds, and whether function is encoded in point representations was untested. On the HFX3D benchmark of 22 mobile-LiDAR buildings, 5,172 elements and 15 functional attributes per element, five building-disjoint folds hold out 3 to 5 buildings and train on the remaining 18, with 10 seeds per configuration; on the 2,472 functionally ambiguous elements of seven classes, the frozen Utonia linear probe scores 0.732 mAP, hand-crafted descriptors 0.610, and a class-only one-hot encoding 0.556, below the 0.569 chance level.
The gain comes from pretraining itself rather than from the capacity of the architecture or its spatial context. PointNet, PointNet++, DGCNN and the 108M-parameter PTv3 trained from scratch land between 0.548 and 0.628 mAP, including a purpose-built model fusing shape, descriptors, class and a neighbour graph, and none reliably exceeds the 21 hand-crafted descriptors. Freezing each pretrained architecture at its random initialisation gives 0.649 for PTv3 and 0.652 for Utonia; Utonia improves by 0.080 over its own random-weight control, the only interval excluding zero, while its lead over Sonata at 0.716 is within uncertainty. Frozen Sonata with a per-point head exceeds PTv3 trained from scratch by 0.130.
The functional signal keeps improving as more labelled buildings are added, and fine-tuning erodes the benefit. Hand-crafted descriptors and random-weight features saturate after a few labelled buildings; in the fine-tuning configurations tested, end-to-end fine-tuned Sonata drops to 0.559, and a 50-fold lower backbone learning rate gives 0.658, still below the frozen backbone at 0.678. With only two labelled buildings, Utonia already reaches 0.646, above the 0.610 of descriptors trained on all 18; adding the 21 descriptors to frozen features changes mAP by at most 0.003, which is not measurable.
Perspective
The conclusion rests on 22 buildings from one city in the HFX3D benchmark, 5,172 elements in total, with evaluation limited to 2,472 functionally ambiguous elements of seven classes under five building-disjoint folds and a linear probe read-out.
The authors note that whether outdoor pretraining data helps further needs more buildings to tell; the gains of Sonata and Concerto over their random-weight controls are not significant at the building level; a linear probe gives only a lower bound on what the representations encode, and a stronger read-out may extract more; the frozen backbones see each element with its surroundings while networks trained from scratch see its points alone, so the two comparisons differ in spatial context and training setup.
