Skip to main content
Back to timeline
arXivSource publication:

Prism drops multi-scale fusion and heavy heads: paragraph PQ on HierText rises from 55.2 to 66.8

Synopsis

Hierarchical text detection must locate words, lines and paragraphs while recovering how they nest, and prior systems bought accuracy with multi-scale feature fusion, heavy detection heads and learnable relation modules, at a cost in inference efficiency. Prism instead attaches parallel lightweight heads to a single-scale backbone feature map to predict words, lines and paragraphs separately, then rebuilds the hierarchy bottom-up from geometric inclusion of minimum bounding boxes. On the HierText test set it reaches word, line and paragraph PQ of 75.8, 71.1 and 66.8, respectively 14.2, 6.6 and 11.6 points above OmniParser V2, and runs at 65.1 FPS on CTW1500, faster than LRANet.

Source-provided article image: Prism: Hierarchical Text Detection with Minimalist Design
arXiv

Interpretation

Hierarchical text detection can drop multi-scale fusion necks and heavy detection heads, keeping only a sufficiently pre-trained backbone plus parallel lightweight heads that emit words, lines and paragraphs together. Earlier systems followed an architecture-stacking route: feature-pyramid-style multi-scale necks for scale variation, high-resolution or parallel heavy heads for pixel-level prediction, and extra relation-reasoning modules to parse word-to-line and line-to-paragraph inclusion. On the HierText test set, Prism reaches word, line and paragraph PQ of 75.8, 71.1 and 66.8, where OmniParser V2 in the same table reports 61.6, 64.5 and 55.2.

All three granularities read the same single-scale feature map from the backbone, each handled by its own lightweight head, while the hierarchy comes from geometry rather than a learned module. The design takes the backbone's 16x downsampled feature map as the sole input, splits word, line and paragraph detection into parallel independent heads, and assigns parents bottom-up from word to line and line to paragraph using set intersection over minimum bounding boxes. On HierText this box-level parsing takes 1.2 ms against 32.4 ms for exact polygon intersection, while line PQ falls only from 71.3 to 71.1.

Among four candidate lightweight head archetypes, pixel-level regression gives the best accuracy-efficiency trade-off and becomes the default. The alternatives are query-level regression, pixel-level segmentation and query-level segmentation, matching the DETR-style, DB-style and MaskFormer-style detection paradigms. On HierText the pixel-level regression head reaches word, line and paragraph PQ of 75.8, 71.1 and 66.8 with 16.1 ms head latency, while the query-level segmentation head reaches 65.7, 60.5 and 55.4 at 27.2 ms.

Perspective

The demonstrated scope is hierarchical and single-granularity scene text detection on public benchmarks: HierText for word, line and paragraph evaluation, and CTW1500, Total-Text, MSRA-TD500 and ArT for single-granularity evaluation, with inference on a single NVIDIA RTX 3090. The default configuration is a ResNet50-DCN backbone truncated at the C4 stage with a pixel-level regression head, and a short side of 2048 on HierText. The authors propose the minimalist design as a plug-and-play detection extension for large vision-language models, which is an outlook rather than a verified result.

The minimalist design holds only with sufficient pre-training: trained from scratch, Prism trails LRANet in F-measure, the gap shrinks as pre-training data grows, and it overtakes LRANet under end-to-end joint training; on ViT backbones it clearly trails with ViT-T and only matches or slightly exceeds LRANet with ViT-B. Backbone scale shows a similar threshold, trailing on ResNet18 and ResNet50, matching LRANet on ResNet50-DCN, with negligible gains from ResNet101-DCN. Code is not yet released.