Skip to main content
Back to timeline
arXivSource publication:

RouTex turns rotary position encoding into a reference router, filling in unseen 3D surfaces and reaching FID 12.06

Synopsis

A single reference image shows only one side of an object, so back-side texture must be guessed from priors and often invents details. RouTex rewrites 3D rotary position encoding inside a 2D multi-view diffusion framework into explicit reference-to-view correspondence without adding parameters, and pairs it with dual-granularity attention for local texture and cross-view consistency. On a 475-asset evaluation set conditioned on four orthographic views it reaches FID 12.06, below TRELLIS at 18.83 and TRELLIS.2 at 12.93.

Source-provided article image: RouTex: Multi-reference 3D Texture Generation by Reprogramming Rotary Position Encoding
arXiv

Interpretation

Multi-reference conditioning is wired directly into a 2D multi-view diffusion framework, with the reference-to-view correspondence carried by position encoding rather than an extra 3D structural prior. Earlier multi-reference work reused TRELLIS's cyclic injection, applying one reference per denoising step, which recovers only coarse base textures and badly distorts fine details such as text and logos; transplanting cyclic injection into a multi-view diffusion framework fails because that framework lacks 3D structural priors to auto-align mesh surfaces with references through cross-attention. On 475 high-quality 3D assets disjoint from training, with every method texturing the same meshes reconstructed by Hunyuan3D-2.1 and decimated to 50K faces, RouTex conditioned on four orthogonal views reaches FID 12.06, KID 8.66, CLIP-I 0.9423 and LPIPS 0.1519, against TRELLIS at 18.83, 28.49, 0.9149 and 0.1727 and TRELLIS.2 at 12.93, 4.53, 0.9246 and 0.1656.

The reference-to-view correspondence is decided entirely by position encoding across discrete azimuth regions, so only coarse azimuth categories are needed rather than precise camera calibration. Positional Routing splits camera yaw into four canonical azimuth regions (front, right, back, left), aligns each reference's spatial coordinates with the first target view in the same region, and places the region-0 reference at the canvas center to broadcast front-view appearance to all views. The semantic axis uses fractional offsets to distinguish views and reference regions; double-stream blocks isolate each target view with its matched reference into an independent sequence for local attention, and single-stream blocks run global joint attention over deduplicated references; during training the reference count is sampled from 1 to 4 with at least one reference required from region 0.

Perspective

Evaluation is limited to 475 3D assets disjoint from the training data, with all methods texturing the same meshes reconstructed by Hunyuan3D-2.1 and decimated to 50K faces, rendered under uniform ambient lighting; RouTex and the multi-reference baselines use front, back, left and right condition images, while single-reference baselines use only the front view.

Ablation shows that removing dual-granularity attention lowers quantitative metrics only slightly, yet the component is still described as preventing localized texture artifacts and enabling a speedup; removing Positional Routing, or both components, produces a clearer performance drop.