Progressive Text-to-3D Generation for Automatic 3D Prototyping

Authors: Han Yi, Zhedong Zheng, Xiangyu Xu, Tat-Seng Chua

Published in ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 2026

Recommended citation: Han Yi, Zhedong Zheng, Xiangyu Xu, Tat-Seng Chua, "Progressive Text-to-3D Generation for Automatic 3D Prototyping." ACM TOMM, 2026. DOI: 10.1145/3797262
Download PDF: https://zdzheng.xyz/files/2026/TOMM_YiHan.pdf

Code is available at: https://github.com/Texaser/MTN

Abstract: Text-to-3D generation is to craft a 3D object according to a natural language description. This can significantly reduce the workload for manually designing 3D models and provide a more natural way of interaction for users. However, this problem remains challenging in recovering the fine-grained details effectively and optimizing a large-size 3D output efficiently. Inspired by the success of progressive learning, we propose a Multi-Scale Triplane Network (MTN) and a new progressive learning strategy. As the name implies, the Multi-Scale Triplane Network consists of four triplanes transitioning from low to high resolution. The low-resolution triplane could serve as an initial shape for the high-resolution ones, easing the optimization difficulty. To further enable the fine-grained details, we also introduce the progressive learning strategy, which explicitly demands the network to shift its focus of attention from simple coarse-grained patterns to difficult fine-grained patterns. Our experiment verifies that the proposed method performs favorably against existing methods. For even the most challenging descriptions, where most existing methods struggle to produce a viable shape, our proposed method consistently delivers. We aspire for our work to pave the way for automatic 3D prototyping via natural language descriptions.

@article{yi2026progressive,
author = "Yi, Han and Zheng, Zhedong and Xu, Xiangyu and Chua, Tat-Seng",
title = "Progressive Text-to-3D Generation for Automatic 3D Prototyping",
abstract = "Text-to-3D generation is to craft a 3D object according to a natural language description. This can significantly reduce the workload for manually designing 3D models and provide a more natural way of interaction for users. However, this problem remains challenging in recovering the fine-grained details effectively and optimizing a large-size 3D output efficiently. Inspired by the success of progressive learning, we propose a Multi-Scale Triplane Network (MTN) and a new progressive learning strategy. As the name implies, the Multi-Scale Triplane Network consists of four triplanes transitioning from low to high resolution. The low-resolution triplane could serve as an initial shape for the high-resolution ones, easing the optimization difficulty. To further enable the fine-grained details, we also introduce the progressive learning strategy, which explicitly demands the network to shift its focus of attention from simple coarse-grained patterns to difficult fine-grained patterns. Our experiment verifies that the proposed method performs favorably against existing methods. For even the most challenging descriptions, where most existing methods struggle to produce a viable shape, our proposed method consistently delivers. We aspire for our work to pave the way for automatic 3D prototyping via natural language descriptions.",
journal = "ACM TOMM",
video = "https://www.youtube.com/watch?v=LH6-wKg30FQ",
code = "https://github.com/Texaser/MTN",
url = "https://zdzheng.xyz/files/2026/TOMM\_YiHan.pdf",
doi = "10.1145/3797262",
funding = "2025A1515012281, 202401035, FDCT/0043/2025/RIA1",
year = "2026" }