New software tool lets users repair AI-generated 3D models

A guiding principle for many software engineers—"What you see is what you get" —means creating programs where the content you're editing looks the same as the final product. But when you're using generative AI (genAI) systems to 3D-print, say, a mug, you'll likely get a cup that can't hold your coffee. September 9, 2026 New software tool lets users repair AI-generated 3D models, then fabricate them just the way they want by Alex Shipps, Massachusetts Institute of Technology edited by Lisa Lock, reviewed by Robert Egan Lisa Lock Scientific Editor Meet our editorial team Behind our editorial process Robert Egan Senior Editor Meet our editorial team Behind our editorial process Editors' notes This article has been reviewed according to Science X's editorial process and policies.

Editors have highlighted the following attributes while ensuring the content's credibility: fact-checked preprint trusted source proofread The GIST Add as preferred source InstructMesh enables fabrication-relevant refinement of generative 3D models through selective editing. A) A user provides an initial prompt. B) The system generates a 3D model, but may include fabrication-related flaws (e.g., a sealed lid).

C) The user selects the problematic region and provides a descriptive edit prompt. D) InstructMesh applies an edit operation in the latent-space to remove the lid while preserving other regions and regenerates the model. E) The final model is fabricated as a functional mug.

The issue is that AI models understand how an object should look, but not how it works, leading to impractical designs that undermine an item's intended use. Even if you want to fix these errors, the models are typically hard to edit, especially for users new to 3D design. A new approach called "InstructMesh" makes it much easier to design and print household items, accessories and robots that work in the real world.

The design software, which was developed by researchers at MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL), Google and Northeastern University, can be prompted to generate a 3D design for a pair of glasses, for instance, and users can then highlight specific parts of the blueprint they'd like refined before 3D printing. It's an AI-driven interface designed to understand how these designs should look and which edits experts and novices alike want to make, helping them create the objects they actually want to see.

The research will be presented at the ACM Symposium on User Interface Software and Technology (UIST) in November and is available as a paper posted to the arXiv preprint server. CSAIL researchers used InstructMesh to put a personalized, creative spin on otherwise regular items.

For example, the tool produced a mug that appears to be enveloped by a dragon, with its tail serving as the handle. It also fabricated a shiny blue whistle resembling a shell and a pair of glasses with butterfly wings spreading out just above the lenses.

Getting even more creative, it made an octopus-like dispenser, with liquids flowing out of each tentacle to distribute drinks into several cups at once. So what makes InstructMesh so adept at following such unique prompts?

It pairs Microsoft's TRELLIS system, which creates 3D models from text and image prompts, with the large language model (LLM) GPT-4, which supports ChatGPT. In other words, visual and textual knowledge combined.

"We wanted to bring together the talents of 3D generators and the reasoning skills of LLMs in an interactive space to make objects that people actually want," says MIT graduate Faraz Faruqi, Ph.D. '26, lead author on a paper presenting the project and CSAIL alumnus.

"Language models are great at text and images, while TRELLIS' talent lies in its ability to create 3D models, since it's seen so many." InstructMesh's strengths come in handy in other, more surprising areas. MIT scientists used the program to fabricate a knee brace that looks like denim to match a patient's jeans.

InstructMesh can even help create robots—that is, clever enclosures that house wireless components. The researchers made a "bristle bot" that resembles a colorful shrimp to demonstrate this.

It has a motor hidden inside, and when switched on, it can slide across surfaces, sort of like a wind-up toy. No problem Faruqi and his colleagues found that InstructMesh could easily make their desired items.