Andrii Bidochko

• Updated: June 14, 2026
• 7 min read

MUSE: Benchmarking Manufacturable, Functional, and Assemblable Text-to-CAD Generation

Direct Answer

The paper introduces MUSE (Manufacturable, Functional, and Assemblable Text‑to‑CAD Benchmark), a comprehensive evaluation suite that shifts the focus of text‑driven CAD generation from isolated geometric shapes to full‑blown, editable B‑Rep assemblies that can be manufactured, function as intended, and be assembled into larger products. It matters because it provides the first systematic way to measure whether AI‑generated designs meet real‑world engineering criteria, closing the gap between academic prototypes and industrial‑grade design automation.

Background: Why This Problem Is Hard

Designing a mechanical part or an assembly is a multi‑step reasoning problem. Engineers must translate vague, often high‑level intent (e.g., “a lightweight bracket that snaps onto a motor housing”) into a precise boundary representation (B‑Rep) that satisfies functional loads, respects manufacturing constraints, and aligns with other components. Existing text‑to‑CAD research has largely treated this as a pure geometry synthesis task, rewarding models for producing shapes that look similar to a reference model. Those metrics ignore three critical dimensions of engineering design:

• Manufacturability: Can the part be fabricated with standard processes (e.g., CNC milling, injection molding) without violating tolerances?
• Functionality: Does the geometry actually perform the intended mechanical function (e.g., bearing a load, guiding motion)?
• Assemblability: Can the part be joined with other components using realistic fasteners, clearances, and alignment features?

Because most benchmarks rely on surface‑level similarity scores (Chamfer distance, IoU, etc.), they fail to penalize designs that are mathematically correct but practically useless. Moreover, current datasets consist of single‑part models, which do not capture the combinatorial complexity of real assemblies where part‑to‑part relationships, hierarchy, and constraints dominate the design space. This mismatch has stalled progress toward AI‑driven product design pipelines that can be deployed in factories or CAD‑centric enterprises.

What the Researchers Propose

The authors propose a three‑tiered benchmark framework—MUSE—that pairs each design instance with a structured Design Specification (DS) and evaluates generated CAD models through a staged protocol:

1. Code Check: Verifies that the LLM‑produced script is syntactically valid, imports the correct libraries, and can be executed without runtime errors.
2. Geometric Check: Confirms that the resulting B‑Rep is watertight, manifold, and respects basic CAD rules (e.g., no self‑intersections).
3. Design‑Intent Alignment: Uses rubric‑based visual language model (VLM) judges, calibrated against human annotations, to assess whether the model satisfies functional, manufacturable, and assemblable criteria defined in the DS.

Key components of the MUSE ecosystem include:

• Design Specification Language: A JSON‑like schema that encodes functional requirements (load, motion), manufacturing constraints (material, process), and assembly relationships (clearance, fastener type).
• Dataset of Real‑World Assemblies: 1,200 multi‑part B‑Rep models drawn from open‑source mechanical libraries, each annotated with a DS and a set of engineering rubrics.
• Rubric‑Based VLM Judge: A vision‑language model fine‑tuned to score generated designs against the rubrics, providing a scalable proxy for expert human review.

How It Works in Practice

When an AI system receives a textual prompt—say, “design a modular drone landing gear that can be 3‑D printed in PLA and snaps together without screws”—the MUSE workflow proceeds as follows:

1. Prompt Parsing: The LLM extracts key entities (components, material, assembly method) and translates them into a Design Specification.
2. Code Generation: Using the DS, the model emits a script (typically in Python with the cadquery or OpenCascade API) that constructs each part as a B‑Rep.
3. Execution & Validation: The script runs in a sandboxed CAD engine. The code check stage catches syntax errors, missing imports, or illegal API calls.
4. Geometry Sanitization: The resulting B‑Rep is passed through a geometric validator that checks for manifoldness, edge continuity, and volume integrity.
5. Rubric Scoring: Rendered images of the assembly are fed to the VLM judge, which evaluates each rubric (e.g., “clearance between gear teeth & housing ≤ 0.2 mm”). Scores are aggregated into three high‑level buckets: Functionality, Manufacturability, Assemblability.
6. Feedback Loop: If any bucket falls below a predefined threshold, the system can request a regeneration or apply corrective heuristics (e.g., increase wall thickness, add fillets).

This pipeline differs from prior work in two fundamental ways. First, it treats the CAD script as a first‑class artifact, enforcing executable correctness before any visual assessment. Second, it embeds engineering intent directly into the evaluation loop, turning abstract “shape similarity” into concrete, rubric‑driven quality signals that align with how human designers judge a product.

Evaluation & Results

The authors benchmarked four closed‑source LLMs (including GPT‑4‑Turbo) and three open‑source models (Llama‑2‑70B, Mistral‑7B, and a fine‑tuned CodeLlama) on the MUSE suite. Evaluation covered three dimensions:

• Executable Success Rate: Percentage of generated scripts that passed the code check.
• Geometric Validity Rate: Proportion of executably generated models that produced watertight B‑Reps.
• Engineering Alignment Score: Weighted average of rubric scores across functionality, manufacturability, and assemblability.

Key findings include:

• Even the strongest closed‑source model achieved only a 68% executable success rate, indicating a “failure cascade” where code errors propagate downstream.
• Geometric validity dropped to 54% for the best model, revealing that many syntactically correct scripts still generate non‑manifold geometry.
• Engineering alignment scores were modest across the board: the top model scored 42/100 on functionality, 35/100 on manufacturability, and 28/100 on assemblability. Open‑source models lagged further behind, often failing to meet any rubric threshold.
• Human annotators agreed with the VLM judge on 87% of rubric judgments, confirming the reliability of the automated scoring pipeline.

These results demonstrate that current text‑to‑CAD systems are still far from producing designs that are ready for real‑world engineering workflows. The benchmark surfaces a clear hierarchy of challenges—first ensuring code runs, then guaranteeing clean geometry, and finally satisfying nuanced engineering constraints.

Why This Matters for AI Systems and Agents

For AI practitioners building autonomous design agents, MUSE offers a realistic yardstick that aligns model objectives with production‑grade outcomes. Instead of optimizing for abstract similarity metrics, developers can now train or fine‑tune models against rubric‑based feedback, effectively teaching agents to “think like an engineer.” This shift has several practical implications:

• Agent Orchestration: Designers can embed MUSE’s three‑stage validation as a micro‑service within a larger workflow, allowing agents to request re‑generation only when a specific rubric fails, thereby reducing wasted compute.
• Evaluation Automation: The VLM judge provides a scalable alternative to costly human review, enabling continuous integration pipelines for AI‑generated CAD assets.
• Productization Path: Companies can leverage MUSE scores as a quality gate before feeding designs into downstream CAM (Computer‑Aided Manufacturing) or PLM (Product Lifecycle Management) systems.

Integrating MUSE into an AI‑driven design stack can therefore accelerate time‑to‑market for custom hardware, support rapid prototyping in startups, and improve the reliability of generative design tools used by large manufacturers.

Explore how you can embed AI‑generated design workflows into your existing infrastructure with the ChatGPT and Telegram integration, or streamline data persistence using the Chroma DB integration. For a broader view of AI‑enabled automation, see the UBOS platform overview.

What Comes Next

While MUSE marks a significant step forward, the authors acknowledge several limitations that open fertile research avenues:

• Domain Coverage: Current datasets focus on mechanical assemblies; extending to electronics, fluid systems, or soft robotics would test the benchmark’s generality.
• Dynamic Functionality: Rubrics presently assess static properties (clearance, thickness). Future work could incorporate simulation‑based criteria such as stress analysis or motion planning.
• Human‑in‑the‑Loop Refinement: Combining VLM scores with active learning from expert designers could accelerate model improvement.
• Cross‑Modal Prompting: Integrating sketches, point clouds, or voice commands alongside text could make the design specification richer and more intuitive.

Addressing these challenges will bring AI‑generated CAD closer to the “design‑to‑manufacture” paradigm that industry demands. Researchers interested in building next‑generation generative design agents may find the Enterprise AI platform by UBOS a useful sandbox for deploying large‑scale model inference and evaluation pipelines. Startups looking to prototype AI‑driven product configurators can also benefit from the UBOS for startups offering.

References

• MUSE: Benchmarking Manufacturable, Functional, and Assemblable Text-to-CAD Generation (arXiv)

Andrii Bidochko

CTO UBOS

Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.