I wanted to see how well it performed on real pictures of parts or hand-drawn drawings, but when I tried setting up the docker image, immediately ran into all kinds of dependencies not being installed. The examples make me suspect it doesn't work well beyond images that were generated from CAD in the first place.
This has been easy with OpenSCAD for a long time. I have made lots of cool, complex models this way. I built a repo of the prompts I use to show the llm how to do this and it includes many of the models I've created this way...
If you want something based on B-Rep, look at projects that use opencascade under the hood, as that is one of the only B-Rep CAD kernels available which is free and open source. Some examples would be CADQuery, CascadeStudio, or RepliCAD.
Neat, but I don't really see the utility. The time consuming part of CAD drawing comes from figuring out the correct dimensions of each feature, spacing, sizing, tolerances, etc., and constraining the drawing in a way so that it's easy to tweak later on- which this doesn't do at all. Maybe you could draw a 2d sketch of what you want then generate it, but you'd still have to do the hard part.
To the author if they happen to see this. Please kill the auto playing video. If someone is listening to something else on their phone this always takes over and interrupts.
Maybe I missed something, if you have the image rendering in the first place, you already (likely) have the CAD. It is a nice demo, but what is the utility?
(forgot to mention, it's wired up to Claude so you can vibe CAD, like OP but with a few more steps - I'd like to train a similar model soon! I also wrote about my first stab at this https://campedersen.com/cad0)
Ideally it would tie in with an llm, no? Like you would want to be able to say something like "create a design of car suspension subject to x,y,z contrains"
So, at this point, it seems like this will work with all CAD programs, since they have yet to encounter any systems that they can't work with. More seriously, my guess would be whatever one is available for free in their lab. Kind of standard operating procedure for academic projects -- do a proof of concept, make a video that avoids known bugs, get a grade, push source to git, graduate. Good ideas come out of that... production code... eh... maybe.
More likely someone ends up in the situation that my kid did, previous graduate student's git repo is stale by 2 versions of C++, and 4 versions of ROS, and neither of the two unit tests still work after porting.
Doesn't matter. CAD models/objects are represented by a sequence of operations on a primitive or sketch. Unlike meshes, that describe the manifested resulting shape of objects in 3D programs like Blender.
So it's about the fact, that their model outputs that hierarchy of operations. The history of development, not just the result.
How does it not matter? Every CAD program is not going to have exactly the same interface and commands. I doubt for example this will for example generate and OpenSCAD text file.
Code to compute fillets and blends gets incredibly complex when multiple surfaces are involved. And when surfaces are barely intersecting, or almost coincident, all bets are off what the command will do - very much depends on the geometry kernel and the tolerances it uses whether it decides the surfaces even intersect. And if it decides they don't intersect, all downstream commands will fail. Handling tolerances is one of the hardest aspects of CAD. (It's no coincidence that most open source CAD applications always demo with the same relatively basic types of models - they just can't do truly complex CAD.)
So a simple set of operations - cube, sphere, intersect - sure that will work anywhere and will be portable across applications and makes a nice simple demo. But once you start doing any serious CAD modeling the result is kernel dependent. That's why portable CAD formats like STEP do not preserve the commands used to generate the results. And why native CAD application formats do preserve the command history but are not portable across applications.
It could be anything which is why the question was asked what it actually outputs. I had a skim through the page and code but couldn't see what the output was.
Is this Google-affiliated? The heading font is Product/Google Sans which IIRC only Alphabet is allowed to use and the entire webpage seems to be Google-style but neither of the two named researchers seem to be employed by Google?
I wanted to see how well it performed on real pictures of parts or hand-drawn drawings, but when I tried setting up the docker image, immediately ran into all kinds of dependencies not being installed. The examples make me suspect it doesn't work well beyond images that were generated from CAD in the first place.
> docker image, immediately ran into all kinds of dependencies not being installed
Ironically the former is engineered to avoid the latter.
If only there was some kind of container that allowed you to bundle all your dependencies together with your software.
This has been easy with OpenSCAD for a long time. I have made lots of cool, complex models this way. I built a repo of the prompts I use to show the llm how to do this and it includes many of the models I've created this way...
https://github.com/cjtrowbridge/vibe-modeling
OpenSCAD has almost zero crossover with B-rep modelling ('true' CAD, what this apparently is), though.
OpenSCAD uses CSG which is generally better. Easy to convert CSG to BREP. Cant generally do the opposite
It's easy to convert because CSG is a small subset of what BREP can do.
It's analogous to "all squares are rectangles, but not all rectangles are squares" (squares=CSG, rectangles=BREP)
CSG by itself isn't suitable for most CAD use-cases.
How can I convert openSCAD models to true STEP files? As in no meshes. I Literally have wanted that for years.
how hard it is ? with AI prevalent, how long ? any pointers to start from ?
If you want something based on B-Rep, look at projects that use opencascade under the hood, as that is one of the only B-Rep CAD kernels available which is free and open source. Some examples would be CADQuery, CascadeStudio, or RepliCAD.
What is the inference overhead on this
Same. Working with an LLM and OpenSCAD has been totally painless.
I’ve been using cadquery and build123 with Claude code and I find it incredibly painful.
What is your workflow for llm integration to openscad?
Neat, but I don't really see the utility. The time consuming part of CAD drawing comes from figuring out the correct dimensions of each feature, spacing, sizing, tolerances, etc., and constraining the drawing in a way so that it's easy to tweak later on- which this doesn't do at all. Maybe you could draw a 2d sketch of what you want then generate it, but you'd still have to do the hard part.
To the author if they happen to see this. Please kill the auto playing video. If someone is listening to something else on their phone this always takes over and interrupts.
TIL: people are still browsing the internet without an ad blocker.
Maybe I missed something, if you have the image rendering in the first place, you already (likely) have the CAD. It is a nice demo, but what is the utility?
Readers may also enjoy my open source Rust BRep CAD kernel https://github.com/ecto/vcad or the hosted version at https://vcad.io.
I also wrote a bit about what goes into CAD apps! https://campedersen.com/tessellation
(forgot to mention, it's wired up to Claude so you can vibe CAD, like OP but with a few more steps - I'd like to train a similar model soon! I also wrote about my first stab at this https://campedersen.com/cad0)
The examples they show are so basic.
Checked out GenCAD. It seems pretty useful for simple circuit designs. Wondering if it supports import/export with other CAD formats?
Ideally it would tie in with an llm, no? Like you would want to be able to say something like "create a design of car suspension subject to x,y,z contrains"
The input is images, and the output is CAD models, so it appears you could use a multi-modal LLM to natural language -> image -> CAD
The demo seems pretty cool, but also pretty simple. When it comes to complicate models, I afriad it would be hard to generate the accurate 3D model.
It says "can convert cad latents into a sequence of parametric CAD commands"
Which CAD program? I'm confused
Am I reading this right?
>Most importantly, GenCAD does not merely generate a 3D solid but also the entire CAD program.
> Which CAD program? I'm confused
Clue here: > Our proposed GenCAD architecture...
So, at this point, it seems like this will work with all CAD programs, since they have yet to encounter any systems that they can't work with. More seriously, my guess would be whatever one is available for free in their lab. Kind of standard operating procedure for academic projects -- do a proof of concept, make a video that avoids known bugs, get a grade, push source to git, graduate. Good ideas come out of that... production code... eh... maybe.
More likely someone ends up in the situation that my kid did, previous graduate student's git repo is stale by 2 versions of C++, and 4 versions of ROS, and neither of the two unit tests still work after porting.
It's DeepCAD* output, it looks like, which is a JSON payload that is the sketch / extrude / whatever steps, which is itself based on Onshape output.
Looks like you can go JSON -> step files, but not really in such a way that you can modify any of the operations.
* https://github.com/mightyhorst/DeepCAD
> Which CAD program?
Doesn't matter. CAD models/objects are represented by a sequence of operations on a primitive or sketch. Unlike meshes, that describe the manifested resulting shape of objects in 3D programs like Blender.
So it's about the fact, that their model outputs that hierarchy of operations. The history of development, not just the result.
How does it not matter? Every CAD program is not going to have exactly the same interface and commands. I doubt for example this will for example generate and OpenSCAD text file.
It could be used as pseudo-code for LLMs to produce specific CAD commands?
It will still be application dependent.
Code to compute fillets and blends gets incredibly complex when multiple surfaces are involved. And when surfaces are barely intersecting, or almost coincident, all bets are off what the command will do - very much depends on the geometry kernel and the tolerances it uses whether it decides the surfaces even intersect. And if it decides they don't intersect, all downstream commands will fail. Handling tolerances is one of the hardest aspects of CAD. (It's no coincidence that most open source CAD applications always demo with the same relatively basic types of models - they just can't do truly complex CAD.)
So a simple set of operations - cube, sphere, intersect - sure that will work anywhere and will be portable across applications and makes a nice simple demo. But once you start doing any serious CAD modeling the result is kernel dependent. That's why portable CAD formats like STEP do not preserve the commands used to generate the results. And why native CAD application formats do preserve the command history but are not portable across applications.
It could be anything which is why the question was asked what it actually outputs. I had a skim through the page and code but couldn't see what the output was.
Nothing stops you from storing a history of mesh operations. This is exactly what modifiers (including geometry nodes) do in Blender today.
Is this Google-affiliated? The heading font is Product/Google Sans which IIRC only Alphabet is allowed to use and the entire webpage seems to be Google-style but neither of the two named researchers seem to be employed by Google?
Per https://fonts.google.com/specimen/Google+Sans/license
"These fonts are licensed under the Open Font License. You can use them in your products & projects – print or digital, commercial or otherwise."
Yeah, TIL, turns out they changed the license: it used to be under https://fonts.google.com/license/productsans
the idea is good, but the examples still feel like a distance to handle real constraints and dimensions
A another take on this problem is zoo.dev . They wrote a brand new from scratch cad engine that is driven a custom openscad style language called kcl.
Then then have a trained llm that has can generate kcl to either create new parts or act as a llm assistant for changes to existing parts.
It’s neat that llms can do 3-D but I wonder how much of the problem is integration.
Website renders so poorly on my phone that I cant read half the text. Fits the bill for a slop project.