AI in engineering. Is the product right if it looks right?

During the development of our Engineering AI, I realized that today's AI models are mostly incredible librarians.

They've read almost everything, and they're brilliant at retrieving and remixing it. Ask for a picture of a chair and you get a convincing chair. But a picture of a chair and a chair you can actually build are very different things. One just has to look right. The other has to hold weight, fit together, and survive contact with the real world.

That gap is where the next phase of AI gets interesting. Not models that know what things look like, but models that understand how things work. Cause and effect. Physics. Whether two parts actually fit, not just whether they look like they do.

We realized that this is the hardest part when using AI in engineering: real, manufacturable products. It's unforgiving but in a useful way. If a language model is slightly wrong, the answer might sound a little off. In an engineering context, if a system is slightly wrong, a part gets driven straight through another one, and the assembly fails. You can't fake your way past reality.

We think that's actually the most honest benchmark for this whole shift. "Looks plausible" is where AI is today. "Works when you build it" is where we are going.

Curious what others working in AI think: where do you see the line between models that describe the world and models that actually understand it? And what's the domain that finally forces the difference?