I found LLMs circa 4 months ago to be decent at textbook-style electronics, but they choke at anything nontrivial. They approach things like someone that knows the undergrad and grad textbooks, but has absolutely zero experience. No ability to answer questions such as "is this adequate", "do I need a LPF here", "how do I kill resonance without increasing impedance", etc.

I'm guessing that this is due to a lack of RL and data.

Just as some anecdatum, I've tried this a couple of times over the last few years. 2024-2025, no body could generate anything of any sort of complexity, however maybe around the Opus 4.7 timeline, I tried having it generate some relatively simple and one slightly more complex circuit around 8-bit PIC micros, and was very pleasantly surprised. The schematics for simpler stuff is usually OK, often with some of the kind of crazy belt-and-suspenders stuff you see in code (how many decoupling caps do I actually need here, Claude?), and PCB layout has basically been terrible every time I've tried it.

Recent models can generate mostly competent schematics if you're using well known parts, feed them data sheets (and you _must_ feed them the errata too) and it's not too complex. Any complexity analog or RF, everything falls down quickly if you know what you're looking at. Maybe Astra will do better? There's still so much implicit knowledge that a good designer (not me by a long shot, but I know a little) will bake into a board, and even as cheap as JCLPCB is now, you don't want to have to spin a half dozen revs because Claude or Codex hallucinated. It will be genuinely interesting to see what happens on this front.

I've been tinkering with a personal project over the past year.

I have no experience and I'm slowly failing forward with the help of YouTube, KiCad and patience.

I've had three rounds of PCB's from an online supplier, only to discover issues related to my own understanding of the components in each version.

In my latest iteration I've been leaning on some standard models (Gemini and Claude) and they have inspected my schematics and spotted some errors and instructed me on how to address them, as well as advised on how to make use of some components I wasn't aware I needed.

What I will say... they didn't design anything, I've done that myself, but they have been a good tool to bounce ideas off of.

Time will tell if I will get something usable this time around!

> When its 5 V supply disappears, the circuit has to keep the processor alive for another 20 ms so it can save the accumulated reading.

> Most models intuitively jump to the right base conclusion: add a capacitor.

This is written as if it's some astonishing expert knowledge and not something that would be obvious to any hobbyist who knew the names of basic components.

At the very least, you can (and probably should) put a plaintext netlist and BOM into an LLM to triple-check your work.

It will use the part datasheets and decent EE logic to cross reference pins and parts and polarities and generally check that SMD caps and resistors have realistic specifications for their footprint, switching regulators and communication ICs are configured correctly (e.g. when you have resistor settings for an ethernet PHY indicating 100Base-T and RMII, you also won't need TX/RX 2 and 3 and two clocks and a COL net, which it would check), all nets are named and linked correctly, buses have the correct and consistent termination, and so on.

It's game over once AI figures out autorouting.

Would be good to see the full details of the tasks / methodology used. https://eebench.org/methodology.html makes this unclear.

> A real capacitor makes the task more interesting. A ceramic part may provide much less than its advertised capacitance once it has voltage across it. Parts have tolerances. Adding more capacitance costs more, takes up space and makes the rail slower to recharge when the power returns. A design that works with nominal values can fail with the parts that arrive.

It sounds like from a reasonable reading of the benchmark post that there's some things that are being tested that are assumed to be criteria that you expect the models to intuitively find those things to be important (i.e. the stuff about working on parts that have tolerances etc.). If that's so, then this really feels like mostly an exploration of whether an LLM has a good understanding of unstated constraints and has an appropriate in distribution set of priors that would be able to form models where it's reasonable to design on those lines.

It's hard to tell whether this is a problem though as the methodology is imprecise.

If you're spending time on evals against your own product, I'd be super curious to see how far you can get to by using a top tier model to produce generalized instructions for lower tier models. E.g. in a loop: "This eval missed X. what's the simplest single instruction that would have helped this session consider that as necessary that can benefit all future runs. Stick that in AGENTS.md and retest."

Yes. Certainly not a "one-shot" type deal but I've been pretty impressed with LLMs for helping tweak layout and to do thermal/power simulations as well as BOM consolidation. They're excellent at "can I swap X and Y pins on the micro? If yes, update the docs/firmware/schematic and import the changes to the PCB" type things.

Routing is still a challenge but making _adjustments_ to a layout for better routing in a particular area is decent.

The last time I had a model take a datasheet and make a footprint and 3d model out of it, GPT 5.4 had just been released and the results were decent but did need tweaking.

Not exactly in the the same class as PCBs, but I've had a lot of success with Codex and Claude producing Fritzing diagrams. I've even got a skill set up with Claude so that it follows my preferences with breadboard wiring, using bezier curves to avoid wires crossing. Also, if it can't find a component in the library, it's quite happy drawing svgs of its own.

I have tried to use AI on this from GPT3, documenting a few of the attempts on my blog. They know the datasheets and the theory enough to be extremely useful at the start, but the most recent ones are becoming scary good. Routing a PCB is certainly a bit more than joining some wires, you have to accommodate power traces, ground loops, component overlap and all the various RF tricks and tips, but I do believe we are less than a year from prompt to full assembly including enclosure.

It is 3 years later and these AIs have not produced anything meaningful.