Posted by vbtechguy 4/7/2025
What was the idea behind this choice?
Are the game state updates and input coupled with the frame rate, as in most beginner game development tutorials out there, or did the "AI" do the right thing and decouple them?
Should we be checking in our prompt history into version control as a kind of source code or requirements spec? Seems prompt revision and improvement would be valuable to keep a history of.
It's amusing that on one hand, there's been a push for "reproduceable builds", where we try to make sure that for some set of input (source code files, configuration, libraries), that we can get an identical output. On the other hand, we have what we see here where, without a huge amount of external context, no two "builds" will ever be identical.
https://harper.blog/2025/02/16/my-llm-codegen-workflow-atm/
The reasoning models seem to respond quite well to the "ask me one question at a time, building on my previous answer" for the sake of coming up with a solid blueprint for the project, then build from there. I used this method to make a nice little inventory/rma tracker for my job using Rails 8, and it got me MVP in a matter of hours, with most delays being just me refining the code on my own since some of it was a bit clunky. I used o3 mini to generate the initial prompts, then fed those to Claude.
The hallucinations/forgetfulness was relatively minor, but if I did not have prior Ruby/JS knowledge, I doubt I would have caught some of the mistakes, so as much as I was impressed by the method outlined in the blog post, I am not at all saying that someone with no knowledge of the language(s) they wish to use is going to create a great piece of software with it. You still have to pay attention and course correct, which requires a working understanding of the dev process.
My lessons learned for prompting were to have project broken down into clearly defined modules (duh...), and to constantly feed the latest module source back in as context along with the prompts. This helps ground the responses to only adjust the code related to the prompt, and to not break stuff that was already working.
What does this mean?