You: I'm not sure, it seems random
Nietzschian: "Random" is the word we use when we haven't looked
hard enough. What do the server logs say for the 403
responses? Go look. I'll wait.
It might seem obvious to look at the logs, but in my experience 90% of being stuck debugging something is just not yet having looked at all the available evidence.
It's true you could learn these habits by exposure to them, and this could be a neat alternative to pairing with a more senior engineer for that. But it looks like you could easily just keep saying "it's random", "I don't know", and the model would keep telling you what to do next, without you actually having to think.
I wonder if making it even more Socratic would fix this? E.g. "how do you know it's returning 403s? Oh, logs you say? Is there anything else they could tell you?"
It depends on where the developer is at in their journey too.
For some, "look in the logs" is a challenge just to find out what they are and where to find them and what they are saying. A great piece of advice for a person in this position.
For others, "There are files that hide in the caves of your computer's consciousness. Find what causes the shadows on the walls." is a valid call to dig deeper :)
1. They ask you a "hard question" but then immediately give you some kind of "hint" which makes it obvious what the "answer" is (or, even worse, they just tell you what it is).
2. They get pedantic (and I mean maximum emphasis on the word), to the point that they will nitpick the exact, precise wording you use, even if the context makes it blatantly obvious what you meant. GPT is currently the biggest offender in this area (it is ridiculously pedantic even though I haven't told it to be so).
And yes, in case anyone is wondering, I do actually read the output of tools like Claude and I have tried, repeatedly, to get them to do something like this, and one of the two aforementioned things always happens for some reason. I'm not sure if this is a problem with how they were trained, an issue with their system instructions, or what.
The saint answered: “I make hymns and sing them; and in making hymns I laugh and weep and mumble: thus do I praise Claude.”
When Zarathustra was alone, however, he said to his heart: “Could it be possible! This old saint in the forest hath not yet heard of it, that Claude is dead!”
He really nailed a godless post-modernism decades before it flourished. Even mentioning the names of the modern protagonists will get me down-voted :)