This was very thoughtful essay; I've had similar lines of thinking myself. How will we debug auto-generated AI code?
That said, on that topic, the essay overlooks one key point and line of reasoning about debugging, one derived from Kernighan's Law. (Kernighan as in the K in AWK + K&R C...)
The law states: "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it."
If Kernighan's law is "true" in some rough sense (as I have long agreed with), then we have a potential solution to the "AI debugging" problem... ask the LLM to make the code four times simpler than it needs to or write code with a 4x dumber model. Then a smarter model (or us) can debug it. Right?
why do you need to debug it?
or, why do you need to debug it?
my crazy thoughts, which I am open to being wrong about:
this is still the old way of thinking. generative AI is a probability function—the results exist in a probability space. we expect to find a result at a specific point in that space, but we don't know the inputs to achieve it until it is solved. catch 22.
instead, we must embrace the chaos by building attractors in the system, and pruning results that stray too far from the boundaries we set. we should focus only on macro-level results, not the code. multiple solutions existing at any given time. if anything is "debugging", it is a generative AI agent.
slightly off-topic (but tangentially mentioned in the article). I google'd "Chicxulub" and saw an easter-egg I hadn't previously known about :)
Great article. I recently finished my second reading of TMM and how/to what extent our current era of generative code affects the ideas of the book was top of mind.
What if the large tech companies systems are actually just old and we don't need that many people to maintain them and change them. that way they don't have to admit the technology companies are a mature industry and their profits about about to get competed away.
I wondered recently how many saas systems could get replaced by a database with a chat front end. is saas going to go the way of cli which was enter by a mouse and gui which was eaten by the web/saas?
if so how much do we need to maintain to keep that kind of system up an running.
A whole lotta words and not a single mention of the actual book referenced in the title. What’s the paradox? How does it relate to the Mythical Man Month? Who knows!
This was very thoughtful essay; I've had similar lines of thinking myself. How will we debug auto-generated AI code?
That said, on that topic, the essay overlooks one key point and line of reasoning about debugging, one derived from Kernighan's Law. (Kernighan as in the K in AWK + K&R C...)
The law states: "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it."
If Kernighan's law is "true" in some rough sense (as I have long agreed with), then we have a potential solution to the "AI debugging" problem... ask the LLM to make the code four times simpler than it needs to or write code with a 4x dumber model. Then a smarter model (or us) can debug it. Right?
why do you need to debug it? or, why do you need to debug it?
my crazy thoughts, which I am open to being wrong about:
this is still the old way of thinking. generative AI is a probability function—the results exist in a probability space. we expect to find a result at a specific point in that space, but we don't know the inputs to achieve it until it is solved. catch 22.
instead, we must embrace the chaos by building attractors in the system, and pruning results that stray too far from the boundaries we set. we should focus only on macro-level results, not the code. multiple solutions existing at any given time. if anything is "debugging", it is a generative AI agent.
Haha, love it.
Step 1: determine how to quantify cleverness
Hm, have they tried to ask Claude debug generated code? It does it pretty well.
slightly off-topic (but tangentially mentioned in the article). I google'd "Chicxulub" and saw an easter-egg I hadn't previously known about :)
Great article. I recently finished my second reading of TMM and how/to what extent our current era of generative code affects the ideas of the book was top of mind.
What if the large tech companies systems are actually just old and we don't need that many people to maintain them and change them. that way they don't have to admit the technology companies are a mature industry and their profits about about to get competed away.
I wondered recently how many saas systems could get replaced by a database with a chat front end. is saas going to go the way of cli which was enter by a mouse and gui which was eaten by the web/saas?
if so how much do we need to maintain to keep that kind of system up an running.
A whole lotta words and not a single mention of the actual book referenced in the title. What’s the paradox? How does it relate to the Mythical Man Month? Who knows!
It's just the same paradox — adding more people^Wmachines to crank out more code won't actually get the project done faster.
No Silver Bullet
More like the Mythical Machine-Minute.