> "...or the parts that have never once executed in production because they’re there for a worst case someone sat down and imagined."
Good stuff.
> "It’s the record of what the code was supposed to do ... written down somewhere a human or an agent can read them back. Ours are markdown files in git..."
I keep getting surprised how many quite technical people I know are doing real work with AI but have never touched markdown.
If you are one of those people, allow me to offer a suggestion:
Stop. No more sending questions to ChatGPT or Claude until you do the following:
- Install Typora on your Windows or Mac machine.
- Write a .md (markdown text) file a full page long using Typora (or another editor that understands markdown, but Typora will do just fine). Make use of Heading 1, Heading 2, and underline formatting, at least.
- Make the one page file be a complex request to an AI. Something like "research x, compare it to y, make a guess what will happen in the future, here are some of my opinions and guesses, make your replies short, provide references".
Hand that .md file to your favorite AI. (Bonus points for using drag and drop or pointing the AI at the dir the file is in.) Look at your results. Make a change to your .md file. Hand it to the AI again, see the different results.
In my opinion, if you work on a keyboard and you've been making use of AI for the last 8 months, you should have done the above 4 months ago. Until you do, you will have a hard time understanding how drastically AI has changed almost all developer's lives, and how it is changing more and more fields that are not software development.
Every time I see a comment like this I feel like you are only 50% into the transition that is occurring.
The crafting of the requirements is better suited to an interactive process. You will have superior results and a quicker turn around by having the AI research the change implications, what current implementations actually do and where (so you can look at the parts you want to), etc.
When working in a BIG codebase, having it detail up a flow of how all the parts fit together is great and helps get you setup to work. Then work through your planned constraints with the AI and make sure the perception matches your intent (with prevents a lot of issues that a fire and forget document creates).
Then when you are satisfied that it sees the problem as you do, have the model capture it and then let it rip.
Oh I agree totally. I find it to be one of the coolest parts of the whole thing: Getting the AI to help put together the docs that will be handed to the AI. But the "quite technical people" that I referenced are not "full technical people"; I would lose them if I tried to explain the above to them.
I also really like telling the AI, "while we are hacking our way through the jungle trying to figure out how this thing should be built, at the same time, I want you to keep a design document up to date so that, if I asked you to rebuild the whole system again but without all the false starts etc we went through while we were figuring it out together, you could do so."
I probably won't use any of the "zombie clone" versions of my app that get built from the AI-generated design doc, but the fact that the zombie app (and design doc) look good at a casual glance, and the design doc is not 200 pages of AI bloatware verbiage, seems like good evidence that if a human expert needed to step in and save the system from some big surprise, that design spec would be found to be exactly what that human needed to help find the problem.
How is this new? The code is the spec has been a thing for a very long time.
Honestly using markdown is really inefficient. Code is much more expressive and terse. Why do you need all these markdown files when you have code already? I could see markdown if you can’t read code, but otherwise…
Don’t get me wrong docs are useful when code isn’t sufficient (for more esoteric things), but in many cases code is much better.
Well, you've changed the subject but fine, let's go there.
That's one of the things that has drastically changed, in my opinion. One way to look at it could be: in the past it was okay to have a minimal design spec doc, if your programmer wrote excellent code and comments, and the system wasn't too big.
But now, you can't complete if you try to do it that way while not using AI. You need to take all that effort that used to go into good code and good comments, and put it into a good spec doc, and hand that file to the AI.
So that once the AI has built your system, you can say either "hey AI, a bug just showed up in the widget area, go look at your strange spaghetti code of yours and the design doc and fix it", or, "hey AI, the whole sub-system seems to have gone critical mess, take the code, spec doc, lessons learned, and the following few high-level design decision changes, and re-write the whole thing!"
Apologies for bluntness, but there just isn't going to be much need in this world for people that are excellent at hand-building intricate watches, in a world where $20 quartz-movement watches have appeared.
> You need to take all that effort that used to go into good code and good comments, and put it into a good spec doc, and hand that file to the AI
This is inefficient. If quality matters at all, you’re going to miss things in your English design doc and/or use 100 words where 1 line of code will do.
If quality (as in product, not code) doesn’t matter then sure, do whatever. LLM will fill in the gaps.
> world where $20 quartz-movement watches have appeared
Much like in real life, if you’re paying for a cheap knock off you get what you pay for. Also much like real life, plenty of people will pay for $20 knock offs and be content.
i have absurdly lightweight amounts of markdown within my projects that are meant to stay for the long term, a minimal agents.md, which whatever harness im using that day consumes, which maybe has some lines that point at another md file or two or whatever if i deem it vital for rules of a specific section
The real use for markdown for me and why i have a TON of markdown usage, is for planning tasks. these aren't "permanent" or anything but end up in a very much archived section of the repo once concluded but will have commit ids and such attached so intent can be retrieved at a later date if needed, as well as session ids for whatever model/harness i was using for that particular task.
It also serves another purpose of allowing me to nicely chunk up work as i see fit, so if i start a task in CC, and would prefer a section be done with some other model i can easily do so, or if i feel the work is suited to parallel sessions i can spin those up as well, with whatever models/harnesses i want.
Yeah, this is where I’ve landed after trying to painfully construct a markdown plan and hope it’d be implemented to spec the first time through. I’d review, find mistakes, get annoyed and then realize it was simpler to just build iteratively. And just have it give me code when possible, since that cut through the middleman, so to speak.
I’ll get a rough markdown plan to start but don’t treat it as much more than a launch point.
> Install Typora on your Windows or Mac machine.
- Write a .md (markdown text) file a full page long using Typora (or another editor that understands markdown, but Typora will do just fine). Make use of Heading 1, Heading 2, and underline formatting, at least.
Decent points, but at this point if the restaurant owner could articulate their needs at a non technical level, I bet Claude or Codex could do a good job. Which is harder than it sounds but not exactly rocket science:
* Make an app for scheduling shifts for my staff.
* Needs to handle multiple users with intelligent reconciling of edits
* Needs to adapt to both English and Spanish speakers.
* Needs authentication
* Need 99.9% reliability
This gets most of the way there. Presumably the restaurant owner will need to iterate, but isn't that always the case? Iterating now doesn't require hiring external help.
I can only read code at a basic level, so I am in the same position as the author of this article.I built a diagnostic app using AI. With my initial prompt, I was able to get it to generate diagnostic results.However, when I ask AI to make changes, something else sometimes breaks. Every time this happens, I have to ask AI to check the code again.
The anecdote the author gave is already kind of outdated, since Claude will write tests and debug itself as it progresses. It doesn't take much insight to realize that the maintainers are going to be AI.
Houses are still standing built with hand saws and elementary-school level tools; we don't tear them down, just cut new pieces with power tools and bolt on new additions built with new pieces made from advanced tools.
Isn't AI just another purity test? nasty VSCode users instead of pure elite Vim users?
While the maintainers surely will be AI, the last bastion is the architects. Claude can write the scaffolding, but does Claude know how the parts go together? Even the concept of fuzz-testing existing means that even humans can't envision all the failure paths, so it would be unfair to hold the metric against AI. What we are going to see more of is the architectural failures: massive data breaches because the architecture aroung the breach will be reveaed to be laughably simple, from prompts that just didn't know what to ask.
As long as claude don't use breakpoints skillfully enough to analyse weird behaviors, you will have to do some research yourself, sometimes. Especially in huge OOP codebase. Because print-based debugging can only get you so far until the context explode (and even then, honestly it's almost always faster to investigate yourself if your code if of high quality, which, i will admit, is becoming an issue lately)
I wonder how big you could say my friend Larry's "context" memory is. He's the long-time architect of a big, complex, international micro-services based system.
I'm guessing that when he goes to debug a complex problem that spans across multiple systems for his app, his brain is paging through the equivalent of a well-written 3GiB markdown design document file.
And my opinion is, yes, pretty soon SOTAs will be able to be that guy, if we are willing to hand that much control to an AI. But I sure hope we don't--it seems like way too much power for an AI. However: Larry sitting next to Fable working together? A team like that could change the world. I hope he's digging in to Claude or something, I'll have to give him a call.
Agreed. Also, Claude could relatively easily add the features he lists as lacking (dark mode, translation, etc.). Responsive design is on by default now in most models. Stuff like logins/auth would be more complicated, but Claude could certainly do it -- it would make for heavier hosting dependency, though.
> "...or the parts that have never once executed in production because they’re there for a worst case someone sat down and imagined."
Good stuff.
> "It’s the record of what the code was supposed to do ... written down somewhere a human or an agent can read them back. Ours are markdown files in git..."
I keep getting surprised how many quite technical people I know are doing real work with AI but have never touched markdown.
If you are one of those people, allow me to offer a suggestion:
Stop. No more sending questions to ChatGPT or Claude until you do the following:
- Install Typora on your Windows or Mac machine.
- Write a .md (markdown text) file a full page long using Typora (or another editor that understands markdown, but Typora will do just fine). Make use of Heading 1, Heading 2, and underline formatting, at least.
- Make the one page file be a complex request to an AI. Something like "research x, compare it to y, make a guess what will happen in the future, here are some of my opinions and guesses, make your replies short, provide references".
Hand that .md file to your favorite AI. (Bonus points for using drag and drop or pointing the AI at the dir the file is in.) Look at your results. Make a change to your .md file. Hand it to the AI again, see the different results.
In my opinion, if you work on a keyboard and you've been making use of AI for the last 8 months, you should have done the above 4 months ago. Until you do, you will have a hard time understanding how drastically AI has changed almost all developer's lives, and how it is changing more and more fields that are not software development.
Every time I see a comment like this I feel like you are only 50% into the transition that is occurring.
The crafting of the requirements is better suited to an interactive process. You will have superior results and a quicker turn around by having the AI research the change implications, what current implementations actually do and where (so you can look at the parts you want to), etc.
When working in a BIG codebase, having it detail up a flow of how all the parts fit together is great and helps get you setup to work. Then work through your planned constraints with the AI and make sure the perception matches your intent (with prevents a lot of issues that a fire and forget document creates).
Then when you are satisfied that it sees the problem as you do, have the model capture it and then let it rip.
Oh I agree totally. I find it to be one of the coolest parts of the whole thing: Getting the AI to help put together the docs that will be handed to the AI. But the "quite technical people" that I referenced are not "full technical people"; I would lose them if I tried to explain the above to them.
I also really like telling the AI, "while we are hacking our way through the jungle trying to figure out how this thing should be built, at the same time, I want you to keep a design document up to date so that, if I asked you to rebuild the whole system again but without all the false starts etc we went through while we were figuring it out together, you could do so."
I probably won't use any of the "zombie clone" versions of my app that get built from the AI-generated design doc, but the fact that the zombie app (and design doc) look good at a casual glance, and the design doc is not 200 pages of AI bloatware verbiage, seems like good evidence that if a human expert needed to step in and save the system from some big surprise, that design spec would be found to be exactly what that human needed to help find the problem.
How is this new? The code is the spec has been a thing for a very long time.
Honestly using markdown is really inefficient. Code is much more expressive and terse. Why do you need all these markdown files when you have code already? I could see markdown if you can’t read code, but otherwise…
Don’t get me wrong docs are useful when code isn’t sufficient (for more esoteric things), but in many cases code is much better.
Well, you've changed the subject but fine, let's go there.
That's one of the things that has drastically changed, in my opinion. One way to look at it could be: in the past it was okay to have a minimal design spec doc, if your programmer wrote excellent code and comments, and the system wasn't too big.
But now, you can't complete if you try to do it that way while not using AI. You need to take all that effort that used to go into good code and good comments, and put it into a good spec doc, and hand that file to the AI.
So that once the AI has built your system, you can say either "hey AI, a bug just showed up in the widget area, go look at your strange spaghetti code of yours and the design doc and fix it", or, "hey AI, the whole sub-system seems to have gone critical mess, take the code, spec doc, lessons learned, and the following few high-level design decision changes, and re-write the whole thing!"
Apologies for bluntness, but there just isn't going to be much need in this world for people that are excellent at hand-building intricate watches, in a world where $20 quartz-movement watches have appeared.
> You need to take all that effort that used to go into good code and good comments, and put it into a good spec doc, and hand that file to the AI
This is inefficient. If quality matters at all, you’re going to miss things in your English design doc and/or use 100 words where 1 line of code will do.
If quality (as in product, not code) doesn’t matter then sure, do whatever. LLM will fill in the gaps.
> world where $20 quartz-movement watches have appeared
Much like in real life, if you’re paying for a cheap knock off you get what you pay for. Also much like real life, plenty of people will pay for $20 knock offs and be content.
i have absurdly lightweight amounts of markdown within my projects that are meant to stay for the long term, a minimal agents.md, which whatever harness im using that day consumes, which maybe has some lines that point at another md file or two or whatever if i deem it vital for rules of a specific section
The real use for markdown for me and why i have a TON of markdown usage, is for planning tasks. these aren't "permanent" or anything but end up in a very much archived section of the repo once concluded but will have commit ids and such attached so intent can be retrieved at a later date if needed, as well as session ids for whatever model/harness i was using for that particular task.
It also serves another purpose of allowing me to nicely chunk up work as i see fit, so if i start a task in CC, and would prefer a section be done with some other model i can easily do so, or if i feel the work is suited to parallel sessions i can spin those up as well, with whatever models/harnesses i want.
Yeah, this is where I’ve landed after trying to painfully construct a markdown plan and hope it’d be implemented to spec the first time through. I’d review, find mistakes, get annoyed and then realize it was simpler to just build iteratively. And just have it give me code when possible, since that cut through the middleman, so to speak.
I’ll get a rough markdown plan to start but don’t treat it as much more than a launch point.
Or at least have your LLM generate a context .MD file for you.
No.
You need to be able to read and write for yourself at some level.
Markdown is like, the closest thing you can get to WYSIWYG without exiting the realm of machine-readable plaintext.
> Install Typora on your Windows or Mac machine. - Write a .md (markdown text) file a full page long using Typora (or another editor that understands markdown, but Typora will do just fine). Make use of Heading 1, Heading 2, and underline formatting, at least.
Who's still typing?
> I keep getting surprised how many quite technical people I know ... > > - Install Typora on your Windows or Mac machine.
I keep getting surprised how many people don't realize Linux is a thing.
I can't get them to try an app that is barely more complicated than Notepad, and you think I can get them to switch to Ubuntu?
Note that I said "quite technical people", not "fully technical people" or developers.
Decent points, but at this point if the restaurant owner could articulate their needs at a non technical level, I bet Claude or Codex could do a good job. Which is harder than it sounds but not exactly rocket science:
* Make an app for scheduling shifts for my staff.
* Needs to handle multiple users with intelligent reconciling of edits
* Needs to adapt to both English and Spanish speakers.
* Needs authentication
* Need 99.9% reliability
This gets most of the way there. Presumably the restaurant owner will need to iterate, but isn't that always the case? Iterating now doesn't require hiring external help.
I can only read code at a basic level, so I am in the same position as the author of this article.I built a diagnostic app using AI. With my initial prompt, I was able to get it to generate diagnostic results.However, when I ask AI to make changes, something else sometimes breaks. Every time this happens, I have to ask AI to check the code again.
were in the slopocalypse
The anecdote the author gave is already kind of outdated, since Claude will write tests and debug itself as it progresses. It doesn't take much insight to realize that the maintainers are going to be AI.
Houses are still standing built with hand saws and elementary-school level tools; we don't tear them down, just cut new pieces with power tools and bolt on new additions built with new pieces made from advanced tools.
Isn't AI just another purity test? nasty VSCode users instead of pure elite Vim users?
While the maintainers surely will be AI, the last bastion is the architects. Claude can write the scaffolding, but does Claude know how the parts go together? Even the concept of fuzz-testing existing means that even humans can't envision all the failure paths, so it would be unfair to hold the metric against AI. What we are going to see more of is the architectural failures: massive data breaches because the architecture aroung the breach will be reveaed to be laughably simple, from prompts that just didn't know what to ask.
As long as claude don't use breakpoints skillfully enough to analyse weird behaviors, you will have to do some research yourself, sometimes. Especially in huge OOP codebase. Because print-based debugging can only get you so far until the context explode (and even then, honestly it's almost always faster to investigate yourself if your code if of high quality, which, i will admit, is becoming an issue lately)
I wonder how big you could say my friend Larry's "context" memory is. He's the long-time architect of a big, complex, international micro-services based system.
I'm guessing that when he goes to debug a complex problem that spans across multiple systems for his app, his brain is paging through the equivalent of a well-written 3GiB markdown design document file.
And my opinion is, yes, pretty soon SOTAs will be able to be that guy, if we are willing to hand that much control to an AI. But I sure hope we don't--it seems like way too much power for an AI. However: Larry sitting next to Fable working together? A team like that could change the world. I hope he's digging in to Claude or something, I'll have to give him a call.
Agreed. Also, Claude could relatively easily add the features he lists as lacking (dark mode, translation, etc.). Responsive design is on by default now in most models. Stuff like logins/auth would be more complicated, but Claude could certainly do it -- it would make for heavier hosting dependency, though.