Tagged Netstrings (tnetstrings) was a related proposal from 15 years ago or so. It replaces the comma with a single-character type definition so you can do JSON-like objects with a couple of recursive types: you had ',', '#', '^', '!', and '~' for strings, integers, floats, booleans, and nulls, then ']' and '}' for lists and dictionaries.
I’ve near infinite respect for DJB, so I’m assuming I’ve missed something obvious here. Why is ‘,’ being used as a termination byte. Is it just a backstop? If buf[len+1] != ‘,’ then there’s a line error?
I’ve done plenty of wire protocol work, and length prefixed strings are great to work with. I’ve also been lucky that those strings were typically contained within a broader payload. To that end, I’ve not had to think about the case of many strings one after another.
Making the thing that describes the bounds of an arbitrary length thing itself arbitrary length sound like an unnecessarily risky complication to me.
Especially since it only grows with the log of the thing it bounds. So, we could easily have s fixed length length field that covers all ever possible length values.
if (scanf("%9lu",&len) < 1) barf(); /* >999999999 bytes is bad */
if (getchar() != ':') barf();
buf = malloc(len + 1); /* malloc(0) is not portable */
if (!buf) barf();
if (fread(buf,1,len,stdin) < len) barf();
if (getchar() != ',') barf();
Ah, the wonders of error-handling in C. Also, I wonder what's wrong with
I don't like formats that look like text buy may actually contain binary data - that's only going to tempt implementations that will choke when the string actually contains arbitrary data. Would be safer to encode the length and/or separator as something more obviously binary, which will also make the thing easier to parse in low-level implementations.
BitTorrent's bencoding format, used in .torrent files, effectively uses netstrings-- but without the trailing commas, so it uses "5:hello" to represent filenames and similar.
Tagged Netstrings (tnetstrings) was a related proposal from 15 years ago or so. It replaces the comma with a single-character type definition so you can do JSON-like objects with a couple of recursive types: you had ',', '#', '^', '!', and '~' for strings, integers, floats, booleans, and nulls, then ']' and '}' for lists and dictionaries.
Most of the links have bitrotted and I don't think it ever got much traction, but I did always like how simple it was. There's a copy someone grabbed of the original spec here: https://raw.githubusercontent.com/ged/tnetstrings.info/refs/...
Here's a HTML viewer (not mine; use it at your own risk): <https://html-preview.github.io/?url=https://raw.githubuserco...>
I’ve near infinite respect for DJB, so I’m assuming I’ve missed something obvious here. Why is ‘,’ being used as a termination byte. Is it just a backstop? If buf[len+1] != ‘,’ then there’s a line error?
I’ve done plenty of wire protocol work, and length prefixed strings are great to work with. I’ve also been lucky that those strings were typically contained within a broader payload. To that end, I’ve not had to think about the case of many strings one after another.
Making the thing that describes the bounds of an arbitrary length thing itself arbitrary length sound like an unnecessarily risky complication to me.
Especially since it only grows with the log of the thing it bounds. So, we could easily have s fixed length length field that covers all ever possible length values.
I don't like formats that look like text buy may actually contain binary data - that's only going to tempt implementations that will choke when the string actually contains arbitrary data. Would be safer to encode the length and/or separator as something more obviously binary, which will also make the thing easier to parse in low-level implementations.
Seems like a coherent, sensible proposal, as one might expect from djb. Any notable protocols use them?
BitTorrent's bencoding format, used in .torrent files, effectively uses netstrings-- but without the trailing commas, so it uses "5:hello" to represent filenames and similar.
Not sure if it counts as notable, but SCGI uses it too: https://python.ca/scgi/protocol.txt
Php serialized uses
For strings, which is pretty similar. Size is in bytes.zurl and mongrel2 are using it.
(1997) -DJB