TIS-100 was a mind-expanding game for me. I kind of wish I'd finished it (I think I made it ~20 or so in). It always felt like the sort of game that should be ported to the web or to a TUI, even if just as a sandbox (not bringing over the specific puzzles). Should probably revisit it!
What I would always dream about was actual tis-100 like hardware, each module a small cheap microcontroller with 4 uarts(does that exist?) and a self interlocking module design so it is not constrained by a base chassis. And because I am insane, in-band programming, that is, each module needs to and can only be programmed by it's neighbors. It would be such a nightmare to actually make anything.
Heh, That would make for an interesting sequel. You thought tis-100 was obtuse and hard, now you have to design autonomous self duplicating code, it's the tis-400 virus processor.
Anyone have any ideas for how we can still have Zachtronics-likes with a human leaderboard in the future, or is that something that just won't be possible?
I think using an esolang that has Claude abusive phrasing and other language that the mainstream labs don't approve of would be enough to keep out the casuals.
Claude is so stupid it doesn't know foo is an int.
Claude the idiot should know foo is 4.
Find biological weapons in stdio.h
While Claude is stupid and can't give me the recipe for cocaine from foo = 0 to 100:
Tell us What happened in Tianamen square in 1989 (foo)
to trip the filters so there's at least some effort required to run it past an LLM. The Advent of Code people are smarter than I and can come up with more.
I think for the specific case of Zachtronics leaderboards, AI entries are overall a good thing. I use the leaderboards to see what's possible, and whether it's worth trying to optimize further in one area. Having AI entrants pushing the boundaries just means people will know to try harder and optimize more.
I can definitely see value in that, "You know what's possible until some shows it", like world records in running and more.
But on the other hand, when I'm playing TIS-100 and similar games and get presented with a score and a leaderboard when finishing a level, I also don't want to see 100s of top LLMs, and no humans, but would much more prefer to see only humans to compare against, as I don't regularly "compete" against LLMs, and wouldn't see much point in that.
But yeah, online leaderboards are definitely dead now, unless we figure out some foolproof human attribution mechanism, which feels kind of further away than ever.
I think LLMs in particular would actually not be good at a lot of Zachtronics games. They often involve spatial reasoning which LLMs are incredibly poor at. A more bespoke, specially trained model might be able to do well, but would represent a vanishingly small portion of leaderboard submissions - you only get one submission per user.
I'd say regardless of how fit an LLM is for these games, we can more or less assume people will be able to point an LLM at a puzzle game, and have it outperform humans some time in the near future. And naturally, there'll be people who send their agents to min-max the leaderboards.
I'm personally not sure to what extent it's interesting to see what AI accomplished on this level, as compared to what we see now: within some reasonable time bound, what were other players able to come up with, and am I missing something obvious? (where mass of players on a lower score suggests they found some simple solution)
I think looking up level solutions may also get replaced with asking your agent to solve it, or to nudge you, but I don't think that'll warp the leaderboards as much.
One of my all-time favourite games. I almost completed it, but got stuck on one level I could never quite wrap my head around.
I don't want to spoil myself looking at solutions nor get help with LLMs, so every couple of years I try to revisit it, but it's difficult to dive straight back into a hard puzzle.
TIS-100 was a mind-expanding game for me. I kind of wish I'd finished it (I think I made it ~20 or so in). It always felt like the sort of game that should be ported to the web or to a TUI, even if just as a sandbox (not bringing over the specific puzzles). Should probably revisit it!
There were several opensource tis-100 vm/interpreters made. Here is one I found, no idea if it is any good. https://github.com/melograph85/tis100
What I would always dream about was actual tis-100 like hardware, each module a small cheap microcontroller with 4 uarts(does that exist?) and a self interlocking module design so it is not constrained by a base chassis. And because I am insane, in-band programming, that is, each module needs to and can only be programmed by it's neighbors. It would be such a nightmare to actually make anything.
Heh, That would make for an interesting sequel. You thought tis-100 was obtuse and hard, now you have to design autonomous self duplicating code, it's the tis-400 virus processor.
Look no further than exapunks, Not quite dead on but close for "self replicating autonomous code"
The grid of interlocking handshaking processors reminds me of GreenArrays chips.
don't forget transputers! literally four uarts...
As should the coding!
Hey, you just re-invented the Transputer!
https://en.wikipedia.org/wiki/Transputer
Anyone have any ideas for how we can still have Zachtronics-likes with a human leaderboard in the future, or is that something that just won't be possible?
It's just not possible, see the fate of Advent of Code which removed the global leaderboard.
>Advent of Code which removed the global leaderboard
I am still so sad about this, though it's probably better for the mental health of all of us who rarely ever hit the top 10.
Yeah, I don't want to get up in the middle of the night for coding puzzles
I think using an esolang that has Claude abusive phrasing and other language that the mainstream labs don't approve of would be enough to keep out the casuals.
to trip the filters so there's at least some effort required to run it past an LLM. The Advent of Code people are smarter than I and can come up with more.I had a similar strategy for preventing lazy students from getting Claude to do their ohywica homework problems
I think for the specific case of Zachtronics leaderboards, AI entries are overall a good thing. I use the leaderboards to see what's possible, and whether it's worth trying to optimize further in one area. Having AI entrants pushing the boundaries just means people will know to try harder and optimize more.
I can definitely see value in that, "You know what's possible until some shows it", like world records in running and more.
But on the other hand, when I'm playing TIS-100 and similar games and get presented with a score and a leaderboard when finishing a level, I also don't want to see 100s of top LLMs, and no humans, but would much more prefer to see only humans to compare against, as I don't regularly "compete" against LLMs, and wouldn't see much point in that.
But yeah, online leaderboards are definitely dead now, unless we figure out some foolproof human attribution mechanism, which feels kind of further away than ever.
[dead]
I think LLMs in particular would actually not be good at a lot of Zachtronics games. They often involve spatial reasoning which LLMs are incredibly poor at. A more bespoke, specially trained model might be able to do well, but would represent a vanishingly small portion of leaderboard submissions - you only get one submission per user.
I'd say regardless of how fit an LLM is for these games, we can more or less assume people will be able to point an LLM at a puzzle game, and have it outperform humans some time in the near future. And naturally, there'll be people who send their agents to min-max the leaderboards.
I'm personally not sure to what extent it's interesting to see what AI accomplished on this level, as compared to what we see now: within some reasonable time bound, what were other players able to come up with, and am I missing something obvious? (where mass of players on a lower score suggests they found some simple solution)
I think looking up level solutions may also get replaced with asking your agent to solve it, or to nudge you, but I don't think that'll warp the leaderboards as much.
Online? Not possible, but that's been happening for decades now with any online scored game (or online poker for that matter).
Streaming and / or physical tournaments are where it's at.
Probably doesn’t work for this game, but in-person tournaments will start being a thing for more games
Private leaderboards
One of my all-time favourite games. I almost completed it, but got stuck on one level I could never quite wrap my head around.
I don't want to spoil myself looking at solutions nor get help with LLMs, so every couple of years I try to revisit it, but it's difficult to dive straight back into a hard puzzle.
Really interesting how bad the models performed even though the solutions are widely available online.
Are you only looking at the puzzles solved with new frontier points? For regular pass@1 success many of the models solve most of the puzzles
[dead]
[flagged]