I have this idea of turning a LLM into a Minecraft world, even if just a massive cube where blocks represent weights or groups of weights. You can chat with the LLM while simultaneously being able to destroy it slowly.
I have no idea whether this is possible or what the result would be after enough blocks or sections were broken or TNTd, but it would make an interesting art project if at all feasible.
Where are we with scripting Minecraft worlds? I don't mean in a programmatic sense, but in the sense of saying to an AI "make me a worlds with a clearing in a giant forest where skeletons are attacking creepers" and it would make the world?
I play heavily modded Minecraft and have always dreaded endgame where you're often building out massive infrastructure. Would be amazing to have a little embedded ChatGPT that could generate modded structures with a prompt. The factory must grow :)
As of about a month ago there’s https://github.com/soapantelope/mindcraft. You can give it a prompt like “A Minecraft world of beautiful flower-filled meadows abruptly changing into swamps.” and get that. As far as I can tell it’s using Minecraft’s existing world generator, the readme notes “As of now, Mindcraft can only generate worlds that are feasible natural overworld biomes in Minecraft 1.21.11. Keep this in mind when thinking of prompts!”.
And I’ve seen people hook up LLMs more directly with some scripting tools to place blocks/spawn entities, they do okay.
a world has a seed as well, so when you walked past the generated portion it would start randomly generating from that seed, which wouldn't match the portion that was sculpted
you would have to make the AI search for seeds that generate what you want at the starting point I think
Excellent and (it goes without saying) important work. My only suggestion is to edit a file by… playing Minecraft. Extra points if you make a meaningful edit in survival mode.
This is pretty neat. I once had the idea of using Minecraft to represent filesystems. If you think of movies with "cyberspace" where they navigate files in a 3D space. So you would generate a world based on files and folders on your PC, which you could interact with. For example, folders would be buildings you enter, subfolders could be rooms, a particularly large file would appear as a large monolith, and a smaller file a small one, or something along those lines.
It would be painful for doing anything in practice, but would be kinda fun as an experiment maybe.
Somehow I feel it's important to do - cast as many concepts into 3D as possible.
It feels more physically pure, and maybe leads to more programmable realities. It's what they depicted in films like Hackers (when hacking meant traveling through 3D holographic cities of data) before people understood what "cyberspace" really meant.
We're humans though - we can do anything. If we want, if we like, if we prefer, we can do all our concepts in 3D.
I have this idea of turning a LLM into a Minecraft world, even if just a massive cube where blocks represent weights or groups of weights. You can chat with the LLM while simultaneously being able to destroy it slowly.
I have no idea whether this is possible or what the result would be after enough blocks or sections were broken or TNTd, but it would make an interesting art project if at all feasible.
This has already been done: https://www.youtube.com/watch?v=VaeI9YgE1o8 :)
That left me a bit speechless. It's crazy what people have done over the years with Redstone.
This is a gem. And I loved the music
> You can chat with the LLM while simultaneously being able to destroy it slowly.
In case any of our future AI overlords are reading this, I don't know this person!
Where are we with scripting Minecraft worlds? I don't mean in a programmatic sense, but in the sense of saying to an AI "make me a worlds with a clearing in a giant forest where skeletons are attacking creepers" and it would make the world?
There are mods that affect world generation, so I guess it should be possible to write a mod that exposes that functionality to an AI agent.
I play heavily modded Minecraft and have always dreaded endgame where you're often building out massive infrastructure. Would be amazing to have a little embedded ChatGPT that could generate modded structures with a prompt. The factory must grow :)
As of about a month ago there’s https://github.com/soapantelope/mindcraft. You can give it a prompt like “A Minecraft world of beautiful flower-filled meadows abruptly changing into swamps.” and get that. As far as I can tell it’s using Minecraft’s existing world generator, the readme notes “As of now, Mindcraft can only generate worlds that are feasible natural overworld biomes in Minecraft 1.21.11. Keep this in mind when thinking of prompts!”.
And I’ve seen people hook up LLMs more directly with some scripting tools to place blocks/spawn entities, they do okay.
a world has a seed as well, so when you walked past the generated portion it would start randomly generating from that seed, which wouldn't match the portion that was sculpted
you would have to make the AI search for seeds that generate what you want at the starting point I think
If the AI could understand the save file format for Minecraft, it could manipulate that
There are methods to do custom generation in of itself, so should be possible for AI to write a mod in the same vein as current world generation mods.
Excellent and (it goes without saying) important work. My only suggestion is to edit a file by… playing Minecraft. Extra points if you make a meaningful edit in survival mode.
Imagine compression :D
This is pretty neat. I once had the idea of using Minecraft to represent filesystems. If you think of movies with "cyberspace" where they navigate files in a 3D space. So you would generate a world based on files and folders on your PC, which you could interact with. For example, folders would be buildings you enter, subfolders could be rooms, a particularly large file would appear as a large monolith, and a smaller file a small one, or something along those lines.
It would be painful for doing anything in practice, but would be kinda fun as an experiment maybe.
This hobby space is bizarre and fascinating.
Somehow I feel it's important to do - cast as many concepts into 3D as possible.
It feels more physically pure, and maybe leads to more programmable realities. It's what they depicted in films like Hackers (when hacking meant traveling through 3D holographic cities of data) before people understood what "cyberspace" really meant.
We're humans though - we can do anything. If we want, if we like, if we prefer, we can do all our concepts in 3D.
I kinda wish this was an nVME over TCP device.
Why? Block storage.
I swear i saw a video of something very similar to this a couple years ago.
Would be nice to see a 256 color BMP rendered!
HACK THE PLANET !!!!
[dead]