Marvin Minsky famously published a book in 1986 called "Society of Mind", in which he argued that people don't really have a "mind", but rather an enormous assortment of specialized subsystems that interoperate to produce the illusion of coherent consciousness. I think that this belief is what's needed to guide us forward towards AGI. IMHO, no further technological breakthroughs are necessary to achieve AGI; the remaining work is subsystem integration. I think that Daniel is hitting the nail on the head here with respect to the direction we should all be working in. After all, there already exists a machine capable of attaining human-level intelligence: the human brain. We know what the functions of many of its components are, and we know that those functions must have evolved for darn good reasons. Mimicking its architecture seems like a logical extension of Rosenblatt's original work of building artificial neural networks in the first place, sixty-odd years ago.
According to a debate I had with Gemini (rofl what an "authoritative statement"), it claimed that LLMs can't get to AGI for three reasons. First, that it has no desire for self-preservation, to which I responded with self-preservation / non-corruption of itself. Arguably enough. Second, that it has no intrinsic motivation, to which I said it can seek to minimize entropy / maximize information gain of the world around us (min/maxing some function built on Information theory) and it relented. But what I can't solve for...LLMs can't learn like people. It is not a 20W piece of wetware running in the real world able to integrate and learn in real-time. It's gotta be batched trained then reinforced. Maybe this is irrelevant if the machine can pretend to be us sufficiently well, but I feel that, in truth, it's not a true AGI until we find a better algo / hardware substrate.
Oh...wait a minute...organoids are hitting the market :D
First, I want to support the same ontology and logic as Karpathy's LLM Wiki, but shifted towards data ownership and safety/consistency. I built in support for local LLMs for low cost / privacy, and use audit hooks + AST validation before writes for safety. Also the caching + indexing is local, and when coupled with anacron for repeated tasks and the Anterior Cingulate Cortex (blocking bad ops, inconsistencies, and infinite loop prevent), this will clean and monitor the wiki without needing to engage a cloud LLM or overspending if you do.
You don't have to switch - just give CoreTex your vault and see how it works on it. Keep in mind we are pre-alpha so it will be wonky.
1. Compacting, cleaning, and truncating terminal outputs in the somatosensory transducer (thanks TokenJuice for the idea)
2. Working memory buffer is capped at 45k characters, but when it starts to exceed, it will chunk the stalest memories, compress then, them reinject them to the context. No amnesia needed.
3. Context is pinned to top of requests for large tool calls.
4. Passing between agents / tools is optimized to give directions for reading files, as well providing a topology search, so whole files don't have to be injected into context.
5. The memory system engages during sleep and dreaming, rotating logs and extracting anything important from them into memory specific domains (md files per domain).
I'll be continually improving this and would love ideas / criticism!
Congrats on the launch, Daniel! Really cool approach. The flat-file architecture and sandboxed execution by default are smart design choices. The zero-token engram replay idea is clever too. Starred, excited to see where this goes.
Thanks dude. When I stopped worrying about scaling past one person or trying to start a business, and just focused on what I needed to organize and enhance myself, everything just sort of followed...naturally :)
I’ve spent the last ~30 days and 364 commits getting to the pre-alpha of CoreTex, an open source, Unix-inspired, biomimetic, text-only, portable, local, AI harness and knowledge engine. Breaking that down, my guiding principles and purpose were:
Purpose: I built CoreTex out of my personal need to organize my thoughts to be a better father, friend, and self-sufficient person. What began as the typical Obsidian-second-brain implementation quickly evolved into a quest to build a 1:1 analog of the brain itself. My hope is that this system will (eventually) help us organize our lives, achieve our goals, and build a community to change what’s happening to our world by working together. Maybe that’s simultaneously grandiose and naive, but I believe that machine intelligence can and should uplift all of us.
Principles:
- UNIX Philosophy: Flat files only like it’s the 1970s! State, IPC, data, queues, are files with concurrency guarded by OS-level locks. No databases here except an ephemeral SQLite FTS5 DB for performance. The payoff is threefold: The user and the AI can know all and see all (including its own source and state). CoreTex works with operators, redirect, pipe, etc. making it highly composable. Development velocity - very few things get in the way of adding features.
- Biomimesis: Why reinvent what nature already optimized? I followed evolutionary pathways to build the 39 biomimetic modules of CoreTex. Even the sensory organs (Sense) are decoupled as their own package and connected through the “Spine” which acts as a HAL. The payoff:
1) Procedural “Muscle Memory” (experimental) - the cerebellum creates engrams of repeated code, e.g. setting up a FastAPI + React app, so you can 0-cost repeat it later. It’s DRY for $. Engrams will be sharable over the exocortex via MCP/ACP.
2) “Self-healing” modules like the microglia intercept CLI errors at low cost.
3) A 5-tier memory system: Working Memory (token aware compression), Short Term Memory (FTS5 + BM25 ranking + snippet truncation), Relation Knowledge (maps dependencies to a serialized graph, protected by the ACC circuit breaker), Episodic Memory (thread-safe WAL JSONL ledger of actions), and Long Term Memory (rotates logs during sleep to extract key memories for alignment).
- Shift Left & Token Economics: Why don’t we shift left ALL THE THINGS? If we’re going to use this to help survive the Age of AI, we’re going to need 0-cost whenever possible. Local LLM support. Enforceable daily token budgets. Prompt caching, env guillotines, sliding context windows, transducers and tool truncation.
- Performance: No vectors in my lookups! FTS5 + BM25 + Ripgrep for text search, semantic compression of large files, Content Addressable Storage against a rapid hashing ledger for O(1) lookups, and deterministic transduction of CLI outputs at zero cost.
- Security: Safe by default, operating only in Cognitive mode - you have to turn on code gen. I resisted containers/microvms at first and attempted to make the most secure sandbox on the host OS possible. I realized my hubris after spending hours building a parent Watchdog daemon, switched to Deno + WASM, and eventually caved to support ephemeral Docker containers. Example - CoreTex’s Defense against Mini Shai Hulud: Code execution is set to false by default. When enabled, it runs in Pyodide and Deno where only secured CDNs are whitelisted over port 443. Dynamic path proxies prevent traversing unsafe paths or symlinks. (Where CoreTex loses: DNS exfiltration, because Deno hands DNS resolution back to the host OS where a worm could use subdomains).
The Current State: It will be rough using the pre-alpha build. Expect it to be “allergic” and “fearful” of your commands (thanks amygdala!). Expect to tweak agents.yaml. Expect to tweak your tasks. Expect a surprise token cost (<20k) for a simple task or API key issues. It may not even install properly. But use ./ctx daydream and tell me that it’s not something special to see a program suggest improving its own daemon? Or when I asked it to take my notes and make a GTM + website (autopoiesis) that it gave me an output that was a solid B? I feel like the bones of something are here.
Quo Vadis: I intend for CoreTex to be the freest, cheapest, and safest personal control plane + always on daemon, interoperable with every major AI system (Pi, Hermes, OpenClaw, OpenHuman, OpenWhatever) coupled with a robust ecosystem. That includes allowing CoreTex to connect to other instances (via the exoreceptor and exocortex modules) to share knowledge, code, memories, and compute in a gift economy. Ultimately, I want to give everyone a personal and free system that is totally aligned to your goals and your self, allowing you to observe and react to the world even as you sleep.
So with all that said, rip it to shreds! Every time I’ve gone through the crucible with my friends, CoreTex has come back stronger. Life…finds a way (even if it’s artificial).
P.S. I developed this on my Windows machine by arguing and co-creating every module, one at a time, with Gemini using my 100 Pro-Prompts-a-Day budget. Maximum portability. Minimal dependencies. No databases. And, most importantly, No items, Fox only, Final Destination.
Developing software w/ AI, It can be all to tempting to spend my time and resources on the AI system and tools itself, rather than the work we could be pointing it at.
What are you most excited / looking forward to using CoreTex for? Is there an immediate use case to try with it, that you think its uniquely capable for?
Honestly, the thing I'm most excited for is building video games, figuring out how to help myself and other folks find the right kind of work (especially our own), and seeing how the rest of the parts of human neuroanatomy fit into CoreTex (like the angular gyrus for enhanced multimodal meaning). Making the analog of the mind was the most fun part, and I learned so much from it! Now, when I look at my daugher, I understand why her mirror neurons work the way they do, and how her Broca's area is forming during the period of hyper-plasticity. Makes me cry.
As for immediate use cases - I think the most powerful thing right now is just the ability to play with the config with system and agents. You can easily compose with them, give it a prompt, no lock in to any single provider. And the review gates are nice. Code gen is still experimental and will be finicky.
I think what CoreTex is uniquely poised for is being the most lightweight and easily extended "control center" of various AI systems. That's why I doubled down on UNIX philosophy, inherent security, and minimal dependencies for maximal portability. I want this thing to work on any system, take a bullet, and still stay up to help you control AI + complete your goals.
I like what you said about 'take a bullet' and 'still stay up'
Real brains don't run on respond cycles. Do you see this getting a heartbeat system or is there anything implemented / planned to make it a persistent daemon?
like it could check different surfaces for deviation for useful discoveries, or problems that need to be remediated, or even browse Hackernews for exciting new tech to learn about and grow!
It is currently a daemon with the /.ctx live command. That will turn on the watchdog process, the webhook listener, the file system listener, and all the other passive systems.
One of the more interesting features I want to implement is the "distributed spine." I want CoreTex to live on multiple machines, offloading particular parts of computation to say - the gaming computer with the GPU running higher local models, sensory data being routed from embedded systems, and a low-power alternative for it to sync to. When the gaming computer turns off, the low power unit will promote itself to primary CoreTex for overnight watching / sleeping / dreaming, then revoke control back to the primary computer when it turns back on. In this sense, CoreTex will be a distributed system, increasing its stability and uptime.
You should be able to use this for anything, really, given its UNIX philosophy and OSS.
I can say that many things are not yet fully wired up - HOWEVER - the true gain of function I see is in the following areas:
- The Anterior Cingulate Cortex - I love enforcing consistency on memories. This is one tool in the toolbox of attacking semantic drift.
- The cerebellum: Even if it's not working well, the concept of turning previously executed code generated by an LLM into 0-cost reusable asset (which can be shared) is an important part of token economics
- The corpus collosum: the idea of right/left brain divide mapping to local LLMs vs cloud LLMs is an important cost saving and air gapping mechanism. We can have a local LLM watch dog off this with relative ease.
- The Blood Brain Barrier, Thymus, and Immune System - Doing AST scanning and using Audit hooks, combined with an efferent secret vault and a watchdog process on CoreTex are things we should have more of.
- Enteric System and Basal Ganglia - even though the BG isn't really working great yet, the idea that this system can repeat commands at 0-analysis-cost and then turn repeated actions into acron jobs based on changes to the file system (to avoid dependency on the host os) is another great quality of life feature.
- Vagus Nerve and Interoception - Creating a global token budget and aborting if things get too crazy is a good idea. I want the system to actively protect people from wasting money or denial of wallet attacks.
Those are a few! For more, you can see the biomimesis in docs. I could not have built this without following biology.
Marvin Minsky famously published a book in 1986 called "Society of Mind", in which he argued that people don't really have a "mind", but rather an enormous assortment of specialized subsystems that interoperate to produce the illusion of coherent consciousness. I think that this belief is what's needed to guide us forward towards AGI. IMHO, no further technological breakthroughs are necessary to achieve AGI; the remaining work is subsystem integration. I think that Daniel is hitting the nail on the head here with respect to the direction we should all be working in. After all, there already exists a machine capable of attaining human-level intelligence: the human brain. We know what the functions of many of its components are, and we know that those functions must have evolved for darn good reasons. Mimicking its architecture seems like a logical extension of Rosenblatt's original work of building artificial neural networks in the first place, sixty-odd years ago.
According to a debate I had with Gemini (rofl what an "authoritative statement"), it claimed that LLMs can't get to AGI for three reasons. First, that it has no desire for self-preservation, to which I responded with self-preservation / non-corruption of itself. Arguably enough. Second, that it has no intrinsic motivation, to which I said it can seek to minimize entropy / maximize information gain of the world around us (min/maxing some function built on Information theory) and it relented. But what I can't solve for...LLMs can't learn like people. It is not a 20W piece of wetware running in the real world able to integrate and learn in real-time. It's gotta be batched trained then reinforced. Maybe this is irrelevant if the machine can pretend to be us sufficiently well, but I feel that, in truth, it's not a true AGI until we find a better algo / hardware substrate.
Oh...wait a minute...organoids are hitting the market :D
Cannot wait to try it. What do you say to those who are leveraging 2nd brains like Karpathy’s LLM Wiki? What are the potential benefits to switching?
First, I want to support the same ontology and logic as Karpathy's LLM Wiki, but shifted towards data ownership and safety/consistency. I built in support for local LLMs for low cost / privacy, and use audit hooks + AST validation before writes for safety. Also the caching + indexing is local, and when coupled with anacron for repeated tasks and the Anterior Cingulate Cortex (blocking bad ops, inconsistencies, and infinite loop prevent), this will clean and monitor the wiki without needing to engage a cloud LLM or overspending if you do.
You don't have to switch - just give CoreTex your vault and see how it works on it. Keep in mind we are pre-alpha so it will be wonky.
What does CoreTex do currently to handle/prevent context degradation?
Here's a few things it does:
1. Compacting, cleaning, and truncating terminal outputs in the somatosensory transducer (thanks TokenJuice for the idea) 2. Working memory buffer is capped at 45k characters, but when it starts to exceed, it will chunk the stalest memories, compress then, them reinject them to the context. No amnesia needed. 3. Context is pinned to top of requests for large tool calls. 4. Passing between agents / tools is optimized to give directions for reading files, as well providing a topology search, so whole files don't have to be injected into context. 5. The memory system engages during sleep and dreaming, rotating logs and extracting anything important from them into memory specific domains (md files per domain).
I'll be continually improving this and would love ideas / criticism!
Congrats on the launch, Daniel! Really cool approach. The flat-file architecture and sandboxed execution by default are smart design choices. The zero-token engram replay idea is clever too. Starred, excited to see where this goes.
Thanks dude. When I stopped worrying about scaling past one person or trying to start a business, and just focused on what I needed to organize and enhance myself, everything just sort of followed...naturally :)
Hello HN! Decades long lurker, first time posting and here to hock my OSS project - https://github.com/mrdanielcasper/coretex.
I’ve spent the last ~30 days and 364 commits getting to the pre-alpha of CoreTex, an open source, Unix-inspired, biomimetic, text-only, portable, local, AI harness and knowledge engine. Breaking that down, my guiding principles and purpose were:
Purpose: I built CoreTex out of my personal need to organize my thoughts to be a better father, friend, and self-sufficient person. What began as the typical Obsidian-second-brain implementation quickly evolved into a quest to build a 1:1 analog of the brain itself. My hope is that this system will (eventually) help us organize our lives, achieve our goals, and build a community to change what’s happening to our world by working together. Maybe that’s simultaneously grandiose and naive, but I believe that machine intelligence can and should uplift all of us.
Principles:
- UNIX Philosophy: Flat files only like it’s the 1970s! State, IPC, data, queues, are files with concurrency guarded by OS-level locks. No databases here except an ephemeral SQLite FTS5 DB for performance. The payoff is threefold: The user and the AI can know all and see all (including its own source and state). CoreTex works with operators, redirect, pipe, etc. making it highly composable. Development velocity - very few things get in the way of adding features.
- Biomimesis: Why reinvent what nature already optimized? I followed evolutionary pathways to build the 39 biomimetic modules of CoreTex. Even the sensory organs (Sense) are decoupled as their own package and connected through the “Spine” which acts as a HAL. The payoff: 1) Procedural “Muscle Memory” (experimental) - the cerebellum creates engrams of repeated code, e.g. setting up a FastAPI + React app, so you can 0-cost repeat it later. It’s DRY for $. Engrams will be sharable over the exocortex via MCP/ACP. 2) “Self-healing” modules like the microglia intercept CLI errors at low cost. 3) A 5-tier memory system: Working Memory (token aware compression), Short Term Memory (FTS5 + BM25 ranking + snippet truncation), Relation Knowledge (maps dependencies to a serialized graph, protected by the ACC circuit breaker), Episodic Memory (thread-safe WAL JSONL ledger of actions), and Long Term Memory (rotates logs during sleep to extract key memories for alignment).
- Shift Left & Token Economics: Why don’t we shift left ALL THE THINGS? If we’re going to use this to help survive the Age of AI, we’re going to need 0-cost whenever possible. Local LLM support. Enforceable daily token budgets. Prompt caching, env guillotines, sliding context windows, transducers and tool truncation.
- Performance: No vectors in my lookups! FTS5 + BM25 + Ripgrep for text search, semantic compression of large files, Content Addressable Storage against a rapid hashing ledger for O(1) lookups, and deterministic transduction of CLI outputs at zero cost.
- Security: Safe by default, operating only in Cognitive mode - you have to turn on code gen. I resisted containers/microvms at first and attempted to make the most secure sandbox on the host OS possible. I realized my hubris after spending hours building a parent Watchdog daemon, switched to Deno + WASM, and eventually caved to support ephemeral Docker containers. Example - CoreTex’s Defense against Mini Shai Hulud: Code execution is set to false by default. When enabled, it runs in Pyodide and Deno where only secured CDNs are whitelisted over port 443. Dynamic path proxies prevent traversing unsafe paths or symlinks. (Where CoreTex loses: DNS exfiltration, because Deno hands DNS resolution back to the host OS where a worm could use subdomains).
The Current State: It will be rough using the pre-alpha build. Expect it to be “allergic” and “fearful” of your commands (thanks amygdala!). Expect to tweak agents.yaml. Expect to tweak your tasks. Expect a surprise token cost (<20k) for a simple task or API key issues. It may not even install properly. But use ./ctx daydream and tell me that it’s not something special to see a program suggest improving its own daemon? Or when I asked it to take my notes and make a GTM + website (autopoiesis) that it gave me an output that was a solid B? I feel like the bones of something are here.
Quo Vadis: I intend for CoreTex to be the freest, cheapest, and safest personal control plane + always on daemon, interoperable with every major AI system (Pi, Hermes, OpenClaw, OpenHuman, OpenWhatever) coupled with a robust ecosystem. That includes allowing CoreTex to connect to other instances (via the exoreceptor and exocortex modules) to share knowledge, code, memories, and compute in a gift economy. Ultimately, I want to give everyone a personal and free system that is totally aligned to your goals and your self, allowing you to observe and react to the world even as you sleep.
So with all that said, rip it to shreds! Every time I’ve gone through the crucible with my friends, CoreTex has come back stronger. Life…finds a way (even if it’s artificial).
P.S. I developed this on my Windows machine by arguing and co-creating every module, one at a time, with Gemini using my 100 Pro-Prompts-a-Day budget. Maximum portability. Minimal dependencies. No databases. And, most importantly, No items, Fox only, Final Destination.
Developing software w/ AI, It can be all to tempting to spend my time and resources on the AI system and tools itself, rather than the work we could be pointing it at.
What are you most excited / looking forward to using CoreTex for? Is there an immediate use case to try with it, that you think its uniquely capable for?
Honestly, the thing I'm most excited for is building video games, figuring out how to help myself and other folks find the right kind of work (especially our own), and seeing how the rest of the parts of human neuroanatomy fit into CoreTex (like the angular gyrus for enhanced multimodal meaning). Making the analog of the mind was the most fun part, and I learned so much from it! Now, when I look at my daugher, I understand why her mirror neurons work the way they do, and how her Broca's area is forming during the period of hyper-plasticity. Makes me cry.
As for immediate use cases - I think the most powerful thing right now is just the ability to play with the config with system and agents. You can easily compose with them, give it a prompt, no lock in to any single provider. And the review gates are nice. Code gen is still experimental and will be finicky.
I think what CoreTex is uniquely poised for is being the most lightweight and easily extended "control center" of various AI systems. That's why I doubled down on UNIX philosophy, inherent security, and minimal dependencies for maximal portability. I want this thing to work on any system, take a bullet, and still stay up to help you control AI + complete your goals.
Thanks for your question!
I like what you said about 'take a bullet' and 'still stay up'
Real brains don't run on respond cycles. Do you see this getting a heartbeat system or is there anything implemented / planned to make it a persistent daemon?
like it could check different surfaces for deviation for useful discoveries, or problems that need to be remediated, or even browse Hackernews for exciting new tech to learn about and grow!
It is currently a daemon with the /.ctx live command. That will turn on the watchdog process, the webhook listener, the file system listener, and all the other passive systems.
One of the more interesting features I want to implement is the "distributed spine." I want CoreTex to live on multiple machines, offloading particular parts of computation to say - the gaming computer with the GPU running higher local models, sensory data being routed from embedded systems, and a low-power alternative for it to sync to. When the gaming computer turns off, the low power unit will promote itself to primary CoreTex for overnight watching / sleeping / dreaming, then revoke control back to the primary computer when it turns back on. In this sense, CoreTex will be a distributed system, increasing its stability and uptime.
You should be able to use this for anything, really, given its UNIX philosophy and OSS.
How much of the biomimetic structure is metaphor vs. architecture that materially changes behavior or performance?
I can say that many things are not yet fully wired up - HOWEVER - the true gain of function I see is in the following areas: - The Anterior Cingulate Cortex - I love enforcing consistency on memories. This is one tool in the toolbox of attacking semantic drift.
- The cerebellum: Even if it's not working well, the concept of turning previously executed code generated by an LLM into 0-cost reusable asset (which can be shared) is an important part of token economics
- The corpus collosum: the idea of right/left brain divide mapping to local LLMs vs cloud LLMs is an important cost saving and air gapping mechanism. We can have a local LLM watch dog off this with relative ease.
- The Blood Brain Barrier, Thymus, and Immune System - Doing AST scanning and using Audit hooks, combined with an efferent secret vault and a watchdog process on CoreTex are things we should have more of.
- Enteric System and Basal Ganglia - even though the BG isn't really working great yet, the idea that this system can repeat commands at 0-analysis-cost and then turn repeated actions into acron jobs based on changes to the file system (to avoid dependency on the host os) is another great quality of life feature.
- Vagus Nerve and Interoception - Creating a global token budget and aborting if things get too crazy is a good idea. I want the system to actively protect people from wasting money or denial of wallet attacks.
Those are a few! For more, you can see the biomimesis in docs. I could not have built this without following biology.