Can you description the deployment setup, somewhere in the docs/maybe with a diagram?
I get this is a backend library, which is great, but like does it use postgres replication slots? Per the inherited queries, do they all live on 1 machine, and we just assume that machine needs to be sufficiently beefy to serve all currently-live queries?
Do all of my (backend) live-queries live/run on that one beefy machine? What's the life cycle for live-queries? Like how can I deploy new ones / kill old ones / as I'm making deployments / business logic changes that might change the queries?
This is all really hard ofc, so apologies for all the questions, just trying to understand -- thanks!
Great questions — happy to clarify how deployment and lifecycle work today.
Let me begin by answering: what exactly is this engine? It's simply a computation + cache layer that lives in the same process as the calling code, not a server on its own.
Think of a LinkedQL instance (new PGClient()) and its concept of a "Live Query" engine as simply a query client (e.g. new pg.Client()) with an in-memory compute + cache layer.
---
1. Deployment model (current state)
The Live Query engine runs as part of your application process — the same place you’d normally run a Postgres/MySQL client.
For Postgres, yes: it uses one logical replication slot per LinkedQL engine instance. The live query engine instantiates on top of that slot and uses internal "windows" to dedupe overlapping queries, so 500 queries that are only variations of "SELECT * FROM users" still map to one main window; and 500 of such "windows" still run over the same replication slot.
As hinted at above, yes; each LinkedQL instance (new PGClient()) runs on the same machine as the running app (just as you'd have it with new pg.Client()) – and maps to a single Live Query engine under the hood.
That engine uses a single replication slot. You specify the slot name like:
new PGClient({ ..., walSlotName: 'custom_slot_name' }); // default is: "linkedql_default_slot" – as per https://linked-ql.netlify.app/docs/setup#postgresql
A second LinkedQL instance would require another slot name:
new PGClient({ ..., walSlotName: 'custom_slot_name_2' });
We’re working toward multi-instance coordination (multiple engines sharing the same replication stream + load balancing live queries). That’s planned, but not started yet.
---
3. Lifecycle of live queries
The Live Query engine runs on-demand and not indefinitely. It begins to exist when at least one client subscribes ({ live: true }) and effectively cleans up and disappears the moment the last subscriber disconnects (result.abort()). Calling client.disconnect() also ends all subscriptions and does clean up.
---
4. Deployments / code changes
Deploying new code doesn’t require “migrating” live queries.
When you restart the application:
• the Live Query starts on a clean slate with the first subscribing query (client.query('...', { live: true })).
• if you have provided a persistent replication slot name (the default being ephemeral), LinkedQL moves the position to the slot's current position and runs from there.
In other words: nothing persists across deploys; everything starts clean as your app starts.
---
5. Diagram / docs
A deployment diagram is a good idea — I’ll add one to the docs.
---
Well, I hope that helps — and no worries about the questions. This space is hard, and happy to explain anything in more detail.
As a potential user, I'd probably be thinking through things like: if I have a ~small-fleet of 10 ECS tasks serving my REST/API endpoints, would I run `client.query`s on these same machines, or would it be better to have a dedicated pool of "live query" machines that are separate from most API serving, so that maybe I get more overlap of inherited queries.
...also I think there is a limit on WAL slots? Or at least I'd probably want not each of my API servers to be consuming their own WAL slots.
Totally makes sense this is all "things you worry about later" (where later might be now-/soon-ish) given the infra/core concepts you've got working now -- looking really amazing!
Thanks — this is a really good scenario to walk through and I'm happy to extend the conversation.
First, I’m implicitly assuming your 10 ECS tasks are talking to the same Postgres instance and may issue overlapping queries. Once that’s the case, WAL slots and backend orchestration naturally enter the story — not just querying.
A few concrete facts first.
PostgreSQL does have a logical replication slots cap controlled via max_replication_slots.
Each LinkedQL Live Query engine instance uses one slot.
Now, as to whether or not "10 instances" become a problem depends entirely on your Postgres config and workload specifics. I'd want to believe that 10 is perfectly fine; but also, maybe not. "It depends ".
---
That said, if you do want strong deduplication across services, the pattern I’d recommend is centralizing queries in a separate service.
One service owns the LinkedQL engine and the replication slot. Other backend services query that service instead of Postgres directly.
Conceptually:
[API services] → [Live Query service (LinkedQL)] → Postgres
From the caller’s point of view this feels like REST (e.g. GET /users?...). But it doesn't have to be "just" REST.
If you happen to run a NodeJS backend or would be willing to, then your entire backend orchestration can get more interesting. We built a backend framework that supports cross-runtime reactivity, and it perfectly fits your use case: piping LinkedQL's "live" results across the wire to a consumer.
Here's an overview:
In your query-hosting service, you'd install Webflo as your backend framework, define routes by exposing request-handling functions, and have these functions simply return LinkedQL's live result rows as-is:
// the root "/" handler
export default async function(event, next) {
if (next.stepname) return next();
const q = event.url.q;
const liveResult = await client.query(q, {
live: true,
signal: event.signal
});
// Send the initial rows and keep the request open
event.respondWith(liveResult.rows, { done: false }); // same as "return new LiveResponse(liveResult.rows, { done: false });"
}
Here, the handler starts a live query and returns the live result rows issued by LinkedQL and returns a "live" response.
- the client immediately receives the initial query result
- the HTTP connection stays open
- any mutations to the sent object are synced automatically over the wire and the client-side copy continues to behave as a live object
- if the client disconnects, event.signal is aborted and the live query shuts down
// Closing the connection kills the live query upstream
liveResponse.background.close();
The resulting backend orchestration becomes:
- no separate “realtime API” / WebSocket setup. The lifetime of the live query is simply the lifetime of the request connection
- live queries as if running on a single machine
In this setup:
- WAL consumption stays bounded.
- Live queries are deduped centrally.
- API services remain stateless.
- Lifecycle is automatic, not managed.
I haven’t personally run this exact topology at scale yet, but it fits the model cleanly and is very much the direction the architecture is designed to support.
Once you use Webflo, this stops feeling like “realtime plumbing” and starts feeling like normal request/response — just with live mode.
The relationship between Webflo and LinkedQL here is that:
- Live Queries in LinkedQL yield live objects (LiveResult.rows).
- Webflo has the concept of "live" responses which allows you to send arbitrary objects as responses – with subsequent mutations on those objects synced over the wire to produce equivalent mutations on the client side (the caller) – without a stream surface you maintain.
Would you clarify what a "transaction" in this instance would mean?
LinkedQL definitely optimizes at multiple levels between a change happening on your database and the live result your application sees. The most significant of these being its concept of query windows and query inheritance which ensure multiple overlapping queries converge on a single "actual" query window under the hood.
- How does authz work? Can I use Postgres RLS? If not, how would you address row or column-level permissions in a system that uses this?
- If you're using logical replication to sync with PG, is there a limit to the number of clients you can have connected? I see there is a lot of work around de-duping live queries, but how well does that work in practice?
- Any thought to making an extension for Postgres? My main hesitation right now is that I have to go through an NPM package to use this but a lot of our tooling expects a plain Postgres connection.
- REALLY looking forward to seeing how the schema migration story looks.
Overall, it seems to address most of the use-cases where I'd reach for an ORM or API server so I'm really interested to see where this could go.
Thanks for reading through and for these questions. I'll take them in their order:
---
Auth / RLS
Yes — LinkedQL works with Postgres Row-Level Security. Each LinkedQL connection is equivalent to a regular DB connection (e.g., new LinkedQLClient(connectionInfo) is like new pg.Client(connectionInfo)). There’s no new permission model to maintain — the DB remains the enforcement point.
Live queries always execute under the same authenticated role you provided, so RLS policies apply on every refresh or incremental update. LinkedQL never uses a “superuser” backend that could widen visibility.
--
Replication limits & scaling
Right now, each database connection supports one logical replication slot. LinkedQL dedupes overlapping live queries on top of it — so 1,000 clients watching the same underlying SELECT only cost the DB one change stream.
We plan to support a distributed architecture as well — multiple instances of the live query engine coordinating load for high-traffic deployments.
---
Why an npm package (and future extension)
Right now LinkedQL plugs directly into JavaScript apps, matching how many teams already query Postgres from frontend or backend code.
We definitely have a Postgres extension in the roadmap for your exact use case – tighter operational integration.
---
Schema migration story
This is also one I’m personally excited about. We previously had an automatic schema versioning layer in the earlier LinkedQL prototype:
The goal in the current version is a cleaner rewrite of that whole feature. So, migration support is returning – with everything we learned in the previous baked in.
For example, while the previous implementation of the diff-based migration feature spoke JSON for schema declarations, we plan to let that be pure SQL – yet, diff-based.
---
Thanks again for the thoughtful look!
We can zoom into any other area of your choice.
Author here — a bit more detail on architecture and guarantees
Happy to dig into internals if anyone’s curious — how live updates propagate, how JOINs and complex queries resolve, consistency expectations, worst-case scaling, etc.
To keep the main post short, here are deep-dive links if you want to explore:
Your docs say live queries for MySQL and MariaDB are "coming soon", but your post here strongly suggests they're already supported. Is this actually implemented yet or not?
Thanks for spotting that — to clarify: the current production implementation of Live Queries is Postgres only.
MySQL/MariaDB support is in progress (binlog-based) and is why the docs say “coming soon.”
The post wasn’t meant to imply that MySQL/MariaDB are already live; the intention was to describe the overall design rather than claim full parity. I’ll update the wording to avoid that confusion.
Short answer: the core LinkedQL live query engine runs on the backend today, and there’s an embeddable variant (FlashQL) that runs directly in the frontend with the same LinkedQL capabilities – live queries, DeepRefs, etc.
1. Pure frontend / local data
For data that can live entirely on the client, you can spin up an in-browser FlashQL instance:
const client = new FlashQL(); // runs in the page / worker
await client.query(`
CREATE TABLE users (
id UUID PRIMARY KEY,
name TEXT
)
`);
// Live query works the same way as on the backend:
const result = await client.query(
'SELECT * FROM users',
{ live: true }
);
From there, result is a live result set: inserts/updates/deletes that match the query will show up in the rows, and all the same features (live queries, DeepRefs, etc.) behave as they do on a backend instance.
At the moment FlashQL is in-memory only; persistence backends like IndexedDB / LocalStorage are on the roadmap.
2. Remote database from the frontend
If your source of truth is a remote Postgres/MySQL instance, the model we’re building is:
a LinkedQL engine next to the database, and
a FlashQL instance in the frontend that federates/syncs with that backend engine.
Ah yes — good catch. The commit history definitely isn’t following Conventional Commits right now. Things got a bit loose during fast iterations, but I’ll follow the convention going forward.
Author here.
Sad to hear that you perceive the docs and comments here as LLM generated.
I'm genuinely curious what in particular gives you that impression.
I'm sorry, this comment [0] was clearly written (or rewritten) by an LLM. Sections are titled, constant m dash usages, and "Thanks again for the thoughtful look! We can zoom into any other area of your choice."
Can you description the deployment setup, somewhere in the docs/maybe with a diagram?
I get this is a backend library, which is great, but like does it use postgres replication slots? Per the inherited queries, do they all live on 1 machine, and we just assume that machine needs to be sufficiently beefy to serve all currently-live queries?
Do all of my (backend) live-queries live/run on that one beefy machine? What's the life cycle for live-queries? Like how can I deploy new ones / kill old ones / as I'm making deployments / business logic changes that might change the queries?
This is all really hard ofc, so apologies for all the questions, just trying to understand -- thanks!
Great questions — happy to clarify how deployment and lifecycle work today.
Let me begin by answering: what exactly is this engine? It's simply a computation + cache layer that lives in the same process as the calling code, not a server on its own.
Think of a LinkedQL instance (new PGClient()) and its concept of a "Live Query" engine as simply a query client (e.g. new pg.Client()) with an in-memory compute + cache layer.
---
1. Deployment model (current state)
The Live Query engine runs as part of your application process — the same place you’d normally run a Postgres/MySQL client.
For Postgres, yes: it uses one logical replication slot per LinkedQL engine instance. The live query engine instantiates on top of that slot and uses internal "windows" to dedupe overlapping queries, so 500 queries that are only variations of "SELECT * FROM users" still map to one main window; and 500 of such "windows" still run over the same replication slot.
The concept of query windows and the LinkedQL inheritance model is fully covered here: https://linked-ql.netlify.app/engineering/realtime-engine
---
2. Do all live queries “live” on one machine?
As hinted at above, yes; each LinkedQL instance (new PGClient()) runs on the same machine as the running app (just as you'd have it with new pg.Client()) – and maps to a single Live Query engine under the hood.
We’re working toward multi-instance coordination (multiple engines sharing the same replication stream + load balancing live queries). That’s planned, but not started yet.---
3. Lifecycle of live queries
The Live Query engine runs on-demand and not indefinitely. It begins to exist when at least one client subscribes ({ live: true }) and effectively cleans up and disappears the moment the last subscriber disconnects (result.abort()). Calling client.disconnect() also ends all subscriptions and does clean up.
---
4. Deployments / code changes
Deploying new code doesn’t require “migrating” live queries.
When you restart the application:
• the Live Query starts on a clean slate with the first subscribing query (client.query('...', { live: true })).
• if you have provided a persistent replication slot name (the default being ephemeral), LinkedQL moves the position to the slot's current position and runs from there.
In other words: nothing persists across deploys; everything starts clean as your app starts.
---
5. Diagram / docs
A deployment diagram is a good idea — I’ll add one to the docs.
---
Well, I hope that helps — and no worries about the questions. This space is hard, and happy to explain anything in more detail.
Thanks for the reply! That all makes sense!
As a potential user, I'd probably be thinking through things like: if I have a ~small-fleet of 10 ECS tasks serving my REST/API endpoints, would I run `client.query`s on these same machines, or would it be better to have a dedicated pool of "live query" machines that are separate from most API serving, so that maybe I get more overlap of inherited queries.
...also I think there is a limit on WAL slots? Or at least I'd probably want not each of my API servers to be consuming their own WAL slots.
Totally makes sense this is all "things you worry about later" (where later might be now-/soon-ish) given the infra/core concepts you've got working now -- looking really amazing!
Thanks — this is a really good scenario to walk through and I'm happy to extend the conversation.
First, I’m implicitly assuming your 10 ECS tasks are talking to the same Postgres instance and may issue overlapping queries. Once that’s the case, WAL slots and backend orchestration naturally enter the story — not just querying.
A few concrete facts first.
PostgreSQL does have a logical replication slots cap controlled via max_replication_slots. Each LinkedQL Live Query engine instance uses one slot. Now, as to whether or not "10 instances" become a problem depends entirely on your Postgres config and workload specifics. I'd want to believe that 10 is perfectly fine; but also, maybe not. "It depends ".
---
That said, if you do want strong deduplication across services, the pattern I’d recommend is centralizing queries in a separate service.
One service owns the LinkedQL engine and the replication slot. Other backend services query that service instead of Postgres directly.
Conceptually:
[API services] → [Live Query service (LinkedQL)] → Postgres
From the caller’s point of view this feels like REST (e.g. GET /users?...). But it doesn't have to be "just" REST.
If you happen to run a NodeJS backend or would be willing to, then your entire backend orchestration can get more interesting. We built a backend framework that supports cross-runtime reactivity, and it perfectly fits your use case: piping LinkedQL's "live" results across the wire to a consumer.
Here's an overview:
In your query-hosting service, you'd install Webflo as your backend framework, define routes by exposing request-handling functions, and have these functions simply return LinkedQL's live result rows as-is:
// the root "/" handler
export default async function(event, next) { if (next.stepname) return next();
}Here, the handler starts a live query and returns the live result rows issued by LinkedQL and returns a "live" response.
- the client immediately receives the initial query result
- the HTTP connection stays open
- any mutations to the sent object are synced automatically over the wire and the client-side copy continues to behave as a live object
- if the client disconnects, event.signal is aborted and the live query shuts down
On the client side, you'd do:
const response = await fetch('db-service/users?q=...');
const liveResponse = await LiveResponse.from(response);
// This is a normal JS array as per what we sent from the upstream… but is a live object – the original
console.log(liveResponse.body);
// …can be observed too
Observer.observe(liveResponse.body, mutations => { console.log(mutations); });
// Closing the connection kills the live query upstream
liveResponse.background.close();
The resulting backend orchestration becomes:
- no separate “realtime API” / WebSocket setup. The lifetime of the live query is simply the lifetime of the request connection
- live queries as if running on a single machine
In this setup:
- WAL consumption stays bounded.
- Live queries are deduped centrally.
- API services remain stateless.
- Lifecycle is automatic, not managed.
I haven’t personally run this exact topology at scale yet, but it fits the model cleanly and is very much the direction the architecture is designed to support.
Once you use Webflo, this stops feeling like “realtime plumbing” and starts feeling like normal request/response — just with live mode.
---
More about Webflo: https://webflo.netlify.app/
Direct link to concepts: https://webflo.netlify.app/docs/concepts
Direct link to Live Responses: https://webflo.netlify.app/docs/concepts/realtime
The relationship between Webflo and LinkedQL here is that:
- Live Queries in LinkedQL yield live objects (LiveResult.rows).
- Webflo has the concept of "live" responses which allows you to send arbitrary objects as responses – with subsequent mutations on those objects synced over the wire to produce equivalent mutations on the client side (the caller) – without a stream surface you maintain.
- Live query results simply fall into this flow.
---
What you think.
Does it have an "optimization step" where it e.g. groups multiple queries into the same transactions and things of that nature?
Would you clarify what a "transaction" in this instance would mean?
LinkedQL definitely optimizes at multiple levels between a change happening on your database and the live result your application sees. The most significant of these being its concept of query windows and query inheritance which ensure multiple overlapping queries converge on a single "actual" query window under the hood.
You want to see the engineering paper for the full details: https://linked-ql.netlify.app/engineering/realtime-engine
A few questions/comments after skimming the docs:
- How does authz work? Can I use Postgres RLS? If not, how would you address row or column-level permissions in a system that uses this? - If you're using logical replication to sync with PG, is there a limit to the number of clients you can have connected? I see there is a lot of work around de-duping live queries, but how well does that work in practice? - Any thought to making an extension for Postgres? My main hesitation right now is that I have to go through an NPM package to use this but a lot of our tooling expects a plain Postgres connection. - REALLY looking forward to seeing how the schema migration story looks.
Overall, it seems to address most of the use-cases where I'd reach for an ORM or API server so I'm really interested to see where this could go.
Thanks for reading through and for these questions. I'll take them in their order:
---
Auth / RLS
Yes — LinkedQL works with Postgres Row-Level Security. Each LinkedQL connection is equivalent to a regular DB connection (e.g., new LinkedQLClient(connectionInfo) is like new pg.Client(connectionInfo)). There’s no new permission model to maintain — the DB remains the enforcement point.
Live queries always execute under the same authenticated role you provided, so RLS policies apply on every refresh or incremental update. LinkedQL never uses a “superuser” backend that could widen visibility.
--
Replication limits & scaling
Right now, each database connection supports one logical replication slot. LinkedQL dedupes overlapping live queries on top of it — so 1,000 clients watching the same underlying SELECT only cost the DB one change stream.
We plan to support a distributed architecture as well — multiple instances of the live query engine coordinating load for high-traffic deployments.
---
Why an npm package (and future extension)
Right now LinkedQL plugs directly into JavaScript apps, matching how many teams already query Postgres from frontend or backend code.
We definitely have a Postgres extension in the roadmap for your exact use case – tighter operational integration.
---
Schema migration story
This is also one I’m personally excited about. We previously had an automatic schema versioning layer in the earlier LinkedQL prototype:
https://github.com/linked-db/linked-ql/wiki/Automatic-Schema...
https://github.com/linked-db/linked-ql/wiki/Migrations
The goal in the current version is a cleaner rewrite of that whole feature. So, migration support is returning – with everything we learned in the previous baked in.
For example, while the previous implementation of the diff-based migration feature spoke JSON for schema declarations, we plan to let that be pure SQL – yet, diff-based.
---
Thanks again for the thoughtful look! We can zoom into any other area of your choice.
Author here — a bit more detail on architecture and guarantees
Happy to dig into internals if anyone’s curious — how live updates propagate, how JOINs and complex queries resolve, consistency expectations, worst-case scaling, etc.
To keep the main post short, here are deep-dive links if you want to explore:
• Live update mechanics https://linked-ql.netlify.app/capabilities/live-queries
• Engineering paper (replication pipelines, differential projection, query inheritance) https://linked-ql.netlify.app/engineering/realtime-engine
Totally open to questions — I’m hanging around the thread to learn what concerns matter most.
Your docs say live queries for MySQL and MariaDB are "coming soon", but your post here strongly suggests they're already supported. Is this actually implemented yet or not?
Thanks for spotting that — to clarify: the current production implementation of Live Queries is Postgres only.
MySQL/MariaDB support is in progress (binlog-based) and is why the docs say “coming soon.”
The post wasn’t meant to imply that MySQL/MariaDB are already live; the intention was to describe the overall design rather than claim full parity. I’ll update the wording to avoid that confusion.
This is an backend library? How to enable Live queries in the frontend?
Author here — thanks for checking it out.
Short answer: the core LinkedQL live query engine runs on the backend today, and there’s an embeddable variant (FlashQL) that runs directly in the frontend with the same LinkedQL capabilities – live queries, DeepRefs, etc.
1. Pure frontend / local data
For data that can live entirely on the client, you can spin up an in-browser FlashQL instance:
const client = new FlashQL(); // runs in the page / worker
await client.query(` CREATE TABLE users ( id UUID PRIMARY KEY, name TEXT ) `);
// Live query works the same way as on the backend: const result = await client.query( 'SELECT * FROM users', { live: true } );
From there, result is a live result set: inserts/updates/deletes that match the query will show up in the rows, and all the same features (live queries, DeepRefs, etc.) behave as they do on a backend instance.
At the moment FlashQL is in-memory only; persistence backends like IndexedDB / LocalStorage are on the roadmap.
2. Remote database from the frontend
If your source of truth is a remote Postgres/MySQL instance, the model we’re building is:
a LinkedQL engine next to the database, and
a FlashQL instance in the frontend that federates/syncs with that backend engine.
That federation/sync path is in alpha right now (early docs here: https://linked-ql.netlify.app/flashql/foreign-io ), so today the “stable” story is:
run LinkedQL on the backend against Postgres/MySQL,
expose whatever API you like to the frontend,
and use FlashQL locally where a client-side store makes sense.
The goal is that the frontend doesn’t need a special framework — just a LinkedQL/FlashQL client wherever JavaScript runs.
Please follow https://www.conventionalcommits.org/
Ah yes — good catch. The commit history definitely isn’t following Conventional Commits right now. Things got a bit loose during fast iterations, but I’ll follow the convention going forward.
How you solve scale of Live queries different from Zero sync? zero.rocicorp.dev
The docs and the comments here are clearly LLM generated. Please don't submit AI slop to HN, or at the very least talk about it in your own words!
The commit history is legitimately insane though: https://github.com/linked-db/linked-ql/commits/master/
Author here. Sad to hear that you perceive the docs and comments here as LLM generated. I'm genuinely curious what in particular gives you that impression.
I'm sorry, this comment [0] was clearly written (or rewritten) by an LLM. Sections are titled, constant m dash usages, and "Thanks again for the thoughtful look! We can zoom into any other area of your choice."
[0] https://news.ycombinator.com/item?id=46194311
Honestly, that was solely crafted by me. That comment needed to be formatted and properly sectioned.