Some weeks ago we open sourced ch-kit, a schema-as-code toolkit for ClickHouse that allows teams to:
- define tables and views in TypeScript to ensure they are always correct
- generate and apply migrations automatically by diffing the current and previous state, with risk classification (safe / caution / danger) and drift detection along the way
- pull your existing ClickHouse schema into your repo in TypeScript with just a command
- and much, much more.
As we mentioned, a python port of the toolkit was a must have for us and for many of the users that are already leveraging ch-kit. After thorough testing, we are excited to also share chkit-py with the ecosystem. It covers all the core features of the original package (some of the plugins are already built but pending testing).
Similarly to its TS counterpart, it's still in beta; so there could be changes based on users feedback, or if we determine there are better ways of managing backfills, mutations, etc.
If you use python to deal with ClickHouse® at scale, we recommend taking a look at it. And if you have any suggestions, you know where to find us!
One of the things that was most challenging and still rough around the edges are backfills. Doing a BC breaking schema change in a chain of materialized view and repopulating can take a week or more for a big dataset. So having stable tooling to help guide that change was super helpful for some of our past migrations.
Hey HN!
Some weeks ago we open sourced ch-kit, a schema-as-code toolkit for ClickHouse that allows teams to:
- define tables and views in TypeScript to ensure they are always correct - generate and apply migrations automatically by diffing the current and previous state, with risk classification (safe / caution / danger) and drift detection along the way - pull your existing ClickHouse schema into your repo in TypeScript with just a command - and much, much more.
As we mentioned, a python port of the toolkit was a must have for us and for many of the users that are already leveraging ch-kit. After thorough testing, we are excited to also share chkit-py with the ecosystem. It covers all the core features of the original package (some of the plugins are already built but pending testing).
Similarly to its TS counterpart, it's still in beta; so there could be changes based on users feedback, or if we determine there are better ways of managing backfills, mutations, etc.
If you use python to deal with ClickHouse® at scale, we recommend taking a look at it. And if you have any suggestions, you know where to find us!
One of the things that was most challenging and still rough around the edges are backfills. Doing a BC breaking schema change in a chain of materialized view and repopulating can take a week or more for a big dataset. So having stable tooling to help guide that change was super helpful for some of our past migrations.