I think it is the first time I hear about a merge of two workflow tools.
I work with workflows and HPC, and the most common is multiple workflow managers in an hpc center, or even within a department. Which is not always necessary (sometimes it feels like just because there is budget and someone skilled they prefer to reinvent than reuse).
I hope this means that trend is stopping.
Another thing I hope I will start seeing more are workflow managers sharing libraries. Most workflow managers need to handle similar tasks like submitting tasks to cloud, slurm, containers. Or even performing graph operations, visualizing graphs.
Workflow managers are the ultimate wheel to be reinvented. Everyone needs some form of them, they're reasonably easy to implement a poorly working version of, and everyone thinks that they can do it better. In the data engineering world there are literally dozens of solutions that are essentially "airflow, packaged slightly differently"
I think it is the first time I hear about a merge of two workflow tools.
I work with workflows and HPC, and the most common is multiple workflow managers in an hpc center, or even within a department. Which is not always necessary (sometimes it feels like just because there is budget and someone skilled they prefer to reinvent than reuse).
I hope this means that trend is stopping.
Another thing I hope I will start seeing more are workflow managers sharing libraries. Most workflow managers need to handle similar tasks like submitting tasks to cloud, slurm, containers. Or even performing graph operations, visualizing graphs.
Workflow managers are the ultimate wheel to be reinvented. Everyone needs some form of them, they're reasonably easy to implement a poorly working version of, and everyone thinks that they can do it better. In the data engineering world there are literally dozens of solutions that are essentially "airflow, packaged slightly differently"