Actually, my question not about possibility, but is there any value in it? May be any message queue or k8s cluster with StatefulSets is easier to operate? What do you think?
It is an interesting idea. What troubles me is the read/write rate in a real production environment. If it is bound by S3, a buffer will eventually be needed and then it all become very complicated. At least it is how it looks on the surface
You are right, the time of a single write/read is Achilles' heel. But if we are talking about parallel processing it can keeps several hundreds milliseconds delay and provides enormous throughput.
Actually, my question not about possibility, but is there any value in it? May be any message queue or k8s cluster with StatefulSets is easier to operate? What do you think?
It is an interesting idea. What troubles me is the read/write rate in a real production environment. If it is bound by S3, a buffer will eventually be needed and then it all become very complicated. At least it is how it looks on the surface
You are right, the time of a single write/read is Achilles' heel. But if we are talking about parallel processing it can keeps several hundreds milliseconds delay and provides enormous throughput.