I think the difficult aspect is that you have to re-train the model after conversion.
I assume you could use this during training (QAT) similar to ternary quantization.
It’s a bit beyond me, but it seems promising–at least for general knowledge models.
I think the difficult aspect is that you have to re-train the model after conversion. I assume you could use this during training (QAT) similar to ternary quantization.
It’s a bit beyond me, but it seems promising–at least for general knowledge models.