Lossless compression of LLMs to 12-bits

1 comments

• AlpinDale 14 hours ago

  A lossless encoding of BF16 weights to NF12, with speedups roughly proportional to amount of memory saved.