Honestly, I was very impressed with this. I did find the latency was quite high, so I tried to tune it to use MLX on Mac GPUs, and it's so much faster. Very cool either way, we're getting close to a full AI interactive tool that's offline and private!
Honestly, I was very impressed with this. I did find the latency was quite high, so I tried to tune it to use MLX on Mac GPUs, and it's so much faster. Very cool either way, we're getting close to a full AI interactive tool that's offline and private!
https://github.com/skorotkiewicz/lfm3-chat/pull/1