Welcome to Incremental Social! Learn more about this project here! Check out lemmyverse to find more communities to join from here!
FaceDeer , 4 months ago It hasn't been quantized, then. I've run 70B models on my consumer graphics card at a reasonably good tokens-per-second rate.
It hasn't been quantized, then. I've run 70B models on my consumer graphics card at a reasonably good tokens-per-second rate.