Welcome to Incremental Social! Learn more about this project here! Check out lemmyverse to find more communities to join from here!
General_Effort , 4 months ago https://huggingface.co/senseable/Smaug-72B-v0.1-gguf/tree/main About 44GB and 50GB for the Q4 and 5. You'd need quite some extra to fully use the 32k context length.
https://huggingface.co/senseable/Smaug-72B-v0.1-gguf/tree/main
About 44GB and 50GB for the Q4 and 5. You'd need quite some extra to fully use the 32k context length.