Welcome to Incremental Social! Learn more about this project here! Check out lemmyverse to find more communities to join from here!
Zworf , 20 days ago It depends on your prompt/context size too. The more you have the more memory you need. Try to check the memory usage of your GPU with GPU-Z with different models and scenarios.
It depends on your prompt/context size too. The more you have the more memory you need. Try to check the memory usage of your GPU with GPU-Z with different models and scenarios.