Welcome to Incremental Social! Learn more about this project here! Check out lemmyverse to find more communities to join from here!
xcjs , 25 days ago No offense intended, but are you sure it's using your GPU? Twenty minutes is about how long my CPU-locked instance takes to run some 70B parameter models. On my RTX 3060, I generally get responses in seconds.
No offense intended, but are you sure it's using your GPU? Twenty minutes is about how long my CPU-locked instance takes to run some 70B parameter models.
On my RTX 3060, I generally get responses in seconds.