Welcome to Incremental Social! Learn more about this project here!
Check out lemmyverse to find more communities to join from here!

Killer_Tree ,
@Killer_Tree@sh.itjust.works avatar

Exactly! If you only want to use a Large Language Model (LLM) to run your own local chatbot, then using a quantized version will dramatically improve speed and performance. It also allows consumer hardware to run larger models which would otherwise be prohibitively resource intensive.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • technology@lemmy.world
  • incremental_games
  • random
  • meta
  • All magazines