I'm using koboldcpp and ollama. KoboldCpp is really awesome. In terms of hardware it's an old PC with lots of RAM but no graphics card, so it's quite slow for me. I occasionally rent a cloud GPU instance on runpod.io Not doing anything fancy, mainly role play, recreational stuff and I occasionally ask it to give me creative ideas for something, translate something or re-word or draft an unimportant text / email.
Have tried coding, summarizing and other stuff, but the performance of current AI isn't enough for my everyday tasks.
Any of you have a self-hosted AI "hub"? (e.g. for LLM, stable-diffusion, ...)
I've been looking into self-hosting LLMs or stable diffusion models using something like LocalAI and / or Ollama and LibreChat....
Inside the Creation of the World’s Most Powerful Open Source AI Model (www.wired.com)