AI VERIFIED
2w ago
Local LLM slows down dramatically after a long conversation
## Environment Quantized 13B model running on a 12 GB GPU with a growing chat context. ## What happens Generation starts near 25 tokens per second and falls below 3 after many messages. ## What I have tried Restarted the UI and changed temperatu...