ai-toolsUpdated
DeepSeek-V4.1-Flash: 5 Real Benchmarks & GGUF Specs (2026)
Real DeepSeek-V4.1-Flash benchmarks: tokens/sec speed, 1M context KV cache limits, 552B MoE architecture, and VRAM requirements on RTX 4090 and A100.
12m read
#local-llm
11 articles
hardware-troubleshootingUpdated
AMD Strix Halo Local LLM: 128GB Benchmarks Tested (2026)
AMD Strix Halo benchmarks for local LLMs. We tested 128GB unified memory running 70B and Qwen models with tokens/sec throughput and thermal numbers.
11m read
ai-automationUpdated
DeepSeek-V3 Hardware Requirements: RAM & GPU Math (2026)
Can you run DeepSeek-V3 671B locally? We calculate exact RAM, NVMe swap, and multi-GPU VRAM requirements for FP8, Q4, and CPU offloading.
15m read
ai-automationUpdated
Why Ollama Truncates Context at 2048 Tokens: num_ctx Fix
Why does Ollama cut off chat history after 2048 tokens? How to override num_ctx in your Modelfile and API parameters without crashing your GPU.
10m read
ai-toolsUpdated
Dual GPU Local LLMs: Fix Tensor Parallelism in vLLM (2026)
Why does tensor parallelism fail across dual consumer GPUs like dual RTX 3090s? Fix PCIe P2P blocks, NCCL timeouts, and pipeline parallelism in vLLM.
14m read
ai-toolsUpdated
Why 32k Context Crashes Local LLMs: VRAM & KV Cache Fix
Why does a 32k context window crash your local LLM with out-of-memory errors? How KV cache scaling works and how to enable FP8 cache quantization.
13m read
ai-automationUpdated
DeepSeek-R1 FP8 vs Q4: VRAM Benchmarks & Speed Guide (2026)
DeepSeek-R1 FP8 vs Q4_K_M quantization tested on local hardware. Real VRAM usage benchmarks, token speeds on RTX 4090 and 3090, and quality loss analysis.
7m read
aiUpdated
DeepSeek-R1 vs Gemini Flash: Local AI Benchmarks & VRAM
DeepSeek-R1 Distill vs Gemini Flash benchmarks: Firsthand tokens/sec, VRAM footprint, MATH500 reasoning accuracy, and API costs on 8GB–16GB consumer GPUs.
12m read
ai-automationUpdated
Ollama Running Slow? 5 num_ctx & GPU Offload Fixes (Tested)
Why is Ollama generating tokens so slowly? Fix CPU fallback caused by large num_ctx values, tune layer offloading, and optimize VRAM in 5 steps.
11m read
ai-automationUpdated
Open WebUI + Ollama RAG: Local IT Runbook Indexing Guide
How our team set up Open WebUI and Ollama with nomic-embed-text to index internal IT runbooks, network topologies, and incident notes 100% offline.
8m read
ai-automationUpdated
Run DeepSeek R1 on 8GB VRAM: 5 Real Benchmarks in 2026
Can you run DeepSeek-R1 on 8GB VRAM? Benchmarked 8B and 14B Q4_K_M models on RTX 4060 and RX 7600 with exact token speeds, VRAM footprints, and Ollama flags.
17m read