ai-tools
DeepSeek-V4.1-Flash: Speed, VRAM & Benchmarks (Tested)
Real DeepSeek-V4.1-Flash benchmarks: tokens/sec speed, KV cache limits, 552B MoE architecture, VRAM usage on RTX 4090, and why V4 Pro was retired.
12m read
#vllm
6 articles
ai-tools
vLLM vs SGLang: PagedAttention vs RadixAttention Benchmarks
We benchmarked vLLM vs SGLang on multi-turn agent workloads. RadixAttention prefix caching cut TTFT by 82% over PagedAttention. Here are the benchmarks.
17m read
ai-tools
Fix Dual GPU Tensor Parallelism: vLLM & llama.cpp Guide
Running dual GPUs for local LLMs? Why Tensor Parallelism fails without NVLink, how to fix NCCL P2P crashes in vLLM, and how to split 70B models with llama.cpp.
14m read
ai-tools
Why 32k Context Crashes Local LLMs: KV Cache VRAM Fix
Why does 32k context trigger CUDA out-of-memory errors on 12GB and 16GB GPUs? Learn the exact KV cache VRAM math, Ollama config, and FP8 fixes.
13m read
ai-tools
Ollama vs vLLM vs LM Studio: 2026 Speed & VRAM Benchmark
Ollama, vLLM, or LM Studio? We benchmarked VRAM, TTFT latency, tokens/s, and concurrency on Windows 11 & WSL2 with RTX 4090/3080. See the empirical winner.
13m read
ai-automation
How to Enable FP8 KV Cache in Ollama & vLLM (128K Context)
Break the VRAM wall in local LLMs. Learn the exact memory math, vLLM FP8 KV-cache flags, and Ollama Flash Attention settings to run 128K context on 24GB GPUs.
7m read