Prompt Caching Can Make Your Agent Slower
You flip on prompt caching expecting the bill to fall and the latency to drop, and most of the time it does. But the first controlled cross-provider study of caching on…
← All topics Topic
retrieval
Chunk on the Syntax Tree, or Don't Bother? What the Benchmarks Say
Structure-aware AST chunking helps code RAG, but the cAST headline overstates how much the chunking rule itself earns. cAST reports a 4.3-point Recall@5 gain on RepoEval and a…
Your Context Window Is a Ceiling, Not a Budget
The context length printed on a model's spec sheet is a marketing ceiling, not an operating budget, so I size a prompt for what the model can actually use, not for what caching…
Hybrid RAG: reciprocal rank fusion in practice
Combine BM25 and vectors without tuning ten weights: rank is enough.
Serving an open-source LLM in production: the real cost
vLLM, continuous batching, KV-cache: where the VRAM really goes.