Transformer KV Cache Memory Calculator
Calculate dense decoder key/value cache storage from entered layer, head, token and element sizes.
YOUR CALCULATION
Result
—
Calculation steps
Work through one step at a time or reveal the complete calculation. The steps use your current inputs.
How to calculate
Count key and value tensors separately using the entered KV-head count rather than query heads.
Cache bytes=2×layers×KV heads×head dimension×tokens×batch×bytes per element
Worked example
32 layers, eight KV heads, 128 dimensions, 4096 tokens and two-byte elements require 512 MiB for one sequence.
Assumptions and input details
Dense equal-length cache only; no padding, page tables, quantization scales, allocator reserve, weights, activations, sliding eviction or model-specific architecture inference. Fractional bytes represent packed elements without metadata.
Using this calculator
Can I enter my own problem?
Yes. Replace the example values with your own inputs. The result, calculation steps, tables and applicable diagrams update from those inputs. Blank or invalid values clear the result; example values are never silently substituted for missing inputs.
Are the results exact?
Results use floating-point arithmetic and displayed rounding. Read the formula, assumptions and any stated data precision before interpreting the result.
Is my input uploaded?
The calculation runs in your browser. No account, file upload or external AI service is required. Inputs are not saved between page reloads.
Compare bits and bytes using clearly separated decimal and binary prefixes.
Calculate an ideal compute-or-memory throughput roof from entered operations, bytes moved and hardware ceilings.
Calculate square 2D convolution output dimensions, grouped parameter count and multiply-accumulate count.
Reserved advertising space.