4 calculators in this category
- LLM GPU VRAM Requirement CalculatorThis calculator adds up the four things that occupy GPU memory when you run a language model: the weights, the KV cache, the…
- KV Cache Size CalculatorTransformer key-value cache memory modelEvery token a transformer has already seen leaves a key and a value vector in memory for each attention layer, and that cache…
- Model Quantisation Memory Savings CalculatorWeight-precision memory accountingA model's weight memory is just parameters times bytes per parameter, so quantisation buys memory in exact proportion to the…
- GPU Hours Cost CalculatorGPU-hour cost accountingA GPU-hour is the unit every cloud accelerator bill is written in: one GPU held for one hour. This calculator turns an hourly…
Which one do you need?
- LLM GPU VRAM Requirement Calculator
- Size the GPU memory a language model needs from its parameter count, quantisation, context length and batch size, and see how many GPUs that takes.
- KV Cache Size Calculator
- Compute the key-value cache a transformer holds at your context length and batch size, the term that dominates long-context serving, and what fits.
- Model Quantisation Memory Savings Calculator
- Compare model footprint across precisions and get the compression ratio, memory freed and effective bits per weight before you run the quantisation.
- GPU Hours Cost Calculator
- Price a block of GPU work from the hourly rate, GPU count and run length, then compare on-demand against spot once interruption overhead is counted.
More ai, llm & machine learning engineering categories
Related categories in other subjects
Where this work overlaps other trades and disciplines.