LLM memory estimator

Deterministic serving math — will it fit, how many KV tokens, how many concurrent N-token requests. Same math as the memory-estimate CLI; no load, no endpoint, no secrets. Structure comes from the model's config.json (fetched from HF or pasted); serving parameters come from you.

Model

…or paste a config.json (works offline, no hub access)

GPU

or custom VRAM (GiB)

Weights

Weight dtype override the checkpoint

MoE runner backend optional

Expert parallel (EP) MoE only, ≤ GPUs

KV cache

KV dtype

Speculative (built-in MTP module) off

HiCache L2 (host RAM)

HiCache L3 (backing tier, GiB / replica)

L2 layout across TP ranks auto is right for every family we know

Engram tables DeepSeek-V4.1 n-gram memory

Engram host budget (GiB) pod memory limit — tables + ~24 GiB runtime

Compress-state dtype SGLANG_DSV4_COMPRESS_STATE_DTYPE

SSM state dtype --mamba-ssm-dtype; float32 doubles the state

Mamba full-memory ratio --mamba-full-memory-ratio; 64 seqs @ 64k ≈ 0.08

Max mamba cache size --max-mamba-cache-size; seqs the pool may hold — demand-sizes the pool (engine sizes by demand up to this)

Target mamba concurrency seqs the pool must carry

Deployment

GPUs

Tensor parallel (TP)

Pipeline parallel (PP) 1 = off

mem-fraction-static

Overhead (GiB/GPU)

Contexts (comma-separated)

TP sweep grid

Deployment sanity check (interactive page)

The fuller checklist version — launch arguments, HiCache tiering, seed catalog import — lives in the standalone memory calculator page rendered by memory-estimate --calculator (see the README, Part 3).