Glossary
Quantization
Answer in brief
Quantization stores the weights of a model at lower precision, such as 8-bit or 4-bit, so the model takes less GPU memory.
Quantization in practice
At 16-bit precision, weights take about 2 GB for every billion parameters. At 4-bit they take about a quarter of that. This is how the weights of a larger language model are brought under 24 GB. These figures are the weights alone: running a model needs room on top for context and activations.
The saving has a cost in accuracy that depends on the model and the task. Test a quantized model on your own prompts before you rely on it.
Questions and answers
Sources for this entry
- Runcrate: Runcrate’s official site: what the platform is, how it bills and the range of GPUs it offers. Read 12 Nov 2026.
- Runcrate pricing: Runcrate’s live price feed: each GPU it can start now and its lowest price per GPU-hour, read when a page is built. Read 12 Nov 2026.
- Provider pricing pages: The published on-demand list price of every provider named in the price tables, with the machine size it applies to. Read 12 Nov 2026.
- Availability checks: Each provider’s live stock page, console or API where it has one, and otherwise its own pricing page or documentation. Read 12 Nov 2026.
- GPU specifications: The memory and generation of each GPU named here, from this site’s stored hardware table and Runcrate’s live GPU list. Read 12 Nov 2026.
Checklists are original editorial guidance. Published prices do not establish availability.
Continue this reading path
Cite this entry
RTX 4090 Rental. “Quantization.” Content reviewed 12 Nov 2026. https://rtx4090rental.com/glossary/quantization