What is Quantization?

Storing a model’s numbers with fewer bits (for example 4-bit instead of 16-bit) so it fits on a normal PC or phone. It gets much smaller and faster, with a small loss in quality.

AI glossary

LLM (large language model)Token Context window Prompt Fine-tuning RAG (retrieval-augmented generation)Embedding Parameters (7B, 70B…)Open weights Hallucination Multimodal Diffusion model AI agent Inference Benchmark