What is Inference?

Running a trained model to get an answer. Training happens once and is expensive; inference happens every time someone uses the model, so its speed and cost matter most for services.

AI glossary

LLM (large language model)Token Context window Prompt Fine-tuning RAG (retrieval-augmented generation)Embedding Parameters (7B, 70B…)Quantization Open weights Hallucination Multimodal Diffusion model AI agent Benchmark