What is Inference?
Inference is the step where a trained AI model is used to make a prediction or generate an answer for new input. Training teaches the model; inference is the model doing its job.
Why it matters for your business
Inference is where most running costs of AI features come from, so model choice, caching and batching have a direct effect on your monthly bill and response speed.
Every time a customer asks the assistant a question, one inference call to the language model generates the answer.