One-sentence definition
Cálculo que cuantifica el error del modelo respecto al objetivo usado para actualizar sus parámetros durante el entrenamiento.
A direct definition: a criterion for discrepancy
A loss function is a mathematical function that assigns a value to a prediction by comparing it with a target or reference signal according to a specified criterion. The value represents the discrepancy that the function considers relevant to the task. It is not a universal measure of a model’s worth: its meaning depends on what the model predicts, which target is used, and which errors the function penalizes.
The word “loss” can suggest that the number measures everything that went wrong. It does not. The number is the result of applying a particular rule to particular data. Two functions can take the same prediction and target and return different values because they assign different weights to different kinds or magnitudes of error. To interpret a value, you need to know the task and the chosen formulation—not just whether the number looks large or small.
In everyday language, people sometimes call loss the model’s “error.” That can be a useful shorthand if you understand that it means the error as defined by the function, not necessarily a failure rate that a person can interpret directly. A loss may use a scale that does not translate immediately into a “percentage of incorrect answers” or a practical measure of quality.
How loss fits into training
The conceptual sequence starts with an input and a model prediction. The loss function compares that prediction with its corresponding target and returns a value. The training procedure uses this signal to adjust the model’s parameters so that later losses may be reduced according to the chosen criterion. The loss function defines what is measured; the optimization procedure determines how the parameters are adjusted.
So the loss function is not, by itself, the algorithm that updates the model. Saying that training “minimizes the loss” means that it aims to reduce the value associated with that criterion. It does not prove that the model is useful in an application, that all its answers are correct, or that the criterion captures every need in real-world use.
The details of an update depend on the training method and implementation. Gradient-based optimization methods commonly use gradients, but you should not infer a specific algorithm merely because a description mentions a loss function. Nor should you confuse calculating the loss with taking an update step: they are related parts of training, but conceptually distinct.
The basic sequence
- 01The model receives an input and produces a prediction.
- 02The loss function compares the prediction with the target defined for that input.
- 03The calculation returns a loss value according to the chosen criterion.
- 04The optimization procedure uses the training signal to adjust parameters; the loss function does not update them by itself.
- 05The process repeats, and the loss is considered alongside other relevant measures.
Loss for one example and aggregated loss
A function can return a value for an individual example. Training usually considers multiple examples, so a rule is needed to combine their values into an aggregate. For instance, an implementation might calculate an average or use another defined reduction. Do not assume that every system aggregates values in the same way.
Aggregation matters because it summarizes a set, but it can conceal differences between cases. A lower average does not, by itself, tell you whether every example improved or whether some examples still have high losses. To interpret a curve, it also helps to identify which data were used to calculate it: training data, validation data, or another set, and whether the data’s composition or processing changed.
So when someone reports that “the loss went down,” questions remain: did the per-example loss decrease, or the aggregate? Which reduction was used? On which data split? Between which points in training? Without those details, the trend may be informative, but it is not a complete description of performance.
What to check when reading a value or curve
| Question | Why it matters |
|---|---|
| Which function was calculated? | It defines which discrepancies count and how they are valued. |
| Which examples were included? | The value depends on the data over which it is calculated. |
| How were values aggregated? | The reduction determines how losses from multiple examples are summarized. |
| Is this training or evaluation loss? | A loss calculated on training data does not, by itself, describe behavior on other data. |
Applied example: numerical regression
Imagine a regression task that predicts how long an order will take to arrive. For one order, the model produces an estimated duration, which is compared with the recorded target duration. A loss function assigns a value to the discrepancy between them. If the prediction is far from the target, the value may reflect a larger discrepancy, according to the chosen rule.
The choice of rule changes which errors receive more weight. For illustration, one formulation may make large errors especially influential, while another may reduce their relative influence. You cannot tell which formulation is being used just from the label “regression”; you need to check the function’s definition and configuration.
This also shows why “minimizing error” is an incomplete description. Training aims to reduce the discrepancy formalized by the function—not necessarily the average delivery delay, operating cost, or proportion of orders delivered within an acceptable window. Those questions may require additional evaluation metrics.
Applied example: text classification
In text classification, a model receives a text and produces an output related to several possible classes, such as “question,” “complaint,” or “request.” Training data include target labels for the examples. A loss function compares the model’s output with those labels using a classification rule and returns a value that can be used during training.
That value is not automatically the percentage of texts classified correctly. Accuracy answers a different question: how many final predictions matched their labels under a particular decision rule. A loss function may use more information from the model’s output than simply whether the selected class was right or wrong. As a result, two models with the same accuracy might have different losses, depending on their outputs and the function used.
To interpret the result, check which labels were used, which function was applied, and which texts were included in the calculation. It also matters that a lower training loss does not guarantee that the classifier will perform just as well on new texts, on a different distribution of topics, or on categories where the cost of an error differs.
Applied example: object detection
In object detection, a model must produce information about the objects present in an image. As a conceptual outline, we can distinguish questions about where an object is and which class it belongs to. A loss for this task might reflect discrepancies across more than one aspect of the prediction; the specific way components are combined depends on the model and its implementation.
This example helps avoid an oversimplification: a complex task does not always have one elementary error that captures everything. However, it would not be rigorous to attribute one formula, a fixed list of components, or a universal weighting to all detectors. To describe a particular detection loss, you would need to consult the technical documentation for that implementation, including which terms it calculates and how it aggregates them.
Nor does the loss automatically replace the measures used to evaluate detections. If a report provides a loss value, find out which criterion it represents. If it provides an evaluation metric, check that metric’s definition separately. Comparing both types of information can be useful, but their values should not be treated as if they were on the same scale.
Related concepts: loss, metric, cost, and optimizer
An evaluation metric summarizes an aspect of performance using a measurement rule. It can help answer questions such as how many predictions were correct or how good an output is according to a specific definition. A loss, by contrast, is the criterion that assigns a value to predictions and targets for the stated task. Some metrics and losses may be related, but they are not synonyms by default and need not be interpreted in the same way.
“Error” is a general and ambiguous term. It might mean the discrepancy calculated by a formal function, an incorrect prediction, or a metric expressed in understandable units. It is clearer to name the exact quantity. “Cost function” is also used in different ways in technical writing, for example to refer to an aggregate criterion. Do not assume that all authors use the same terminology to distinguish an individual loss from an aggregated cost.
The training objective is the broader criterion that a training setup aims to optimize. The loss function may be part of that objective, but a complete description can include other elements. The optimizer is the procedure that carries out parameter updates. In short: loss is a defined measure; the objective is what the setup seeks to optimize; the optimizer is the method that organizes updates; and a metric is a way to evaluate a property.
Useful distinctions
| Concept | Question it helps answer | Do not confuse it with |
|---|---|---|
| Loss function | What value does the criterion assign to a prediction relative to its target? | A complete account of the model’s usefulness. |
| Evaluation metric | How is an aspect of performance summarized? | Training loss, unless it is explicitly defined that way. |
| Optimizer | How are parameters adjusted during training? | The function that calculates the loss. |
| Cost or aggregate objective | Which overall quantity is a particular formulation trying to optimize? | A universal term with the same meaning in every source. |
What a low loss does not prove
A low loss shows that, on the data considered and according to the defined function, the value of that criterion is low. By itself, it does not prove that the model generalizes well to unseen examples, is calibrated, is useful to the people who will use it, or treats different groups fairly. Those properties require appropriate definitions, data, and evaluations.
There may also be a difference between the training data and the data encountered in use. If the model fits the criterion very well on training examples but behaves worse on other data, training loss alone is not enough to detect or explain the problem. Comparing results on suitable datasets can provide evidence, although the selection and representativeness of those datasets also matter.
A curve does not speak for itself. Its interpretation depends on the function, aggregation, data, and point in training at which it was calculated. A downward trend may indicate that the procedure is reducing the optimized criterion, but it does not, without further evidence, show that every capability of the model is improving. This distinction is central to understanding generalization and overfitting.
Practical steps for interpreting loss
- 01Identify the task, the specific function, and the target against which the prediction is compared.
- 02Find out whether the value is for one example or an aggregate, and which aggregation rule was used.
- 03Check which data split produced the value and whether it is comparable with the value you are contrasting it with.
- 04Consult metrics suited to the practical consequences of errors in that task.
- 05Do not infer generalization, calibration, usefulness, or absence of bias from an isolated loss value.
Common mistakes and related concepts
One common mistake is to say that a model “learned because the loss reached zero.” A value of zero has a specific interpretation only with respect to a particular function and particular data; it does not show that the system handles every relevant case correctly. Another mistake is to compare losses from different tasks, functions, or configurations directly. Without compatible formulations, the numbers may not be comparable.
People also confuse the loss function with the optimization algorithm, or report a loss as if it were a product metric. The clearest way to avoid this is to state which function was used, which dataset it was calculated on, and which metric evaluates the operational result. When discussing training, loss describes the criterion; when discussing results, the evaluation measures should be made explicit.
For further reading, this entry connects to evaluation, generalization, and overfitting: loss is one part of interpreting training, while those concepts help examine behavior beyond the optimized value. It is also relevant to fine-tuning and self-supervised learning, where the objective formulation determines which signal is used. The glossary index and the comparison of related concepts can serve as starting points, but each term needs to be defined in its own context.