Cohere Embed 4: How to Measure the Cost of a Useful Retrieval

The economic question is not how much a call costs

The cost of an embedding model is usually expressed in a billing unit that depends on the access channel. Even when confirmed, that figure alone does not answer the question that matters to a search team: how much does it cost to obtain results that meet a usefulness criterion for its users? A low-cost call may produce irrelevant retrievals. A more expensive configuration might improve results, but it is economically preferable only if that improvement matters for the corpus and the queries people actually use.

It helps to separate three questions. The first is commercial: exactly which product is billed, through which provider or channel, in what unit, and under what terms? The second is technical: what retrieval quality does the evaluated configuration achieve? The third is operational: what other components, such as storage or search infrastructure, are part of total spend? Combining them into a single figure can make a comparison look more conclusive than the evidence warrants.

The evidence provided for this article does not confirm a specific price or billing unit for Embed 4 Small or Medium. It therefore does not support publishing a cost ranking between the variants. It does, however, support setting out a reproducible test and specifying what information must be collected before making that comparison.

What to verify before comparing Small and Medium

Before running an economic test, record the exact model identifier, the variant you intend to use, and the access channel. Do not assume that a price associated with Cohere also applies to a cloud service offering the model, or that one provider’s terms carry over to another. The available documentation identifies Embed 4 in Azure AI, Amazon Bedrock, and Oracle Cloud Infrastructure services, but the information provided does not establish that their access or billing terms are the same.

For each channel, check an official source for the current price, billing unit, any applicable minimums or additional charges, the date you checked, and whether the price distinguishes between variants. Keep a copy or another verifiable record of what you found: prices can change, and the date is part of the comparison. If one price is expressed in a different unit from another, do not convert the figures by assuming equivalences that the documentation does not support.

You also need to determine what counts as a billable input and whether handling differs by modality or configuration. A service’s documentation may describe request parameters or limits, but that does not, by itself, establish how the service calculates the bill. If you cannot confirm the relationship between a technical limit and a commercial billing unit, keep those two facts separate.

Design a test that reflects real work

A useful comparison starts with a representative corpus, not a collection selected because it favors one variant. Include documents, queries, and usage patterns that resemble production: relevant languages, lengths, content types, and difficult cases for the system being evaluated. If the application searches multimodal content, include it only when the applicable channel and documentation allow it to be tested under clearly defined conditions.

Define what counts as a useful retrieval before measuring. For example, the team can specify which results are relevant to a query and what relative ordering it expects. The important thing is not to adopt a metric out of habit, but to document the criterion and apply it consistently to every variant. Queries used to tune the configuration should not be confused with those reserved for final evaluation. Keeping the sets separate helps prevent the result from reflecting decisions made on the same cases being evaluated.

Relevance judgments also require an explicit procedure. Record who evaluates the results, what instructions they receive, and how disagreements are resolved. If evaluators do not agree, preserve that uncertainty in the analysis. An aggregate score cannot fix inconsistent labels or turn a small sample into conclusive evidence.

Measure quality per query before summarizing

The practical unit of analysis is each evaluated query. For every query, record what the configuration retrieved, which items were judged relevant, and whether the agreed criterion was met. Depending on the system’s goal, you may care about whether relevant results appear near the top, how many relevant items are retrieved, or the quality of the ranking. The right choice depends on how retrieval will be used; do not present any one metric as universally sufficient.

After reviewing query-level results, calculate a summary for the set and retain the distribution as well. Two variants can have similar averages but behave differently on difficult queries, particular languages, or document types. Showing those cases helps reveal whether an overall improvement comes at the expense of a user group or part of the corpus.

Set an acceptance threshold before seeing the comparison results. If the team redefines it after learning which variant passes, the decision is no longer a neutral test. When the sample or available judgments cannot distinguish a small difference from noise, describe the result as inconclusive. Do not turn a numerical variation into a proven improvement without sufficient evidence.

Calculate the cost associated with meeting the threshold

Once the channel’s billing unit and price are verified, calculate separately the components that the evidence allows you to quantify. If the price applies to embedding generation, record the volume actually processed and apply the published unit to that volume. Do not fill in the calculation with an assumed unit or present an estimate as observed spend. If the channel does not publish the information needed to reproduce the calculation, state that the cost cannot be calculated from the available information.

To express the result as cost per retrieval that meets the objective, define the denominator clearly. It could be the number of evaluation queries that meet the relevance threshold under the tested configuration. The numerator must correspond to the spend being attributed and to the same period or body of work. If the measured spend covers initial embedding generation, do not describe it as the complete cost of each query without explaining how that spend is allocated and what assumptions are used.

It is worth publishing the intermediate quantities, not just the final ratio: volume processed, attributable spend, number of queries evaluated, and number that met the criterion. That way, someone else can review the calculation and change assumptions, such as corpus size or update frequency. A figure without those details can hide differences in scope between variants.

Text and additional modalities require equivalent conditions

The available sources describe Embed 4 as a multimodal model and mention the ability to work with text and images in certain services. That general description is not enough to conclude that a multimodal evaluation is directly comparable across channels, or that all channels offer the same input conditions, parameters, or billing. The test must specify which service is being used and what format it supports in that context.

For a text evaluation, use queries and text documents that match the application. For an evaluation involving images or interleaved content, define the material, queries, and relevance criteria for that modality, and confirm that the selected service accepts that input. Do not combine text and multimodal results into one figure if the sample composition or cost scope differs.

Technical limits described by one provider must not be transferred to another. The Oracle documentation included among the sources concerns Embed 4 in OCI; the Amazon Bedrock documentation applies to Bedrock, and the Azure AI listing applies to its catalog. Before using an input limit in the test design, verify that it belongs to the specific service and usage mode being evaluated.

Interpret differences without extrapolating

A comparison describes only the corpus, queries, configuration, and conditions that were evaluated. It does not automatically show that the same variant is preferable for another language, document set, usage volume, or service. If the evaluation used a limited sample, keep the conclusion limited to that sample and state its constraints.

Examine failed queries, not just the average. An improvement on a small number of queries may matter if those queries are critical to the use case; it may also be unstable if it depends on a small sample or disputed judgments. The team should decide in advance which types of failure are acceptable and what impact each would have. Economic analysis does not replace that decision about quality risk.

When one variant improves retrieval slightly, ask whether the change exceeds the uncertainty of the evaluation and whether it usefully changes the proportion of queries that meet the threshold. If there is not enough evidence, report that no reliable difference has been established. Avoid presenting a definitive ranking or universal recommendation based on a local test.

What to do when prices are not comparable

If official information does not clearly identify the price, billing unit, or covered variant, do not create a ranking from page fragments, general announcements, or prices for another channel. The Cohere pricing page included among the sources does not confirm, in the available material, a specific price or unit for Embed 4 Small and Medium. The developer pricing announcement provides context about pricing policy, but does not by itself establish the price of those variants.

In that situation, publish only what has been verified: the evaluated channel, the date checked, confirmed technical conditions, and observed quality metrics. State which commercial details are missing and which conclusion cannot be reached because of their absence. If a marketplace says billing is based on usage but the material consulted does not show a specific unit or price, that description is not enough to derive a cost per query.

This limitation does not invalidate the quality test. It allows a reproducible technical evaluation to be separated from a cost analysis that cannot yet be completed. Once comparable official prices are available, the calculation can be completed without changing the recorded metrics, provided the relevant volumes and conditions have been saved.

Practical checklist for repeating the comparison

A useful comparison leaves a record that another team can repeat. Save the exact model identifier, channel, and configuration; the composition of the corpus and queries; the instructions and relevance judgments; the acceptance criteria set before measurement; and query-level metrics as well as summaries. Also record any changes made during the test, since a modification can make comparison with later runs more difficult.

For the economic analysis, retain the official price consulted, its date, unit, currency, and scope whenever those details are available and unambiguous. Record processed volumes and separate storage, infrastructure, and operations costs when known. If those costs cannot be documented, leave them out of the calculation and state that the result does not represent the system’s total cost.

Finally, decide when the test will be repeated. A change in price, channel, variant, configuration, or corpus can alter the result. The goal is not to produce a permanent label for the cheapest variant, but to maintain a decision that can be revisited and tied to evidence: which retrievals meet the objective, under what conditions they were measured, and what verifiable spend can be attributed to them.

Keep exploring

Corrections and transparency

If you spot incorrect or outdated information, send us a correction with the page and source we should review.