A finance team must decide whether a high-volume document-classification workload should stay on a cloud model API or move to a locally hosted open-weight model. The workload runs around the clock at a steady rate. Which method gives a sound comparison?
- A.
Compare the providers' published per-token list prices only, because token price is the single cost that differs between hosting options and every other cost is identical.
- B.
Compare total cost of ownership at the expected steady utilisation, including hardware, power, networking and operations staff for local hosting, against the per-token charges for the same volume and required quality.
- C.
Compare the leaderboard scores of the cloud model and the open-weight model, choosing the higher-scoring one, because the most capable model always produces the lowest total cost.
- D.
Compare the purchase price of one GPU server against a single month of cloud API charges, choosing local hosting if the server costs less than two months of cloud usage.
Show answer
Answer: B
Hosting economics need a full TCO comparison at real utilisation, matched on the output quality required.
- A. Incorrect: infrastructure and operations costs differ sharply between hosting options, so token price alone misleads.
- B. Correct: a TCO comparison at real utilisation, matched on required quality, captures every cost that differs.
- C. Incorrect: leaderboard scores measure capability rather than cost, and the most capable model is not automatically cheapest.
- D. Incorrect: purchase price against one month of charges omits power, staff, networking and lifecycle costs.