Idea Ripper // ArchiveCards: 1555 [Live] | Updated: 2026-10-14
Card #c27IR // #c27Synthesis

For generative systems, quality is vector-valued. A financial LLM is not judged only by a semantic metric such as ROUGE, BLEU, cosine similarity, or pairwise preference. It is judged by a tuple: (3) covering task usefulness, evidence faithfulness, numerical consistency, calibration, policy compliance, latency, and cost.

— In our words · Source: Large Language Models in Finance (Miquel Noguer i Alonso)

[Use when]

When comparing models or vendors, demand the full tuple — usefulness, faithfulness, numerical consistency, calibration, compliance, latency, cost — not a single score.

[Why it matters]

A model that tops the semantic-similarity leaderboard can still hallucinate numbers, leak the future, and cost a fortune to run. Judging by one metric optimizes for the metric; judging by the tuple forces the trade-offs into the open.

Related principles

#c16Production systems should separate language generation from numerical computation. Let the LLM propose hypotheses, retrieve context, or…Use when an LLM-generated forecast, ratio, or expected return is about to enter a report, model, or trade.

More from Large Language Models in Finance

#c13Prediction and agency should not be conflated. A credit model can estimate the probability that a borrower breaches a covenant. An agentic…Use when a team proposes a stronger model as the fix for an agent that lacks memory, permissioning, or an… #c14The technically and institutionally mature objective is bounded autonomy : the agent can plan, retrieve, calculate, and propose actions…Use when deciding what an AI agent is permitted to do in production: which actions stay read-only, which need… #c15This principle prevents a common category error. A model that says it will respect a mandate has not yet respected the mandate. A model…Use when a vendor demo, benchmark score, or red-team chat log is being offered as proof that an agent is safe…