Practical LLM JSON Schema: How to Make AI Consistently Output Financial Report Revenues, Cash Flows, and Risk Factors?

Problems Addressed in This Article

• LLMs often confuse Non-GAAP and GAAP financial metrics during extraction and may even miss units, years, or the scope of consolidation.
• When the model encounters undisclosed data in financial reports, it tends to guess and fill in numbers using common sense or vague formulas.
• Risk factors and management discussion sections are overly compressed by the model into meaningless boilerplate, losing sensitivity to subtle tonal variations.
• The JSON format output by the model is prone to parsing errors in production due to truncated quotes, extra commas, or explanatory prefaces.

Who Should Read This

• Full-stack engineers developing AI Agents or RAG applications who face challenges extracting high-precision data from unstructured text.
• Investment researchers looking to use LLM tools to analyze financial reports more efficiently while maintaining strict control over hallucinations and data accuracy.
• Technical architects focused on stable structured output, validation gates, and fault-tolerant LLM design.

Position in the Toolchain

This article only addresses structured output and JSON Schema constraints. For PDF table extraction, see Financial Report PDF Table Extraction; for asynchronous task services, see AI Financial Report Task Queue; for product entry, see AI Finance Tool.

Structured Verification Fields

Before launch, it is recommended to retain source_page, evidence, confidence, review_status, and null_reason, so that each conclusion can trace back to the original text location, evidence segments, and manual verification status.

Why Financial Report Analysis Cannot Allow Free LLM Output

Free natural language output causes the financial analysis system to lose basic verifiability and programmability.

One hot June afternoon in Beijing, I set the Gree air conditioner to 26°C and placed a 3-yuan bottle of iced black tea from the refrigerator beside me. Hundreds of pages of NVIDIA and Microsoft financial report PDFs filled the monitor. While developing an AI financial report analysis system, the real difficulty was not reading the text, but dealing with the variety and unreliability of LLM output.

If you let the model freely answer “Summarize this company’s revenue, cash flow, and risk factors,” you usually get a paragraph that looks fluent but has no standard format. For example, it might confuse net cash flow from operating activities with free cash flow, or apply prior year data to this year. Worse, it might even consider FY2024 Q4 figures as full-year numbers.

Another fatal problem with free output is “format drift.” In automated analysis workflows, the backend database requires fixed field values for plotting or quantitative scoring. If the model outputs a markdown table one time, a key-value pair the next, and then wraps JSON externally with json ... next time, the downstream parser will throw JSONDecodeError.

Therefore, in serious investment research scenarios, we must forcibly narrow the model’s expression range, constraining it from “speaking freely” to “following format definitions.” This is the underlying value of JSON Schema. It’s not just a front-end display convention; it is the data contract for the entire AI financial report extraction pipeline.

Modules Needed for Structured Financial Report Output

Breaking down financial report output into four core modules—financial metrics, risk factors, management tone, and verification checklist—is crucial for achieving system stability.

A production-ready financial report extraction schema should not be a flat structure; it should be a multidimensional dictionary. In the AI financial assistant I designed, the overall schema framework is as follows:

{
  "company_metadata": {
    "company_name": "NVIDIA Corporation",
    "report_period": "FY2024",
    "reporting_currency": "USD"
  },
  "financial_metrics": {},
  "risk_factors": [],
  "management_tone": [],
  "review_questions": []
}

Every submodule here carries specific data constraint logic.

First, the company_metadata module verifies which company and period the report covers. An LLM’s attention can drift when it processes long documents. If it is analyzing an NVIDIA PDF but encounters competitor data in the context, it may mix the figures together. Constraining metadata at the outermost layer forces the model to identify the company and reporting period first.

Second, the financial_metrics module precisely extracts core financial metrics such as revenue, gross profit, operating expenses, net income, operating cash flow, and free cash flow.

Third, the risk_factors module captures risk warnings, which in financial reports often span dozens of pages, and extracts them into comparable feature dictionaries.

Fourth, the management_tone module tracks subtle wording changes by management regarding business outlook.

Fifth, the review_questions module consists of audit questions generated by AI, serving as a starting point for human review.

financial_metrics: Precise Definitions for Financial Metrics

Financial metric extraction should use structured objects that bind each value to its unit, period, source page, and source text.

Many developers, when designing financial metric schemas, prefer to use flat key-value pairs. For example, defining revenue as “revenue”: 60922. This is a major risk in practice.

The number 60922 alone carries too little context. Is the unit millions or thousands? Does the figure cover the year ended January 28 or a particular quarter? Which PDF page contains it? What is the original context?

To solve this problem, I define all members of financial_metrics as independent objects. Below is the specific modeling code for the metric schema using Python’s Pydantic library:

from pydantic import BaseModel, Field
from typing import Optional, List, Dict, Literal

class MetricDetail(BaseModel):
    value: Optional[float] = Field(
        default=None,
        description="Extracted metric value. If the financial report does not mention it, this must be null."
    )
    unit: Optional[str] = Field(
        default=None,
        description="Unit, such as USD million or CNY thousand; it must match the source text."
    )
    period: Optional[str] = Field(
        default=None,
        description="Corresponding financial period, such as FY2024 or Q4 FY2024."
    )
    source_page: Optional[int] = Field(
        default=None,
        description="Absolute page number for this data in the original PDF file."
    )
    source_text: Optional[str] = Field(
        default=None,
        description="Original sentence or table-row text from the financial report used to extract this data."
    )
    confidence: Literal["high", "medium", "low"] = Field(
        default="low",
        description="Confidence. Use high when the data comes from a table with clear context; use medium/low when inference is required or the context is ambiguous."
    )

class FinancialMetrics(BaseModel):
    revenue: MetricDetail = Field(description="Operating revenue")
    gross_margin: MetricDetail = Field(description="Gross margin")
    net_income: MetricDetail = Field(description="Net income")
    operating_cash_flow: MetricDetail = Field(description="Cash flow from operating activities")
    free_cash_flow: MetricDetail = Field(description="Free cash flow, defined as operating cash flow minus capital expenditures; look for an official disclosure or explicit statement")

With this structure, what we get is no longer just simple numbers, but a dataset with its own chain of evidence.

For example, when debugging Nvidia’s FY2024 financial report, if the model’s extracted gross margin is confused between Non-GAAP and GAAP, when we verify the source_text, we can clearly see whether it reads 76.0% of Non-GAAP or 72.7% of GAAP. By tracing source_page, analysts can jump to the corresponding PDF location with one click, making the AI’s output fully traceable.

Why Must Missing Fields Be Returned as null?

Preserving uncertainty in financial report analysis is far more important than forcing every field to contain a value. A null state is the first line of defense against invented formulas and hallucinated data.

In traditional software engineering, we always try to make each field have a definite value. But the logic behind AI financial report analysis is exactly the opposite.

What should we do if a company does not separately disclose the Free Cash Flow metric in its financial report, or does not provide details of capital expenditures that quarter?

If the Schema does not allow null, or the prompt lacks strict constraints, LLMs often try too hard to satisfy the request by calculating from the available data. A model may piece together free cash flow by subtracting spending on fixed assets and construction, or simply fabricate a proportional figure from the previous quarter’s data.

This is extremely dangerous because companies calculate free cash flow in different ways. Some deduct interest attributable to non-controlling shareholders, while others include proceeds from asset disposals. The AI’s seemingly clever assumptions directly undermine the rigor of the financial data.

Therefore, in the Schema definition, every field other than required metadata should default to None, and the description should repeatedly state:

[If the metric is not directly stated in the text or cannot be derived directly, it must be set to null. Do not estimate it with a formula or infer it from background knowledge.]

This way, when the frontend receives null, we know the information was not disclosed; it is not a false signal calculated by the LLM.

risk_factors: Do More Than Summarize Risk Factors

Risk factors need multidimensional, structured extraction—with enumerated types, severity levels, and source-evidence chains—to support comparisons across periods and companies.

Typically, the Item 1A (Risk Factors) section in a financial report contains tens of thousands of words. Most AI tools use a “concise summary” and finally output a paragraph summary.

This kind of summary is easy to read, but in reality, it erases the most valuable micro-changes.

If a semiconductor company changes its wording on geopolitical supply-chain disruption from “may have an impact” to “substantial delays have already occurred,” that signals a significant escalation. A natural-language summary may reduce both statements to “facing supply-chain risks,” erasing a subtle but critical warning.

We can extend the design of risk factors to multidimensional list elements:

class RiskFactor(BaseModel):
    risk_type: Literal[
        "customer_concentration",   # Customer concentration
        "supply_chain",             # Supply-chain risk
        "regulation",               # Regulatory risk
        "currency_fluctuation",     # Currency risk
        "litigation",               # Litigation risk
        "inventory_impairment",     # Inventory impairment risk
        "debt_solvency",            # Debt-servicing risk
        "competition",              # Industry competition
        "macro_economy",            # Macroeconomy
        "other"
    ] = Field(description="Risk category")
    risk_summary: str = Field(description="A concise, single-sentence summary of the substance of the risk")
    severity: Literal["critical", "high", "medium", "low"] = Field(description="Potential degree of negative impact on the company's performance")
    is_new_or_intensified: bool = Field(description="Whether the risk is new since the previous period or its wording has become materially stronger")
    evidence: str = Field(description="Specific factual statement in the financial report about how the risk has developed")
    source_page: int = Field(description="Page number for the risk in the financial report PDF")

With such a structured list, we can compare fields of the same company across different years.

If is_new_or_intensified returns True, the risk is new or has intensified in the current period. If risk_type is customer_concentration and the evidence says “a single customer accounts for 15%,” that specific figure can immediately alert the analyst.

management_tone: How Should Management Tone Be Structured?

Management-tone extraction must be based on objective changes in wording and context, not an LLM’s subjective classification of emotion.

In the MD&A (Management’s Discussion and Analysis) section, changes in the management’s wording represent their actual attitude toward the future.

Because LLMs have strong semantic capabilities, simply asking, “Is management’s attitude good or bad?” usually produces an answer such as “the attitude is optimistic.” Financial reports typically use positive words such as strong, progress, and commitment throughout their corporate messaging.

To extract the real attitude change, we need to adopt a “dual wording verification” structure in the schema:

class ToneAnalysis(BaseModel):
    topic: str = Field(description="Topic under discussion, such as demand, pricing_power, or capital_expenditure")
    current_statement: str = Field(description="Summary of management's statement for the current period")
    tone: Literal["positive", "neutral", "cautious", "negative"] = Field(description="Management's tone on the topic")
    change_from_previous: str = Field(description="Whether the stance has become more cautious, remained stable, or become more optimistic relative to previous known statements")
    evidence: str = Field(description="The most representative sentence in the source text supporting the tone assessment")
    source_page: int = Field(description="Corresponding page number")

For example, if management says, ‘Customers are becoming more cautious with their budgets,’ the model classifies the current tone as cautious. By comparing change_from_previous with the prior period, the system records the shift toward caution.

This approach limits subjective speculation by forcing the LLM to support its classification with source evidence and a specific change in wording.

review_questions: The Final Output Should Be a Review Checklist

AI should not serve as the final decision-maker in serious investment research, but should output a review checklist that analysts can manually verify.

I often see many so-called AI investment assistant systems that directly give a buy or sell conclusion at the end. In my architecture logic, this is an absolute no-go.

First, LLMs lack real-time macroeconomic awareness and can be misled by official wording in financial reports when industry dynamics are complex.

Second, directly giving trading decisions would push the system into extremely complex compliance and liability disputes.

The best endpoint for AI financial report analysis is not to provide conclusions but to provide a review checklist.

We can design a review_questions module, allowing AI, after extracting all the data, to automatically generate 3 to 5 critical review questions about the company:

class ReviewQuestion(BaseModel):
    question: str = Field(description="Key financial or operating question requiring human review")
    why_it_matters: str = Field(description="Why this question is critical to the current valuation or risk assessment")
    related_metrics: List[str] = Field(description="List of financial-metric keys associated with this question")
    source_pages: List[int] = Field(description="List of original financial report PDF pages the analyst should review")

For example, when AI detects that a company’s revenue has increased by 20%, but net cash flow from operating activities has dropped by 15%, it will generate a question like this on the review checklist:

[The company’s net income and operating cash flow are moving in different directions. Are accounts receivable or inventory accumulating abnormally? This could affect cash conversion in coming quarters. Review changes in Accounts Receivable on page 68 of the PDF and Inventory impairment provisions on page 72.]

In this way, AI does not overstep by making trading decisions for humans. Instead, it plays the role of an extremely sharp assistant, helping analysts compress thick financial reports into a few key clues worth investigating.

Common Pitfalls / Common Errors (Error Logs)

When enforcing JSON Schema constraints in production, the most common issues are incomplete JSON output and values outside the defined enums.

Due to the randomness of the model, even if the schema is defined, format crashes still occur. Below is a real error log snippet I captured from the backend logs:

ValidationError: 1 validation error for FinancialReportSchema
  risk_factors -> 0 -> severity
    Input 'high_severity' is not a valid enum member (should be one of: 'critical', 'high', 'medium', 'low')

Or this kind of JSON parsing error caused by the model output being truncated prematurely:

JSONDecodeError: Expecting ',' delimiter: line 42 column 18 (char 1289)

To handle these two types of errors, we cannot simply return them to the frontend. We need a two-layer fault-tolerance mechanism:

The first layer uses Pydantic to catch errors; The second formats the error details and feeds them back to the LLM for automatic retry (Auto-Retry).

Below is the complete Python code implementation for error interception and automatic retry:

import json
import openai
from pydantic import BaseModel, ValidationError

def extract_financial_data_with_retry(prompt: str, max_retries: int = 3) -> dict:
    client = openai.OpenAI()

    # Construct a system prompt that directs the model to output valid JSON
    system_instruction = (
        "You are an exceptionally rigorous financial report analysis assistant. You must output data strictly according to the required JSON Schema.\n"
        "If a field has no direct support in the source text, set it to null. Never invent any numbers or formulas.\n"
        "The output must be valid JSON and must not include a preface, afterword, or Markdown code fence."
    )

    current_prompt = prompt

    for attempt in range(max_retries):
        try:
            response = client.chat.completions.create(
                model="gpt-4o",
                messages=[
                    {"role": "system", "content": system_instruction},
                    {"role": "user", "content": current_prompt}
                ],
                response_format={"type": "json_object"},
                temperature=0.0  # Set temperature to 0 to ensure deterministic output
            )

            raw_output = response.choices[0].message.content

            # Attempt to parse and validate with Pydantic
            # Assume the complete FinancialReportSchema has been defined externally
            parsed_data = json.loads(raw_output)

            # If successful, return the parsed dictionary
            print(f"Data extracted successfully. Attempts: {attempt + 1}")
            return parsed_data

        except json.JSONDecodeError as je:
            print(f"Parsing failed on attempt {attempt + 1}: {str(je)}")
            # Assemble the error message to force the model to correct its output
            current_prompt = (
                f"{prompt}\n\n"
                f"Warning: Your previous output was not valid JSON. The parsing error follows:\n{str(je)}\n"
                f"Output it again and ensure there are no truncation or formatting issues."
            )

        except ValidationError as ve:
            print(f"Schema validation failed on attempt {attempt + 1}: {str(ve)}")
            # Extract the specific Pydantic error path and reason
            current_prompt = (
                f"{prompt}\n\n"
                f"Warning: Your JSON output failed Schema validation. The error follows:\n{ve.json()}\n"
                f"Compare the output strictly against the Schema definition, correct any invalid field values or enum members, and output it again."
            )

    # Enter the fallback path after the maximum number of retries
    print("Maximum retry count reached. Returning an empty data template and flagging the result for human review.")
    return {
        "status": "needs_human_review",
        "error": "Failed to extract valid structured data after multiple retries."
    }

By feeding Pydantic’s error details directly to the model as context, the LLM can often correct itself on the second or third attempt by changing high_severity back to high or supplying a missing closing brace.

Comparison Blocks

Constraining output with JSON Schema provides better format stability and lower hallucination rates than traditional system-prompt constraints or Function Calling, but slightly increases time to first token.

When developing an AI financial statement analysis system, I compared the actual performance of three different constraint schemes.

Below is a specific comparison of the differences among the three schemes in production practice:

• Solution 1: Pure System Prompt constraints

  • Format stability: extremely low. Under high concurrency or with very long contexts, models often omit fields or add explanatory text.
  • Hallucination rate: High. Without a schema contract, the model tends to invent missing values conversationally instead of returning null.
  • Initial latency: extremely low. No preparsing or constraint processing is required, so output begins fastest.
  • Code integration difficulty: low. You only need to write the prompt well, without relying on specific SDKs.

• Solution 2: LLM Function Calling (Tool Call) constraints

  • Format stability: Medium. It is accurate when extracting a single metric, but parameter mapping often becomes confused in nested lists or deeply nested objects.
  • Hallucination rate: Moderate. Defining required parameter properties can limit it, but null handling remains average.
  • Initial latency: Moderate. Since the model needs to output specific tool_calls structures, there is a certain generation overhead.
  • Code integration difficulty: Moderate. You need to parse message.tool_calls and handle the subsequent processing.

• Solution 3: Native JSON Schema constraints (Structured Output)

  • Format stability: extremely high. During decoding, the model applies token-level probability constraints so tokens that violate the schema are not generated. Format accuracy can exceed 99.9%.
  • Hallucination rate: extremely low. Combined with strict null definitions, it suppresses the model’s tendency to improvise.
  • Initial latency: High. Since decoding requires dynamically loading the schema and verifying the token status of each step, there is a slight increase in latency.
  • Code integration difficulty: High. You must define the Pydantic class precisely and serialize it into a schema dictionary that meets OpenAI or Gemini requirements.

In serious financial data-extraction pipelines, a modest increase in initial latency is an acceptable tradeoff. Data accuracy and 100% format stability are the foundations on which the system depends. Therefore, native JSON Schema constraints are the irreplaceable best option.

FAQ

Why does AI financial report analysis require JSON Schema?

JSON Schema constrains LLM output fields and formats, reducing hallucinations, missing fields, and format drift while returning revenue, profit, cash flow, risk factors, and review questions in a stable structure. Without a Schema, backend databases cannot use unstructured LLM output directly, and the system cannot make quantitative comparisons across companies.

Why must missing financial data output a null state?

If the financial report does not clearly disclose the data, the model must not guess. Returning null preserves uncertainty and prevents the model from fabricating numbers or drawing incorrect conclusions from crude calculations. This blocks the AI’s imagination at the data source and preserves the rigor of the data.

Why are the source_page and source_text fields mandatory?

They bind the AI’s extracted conclusions to the exact location and source passage in the original financial report PDF. If analysts suspect an error, source_page and source_text let them return quickly to the relevant passage for manual comparison, addressing the black-box, unverifiable nature of LLM output.

Can AI Financial Report Assistants Be Used Directly as Trading Signals?

No. AI financial report assistants are suitable only for information compression, structured extraction, and checklist generation. They lack expert-level macroeconomic awareness and real-time sentiment judgment. They cannot predict stock prices or directly provide buy or sell recommendations; they can only help human analysts reduce noise.

Keep reading

If you’re interested in other aspects of financial report analysis, you can continue reading my series of practical summaries:

• A practical guide to the preceding table-ingestion stage: Financial Report PDF Table Parsing in Practice: How to Prevent AI from Misreading Revenue, Cash Flow, and Risk Factors?
• An architectural breakdown of how the backend components work together: AI Financial Report Asynchronous Task Queue: Task Flow, Structured Extraction, and Review Workflow
• A guide to building this automated pipeline from scratch with open-source tools: 7 Steps to Analyze Financial Reports with AI: From PDF to Risk Checklist

Tool / AI Finance

Run the financial-report workflow instead of only reading about it

The AI Finance tool turns report extraction, source-page evidence and review steps into an interactive workflow. It compresses information and does not provide investment advice.

Open AI Finance → Browse developer tools Subscribe for updates