Practical Guide to AI Financial Report Assistant Task Queues: Designing PDF Parsing, LLM Calls, and Progress Updates

Problems This Article Addresses

• Why should large language model (LLM) financial report assistants avoid traditional synchronous HTTP endpoints for PDF parsing?
• How can a financial report analysis task be broken into a well-defined, multi-stage state machine so the processing pipeline does not become a black box?
• How should a production-grade job task table be designed? How should its fields be configured?
• How can low-latency processing progress feedback be elegantly implemented between the frontend and backend?
• How should differentiated exponential backoff retry strategies be designed when facing corrupted PDFs, LLM timeouts, rate limiting (429), and schema validation failures?
• How can API compute waste caused by users repeatedly uploading identical financial report files be prevented?
• Which low-confidence anomalous results must be physically intercepted and routed to a manual review queue?

Who Should Read This

• Backend Architects: Designing high-throughput, long-running backend processing systems for AI Agent workflows.
• AI System Developers: Working to resolve timeouts and queue backlogs in RAG systems or long-text extraction pipelines.
• Full-Stack Engineers: Planning to develop serious AI tools for investment research and auditing, seeking a closed-loop architectural solution.
• Independent Developers: Looking for system-level caching and deduplication designs to reduce LLM API costs and improve RAG response latency/experience.

1. Why AI Financial Report Assistants Cannot Process PDFs Synchronously

After returning from a bike ride along the Shili Hetan riverbed in Huaxi District, Guiyang, I sat in the Digital Everything Lab, took a sip of ice-cold cola, and immediately began refactoring the backend task pipeline for this AI financial report assistant. In many demos written by developers, a synchronous endpoint is created directly: the frontend uploads the PDF, the backend passes it to the LLM, and finally returns the result. This intuitive approach is a disaster in production environments.

When building an AI financial report assistant, if the interface remains synchronous, clients will almost 100% encounter 504 Gateway Timeout errors.

A standard financial report PDF (such as NVIDIA’s 10-K annual report) typically spans hundreds of pages and contains dense financial tables and explanatory notes. After receiving the file, the backend must first use a PDF parser to identify the page layout and extract complex multicolumn tables, then chunk the document and feed the context to the LLM in batches. The LLM must generate structured JSON that may span several pages. Network jitter, model API rate limits, and strict JSON Schema validation can make the process take seconds, tens of seconds, or even minutes.

What’s even more frustrating is that with a synchronous interface, users just stare at a spinning loading icon with no idea whether the backend is stuck parsing the PDF, calling the LLM, or has already failed because of a network interruption. Therefore, we must refactor synchronous calls into an “asynchronous task queue mode,” decouple upload from analysis, and give users status updates within seconds.

2. What Stages a Financial Report Analysis Task Should Be Split Into

The core of asynchronous design is establishing a fine-grained state machine.

We cannot simply set task states to “processing” and “completed.” We must break the financial report pipeline into detailed states so users and monitoring systems can see exactly where the task is at any time. In my engineering practice, I divided a financial report analysis Job into ten states and defined the following enum in the database:

CREATE TYPE job_status AS ENUM (
    'uploaded',                  -- File upload succeeded and the record was written to the database
    'queued',                    -- The task entered the Redis/RabbitMQ message queue
    'parsing_pdf',               -- A worker claimed the task and is extracting plain text from the PDF
    'extracting_tables',         -- Locating tables and extracting structured data
    'chunking_document',         -- Semantically chunking the text and preparing vector retrieval
    'running_llm',               -- Calling the large language model (LLM) to extract metrics and risk factors
    'validating_schema',         -- Strictly validating the model's JSON output against the Schema
    'generating_review_questions',-- Generating human-review questions from anomalous extraction results
    'needs_human_review',        -- Data confidence is low; processing has been halted pending analyst review
    'completed',                 -- The task is complete and its results have been persisted
    'failed'                     -- The task failed and an exception log was recorded
);

With this fine-grained state breakdown, the frontend can render a progress bar with specific steps such as “Parsing PDF,” “Extracting financial tables,” and “Running AI analysis,” which gives users much greater confidence in the process.

3. Job Data Table Design

A highly available task queue system must rely on a rigorously designed underlying database table for task persistence and state tracking. This is not only to enable task recovery in the event of a system outage, but also to support subsequent performance auditing and billing settlement.

Below is the DDL for the financial_analysis_jobs task table that I designed in PostgreSQL:

CREATE TABLE financial_analysis_jobs (
    job_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    user_id VARCHAR(100) NOT NULL,                                          -- ID of the user who submitted the task
    file_id UUID NOT NULL,                                                  -- ID of the associated file
    status job_status NOT NULL DEFAULT 'uploaded',                           -- Current task status
    progress INT NOT NULL DEFAULT 0,                                        -- Percentage progress from 0 to 100
    current_step VARCHAR(50) NOT NULL DEFAULT 'upload_success',             -- Current atomic processing step
    error_code VARCHAR(50),                                                 -- Standard error code on failure
    retry_count INT NOT NULL DEFAULT 0,                                     -- Number of retries so far
    model_version VARCHAR(50) NOT NULL,                                     -- Version of the large language model used for this task
    prompt_version VARCHAR(50) NOT NULL,                                    -- Version of the prompt used for this task
    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
    completed_at TIMESTAMP WITH TIME ZONE
);

-- Index frequently queried fields to maintain query speed under high concurrency
CREATE INDEX idx_jobs_user_id ON financial_analysis_jobs(user_id);
CREATE INDEX idx_jobs_status ON financial_analysis_jobs(status);
CREATE INDEX idx_jobs_file_id ON financial_analysis_jobs(file_id);

IV. Why Separate PDF Parsing and LLM Calls

At the system architecture level, the “PDF Parser” and the “LLM Caller” must be implemented as independent microservices or separate task queues.

The fundamental reason for decoupling is that PDF parsing and LLM calls are two entirely different types of computational tasks, with completely different failure rates and error characteristics:

• PDF parsing is a CPU-intensive task that primarily consumes local server computing resources. Errors usually occur due to corrupted PDFs, format incompatibilities, or large embedded scanned images;
• LLM calls are network and I/O-intensive tasks that primarily consume external API quotas (tokens). Errors typically stem from rate limiting by the model provider, network connection timeouts, or output formats that do not comply with JSON Schema constraints.

If they are bundled into one synchronous task script, a failure at the LLM step would force us to repeat the time-consuming PDF parsing step, wasting server resources and time. By separating them, we can temporarily store the parsed plain text and table data in object storage or Redis. If an LLM call fails, we only need to reload the cached text and call the LLM again, enabling retries within seconds.

V. Progress Feedback Mechanism Comparison: Polling vs SSE vs WebSocket

Once a task enters asynchronous processing, the frontend needs to retrieve the processing status in real time. In practical engineering implementations, there are three common communication schemes. Below is a comparison of their development costs, determinism, and real-time user experience.

Evaluation Dimension	Polling (Short Polling)	Server-Sent Events (SSE)	Bidirectional Communication (WebSocket)
Development Cost	Very Low (Frontend just needs a simple setInterval)	Moderate (Backend must support streaming response)	High (Requires managing long-lived connection lifecycles and heartbeat checks)
Network Overhead	High (Frequent HTTP three-way handshakes and header transmissions)	Very Low (Establishes a unidirectional long-lived connection, pushes data on demand)	Very Low (Establishes a persistent bidirectional channel with low overhead)
Server Load	Moderate (Can easily overwhelm the database under high concurrent access)	Low (Requires optimizing file descriptor limits when connection counts are high)	High (Maintaining long-lived connections consumes memory and requires a distributed gateway)
SPA Route Adaptation	Simple (Just destroy the timer upon navigation)	Simple (Close the EventSource upon route switching)	Complex (Requires handling connection drops and automatic reconnection upon navigation)
Recommended Approach	Suitable for v1 rapid deployment and scenarios with low concurrency requirements	Suitable for unidirectional progress updates; the most recommended alternative	Suitable for interactive AI chat interfaces requiring complex bidirectional interactions

In the architectural design for the AI Financial Report Assistant, I recommend starting with either “Polling” or “Server-Sent Events (SSE)” for the first version. Since financial report analysis progress is entirely unidirectional (from 0% to 100%), SSE provides a very smooth progress bar update experience without the heavy burden of maintaining bidirectional connection management and complex gateway clusters like WebSocket does.

VI. Failure Retry Strategy Design

If fault tolerance is not designed into asynchronous tasks, they will fail spectacularly in production environments. We must design targeted Exponential Backoff retry strategies based on different error types, rather than blindly retrying in an infinite loop.

Below is the Python fault-tolerance retry module I designed for the financial analysis Worker:

import time
import random
from typing import Callable, Any

def execute_with_exponential_backoff(
    task_fn: Callable[[], Any],
    max_retries: int = 3,
    initial_delay: float = 1.0,
    backoff_factor: float = 2.0
) -> Any:
    """
    Retry temporary errors such as LLM rate limits and timeouts with exponential backoff, and fail fast on hard infrastructure failures.
    """
    delay = initial_delay
    for attempt in range(max_retries):
        try:
            return task_fn()
        except Exception as e:
            err_msg = str(e).lower()
            current_attempt = attempt + 1

            # 1. Irreversible infrastructure failure: fail fast without pointless retries
            if "pdfsyntaxerror" in err_msg or "corrupted" in err_msg:
                print(f"[-] Infrastructure failure detected: {e}. Failing fast and marking the task as failed.")
                raise ValueError("ERR_PDF_CORRUPTED") from e

            # 2. Strict JSON Schema validation failure: possibly a prompt defect; retry at most once
            if "validationerror" in err_msg or "json_schema" in err_msg:
                if current_attempt >= 2:
                    print(f"[-] Schema validation failed repeatedly. Retry limit reached; routing to human review.")
                    raise ValueError("ERR_SCHEMA_VALIDATION_FAILED") from e
                # Wait briefly before retrying to give the model one chance to correct itself
                time.sleep(1.0)
                continue

            # 3. Temporary network failure or LLM rate limit (429 / Timeout): exponential backoff + random jitter
            if "429" in err_msg or "ratelimit" in err_msg or "timeout" in err_msg or "deadline" in err_msg:
                if current_attempt == max_retries:
                    print(f"[-] Network/rate-limit retries reached the maximum of {max_retries}; task failed.")
                    raise e

                # Add random jitter so concurrent workers do not retry simultaneously and overwhelm the LLM gateway
                jitter = random.uniform(0.1, 1.0)
                sleep_time = (delay * pow(backoff_factor, attempt)) + jitter
                print(f"[!] Temporary error encountered: {e}. Backoff attempt {current_attempt}; waiting {sleep_time:.2f} seconds...")
                time.sleep(sleep_time)
            else:
                # 4. Unknown business exception: raise immediately without retrying
                raise e

    raise TimeoutError("ERR_MAX_RETRIES_EXCEEDED")

7. How to Handle Result Caching and Duplicate Uploads

In a serious production environment, parsing large financial report PDFs and calling LLMs is expensive. If a user uploads the same PDF again, or several users upload the same NVIDIA 2024 annual report on the same day, reparsing it every time is essentially handing money to the LLM vendors.

We need to implement content-based deduplication and result caching at the architecture level.

When a user uploads a PDF, we first read the file’s binary stream at the frontend or API gateway layer and compute its SHA-256 hash. Then, we query the financial_reports_cache table in the database:

CREATE TABLE financial_reports_cache (
    file_hash CHAR(64) PRIMARY KEY,                                         -- SHA-256 hash of the file
    file_name VARCHAR(255) NOT NULL,
    parsed_text_url VARCHAR(512),                                           -- Path to plain text temporarily stored in object storage
    extracted_kpis JSONB,                                                   -- Cached extracted KPI results
    model_version VARCHAR(50) NOT NULL,
    prompt_version VARCHAR(50) NOT NULL,
    cached_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
);

The deduplication logic works as follows:

1. Compute the SHA-256 hash of the uploaded file;
2. If the hash matches, and the model version, prompt version, and current system configuration are all consistent, skip the queuing and analysis steps. Directly clone the cached result to generate a new Job record with its status set to completed;
3. The frontend can display a popup within 1 second saying “Detected existing parsing results, loading for you…” This instantly elevates the user experience from “waiting several minutes” to “sub-second loading,” while also saving our system a significant amount of token costs.

8. Which Tasks Enter the Manual Review Queue

In serious financial settings such as investment research and auditing, we cannot rely entirely on LLMs to achieve 100% accuracy. If the model output triggers a predefined anomaly threshold, the system must halt automated processing, set the task status to needs_human_review, and route it to a dedicated human-review interface.

In my queue logic, the following six scenarios will be flagged by the evaluation engine as low-confidence and intercepted, routing them to the manual queue:

• Missing evidence page number: In the returned financial KPI metrics, the source_page field is null or less than or equal to 0;
• Evidence verification failed: The model returned the original evidence text, but when we search for that text in the corresponding PDF source_page, the contains validation fails;
• Extreme anomalies in financial metrics: Year-over-year data fluctuates by more than 500%, or data violating basic financial common sense appears, such as “negative revenue”;
• The LLM output confidence score is below 0.85;
• JSON Schema strict validation fails, and even after exponential backoff retries, a compliant format cannot be generated;
• Extracted Risk Factors contain sensitive words from the system blacklist or exhibit severe semantic conflicts.

For specific UI interactions and queue interception mechanisms regarding manual review, refer to what I previously summarized in the lab: 👉 Manual Review Queue Design — learn how to establish a true double-blind verification safety chain between AI and human analysts.

9. Integrating with the AI Financial Report Analyzer

This asynchronous task queue solution serves as the foundation for the AI Financial Assistant. At the XBSTACK Lab, we built a complete technical closed-loop based precisely on this system:

• To try it directly, open the AI Financial Report Analyzer: 👉 AI Financial Assistant.
• If you’re unfamiliar with the tool’s positioning, start here: 👉 What is the AI Financial Assistant?.
• To understand the usage workflow, see: 👉 7 Steps to Analyze Financial Reports with AI.
• For system architecture design details, check: 👉 Overall Architecture of the AI Financial Analysis System.
• For PDF parsing details, read: 👉 Practical Guide to Parsing Financial Report PDF Tables.
• For structured output, see: 👉 Practical Guide to LLM JSON Schema.
• For quality evaluation, refer to: 👉 Golden Dataset Evaluation Framework.
• For automation workflows, continue reading: 👉 AI Workflow Automated Task Processing Methods.

Frequently Asked Questions

Why is it not recommended to use Redis + Celery’s default queue directly for PDF parsing?

Because Celery’s default prefetch mechanism tends to cause severe starvation or memory crashes on Worker nodes when tasks are long-running and consume highly uneven resources. PDF parsing is CPU-intensive, while LLM calls are I/O-intensive. If both task types share one Celery queue, a few large PDF parsing jobs can saturate every Worker and cause memory leaks, leaving lightweight LLM calls stuck behind them. Independently named RabbitMQ queues are strongly recommended for workload isolation.

How should the processing progress (progress field) be calculated and updated in the backend?

We don’t need to implement highly precise dynamic time estimates, as those are physically inaccurate. A “milestone-based progress update” is recommended instead. For example: successful file upload at 10%, PDF parsing completed at 30%, table extraction finished at 50%, LLM inference in progress at 80%, schema validation passed at 95%, and fully archived in the database at 100%. This phased, discrete step approach is the most stable to implement and aligns best with users’ intuitive perception.

How do we prevent long connections (SSE / WebSocket) from exhausting the file descriptors of the backend Express or Node service under high concurrency?

When SSE pushes progress updates while many users have pages open, each user occupies a persistent TCP connection. On the Node.js backend, run ulimit -n 65535 at the operating-system level to raise the maximum file-descriptor limit. Also configure connection timeouts. For example, if a task remains queued for more than 10 minutes with no status update, or when a user leaves the page, the frontend should proactively call eventSource.close() to release the server-side connection and avoid keeping it open unnecessarily.

Continue Reading

• 👉 LLM JSON Schema in Practice: How to Make AI Consistently Output Financial Report Revenue, Cash Flow, and Risk Factors — Learn how strict Schema constraints reduce extraction-format failures at the first stage.
• 👉 Practical Guide to Parsing Financial Report PDF Tables: How to Prevent AI from Misreading Revenue, Cash Flow, and Risk Factors? — Dive deeper into the PDF parsing stage to resolve row/column misalignment issues caused by cross-page table merging and borderless designs.

Tool / AI Finance

Run the financial-report workflow instead of only reading about it

The AI Finance tool turns report extraction, source-page evidence and review steps into an interactive workflow. It compresses information and does not provide investment advice.

Open AI Finance → Browse developer tools Subscribe for updates