LLM Analysis of SEC Filings: Methods, Limits, Controls

Large language models can read an SEC filing faster than any analyst, but they cannot be treated as a system of record. Reliable LLM analysis of SEC filings depends on three design decisions: constraining the model to retrieved source text, separating evidence extraction from interpretation, and logging every claim back to a filing, section and period. Without those controls, fluent output is easily mistaken for verified output.

This article sets out how LLM-based filing analysis is structured, where it fails in documented ways, how EDGAR access constraints shape any pipeline, and how the same discipline applies to the question the Lacuna methodology addresses: whether what a company says in its narrative is matched by what it delivers.

TL;DR

• LLMs perform acceptably on isolated extraction from a single filing, but accuracy degrades on multi-year tracking and cross-company comparison, which is where most analyst work sits.
• Retrieval-augmented generation (RAG), hybrid retrieval and source citation are the primary architectural defenses against fabricated figures.
• Regulators state that AI-generated information must be verified against the original filings; any workflow should preserve a path back to the primary document.
• The prompt persona can change the conclusion drawn from identical evidence, so extraction and judgment should be separate stages.
• The harder analytical problem is cross-channel: comparing 10-K language with earnings calls, press releases and proxy statements.

Table of Contents

• 1. What LLM analysis of SEC filings covers
• 2. Where accuracy holds and where it breaks
• 3. Architecture: RAG, hybrid retrieval and reranking
• 4. Separating evidence from interpretation
• 5. EDGAR access: constraints that shape the pipeline
• 6. Why narrative-versus-filing consistency is the harder problem
• 7. Our perspective
• 8. Evaluating a filing-analysis workflow: criteria
• 9. A worked example of a consistency check
• 10. Where Lacuna Index fits
• FAQ
• Sources

1. What LLM analysis of SEC filings covers

SEC filings are a mixed corpus. Periodic reports (10-K, 10-Q) combine narrative sections such as business description, risk factors and management's discussion with audited financial statements and XBRL-tagged data. Current reports (8-K) are event-driven and short. Proxy statements (DEF 14A) carry governance and compensation disclosure. Each document type rewards a different analytical approach.

LLM use cases generally fall into four categories:

The difficulty gradient matters because published vendor and hobbyist demonstrations tend to showcase the first two rows. Practitioner workflows, including equity research, governance review and investigative reporting, depend heavily on the last two.

2. Where accuracy holds and where it breaks

Research indicates that hallucination rates in financial data extraction are generally under 2% for simpler tasks. That figure should not be generalized. The Fin-RATE benchmark, built specifically on SEC filings, measured accuracy drops of 18.6% and 14.35% when models moved from single-document reasoning to longitudinal tracking and cross-entity analysis. The benchmark's authors point to weaknesses in temporal alignment and comparative logic across reporting periods.

The practical reading is that a model may correctly quote a revenue figure from one filing and still misattribute it to the wrong fiscal year when asked to describe a trend. Standardized question-answering benchmarks often test isolated passages, whereas analyst workflows require synthesizing interlinked evidence across documents.

Failure modes worth testing for

• Period misalignment: a figure from a prior-year comparative column is reported as current.
• Restatement blindness: an original figure is cited without the amended filing that superseded it.
• Definition drift: a non-GAAP metric is compared across issuers that define it differently.
• Boilerplate inflation: generic risk-factor language is treated as a company-specific signal.
• Persona spillover: the conclusion shifts with the framing of the request, discussed in section 4.

The SEC and FINRA, through an investor alert at Investor.gov, advise that investors should not rely solely on AI-generated information, noting that outputs can be hallucinated or based on outdated data, and that independent verification against official filings is required. For an institutional workflow, this converts into an engineering requirement: every model-generated statement must resolve to a document a reviewer can open.

3. Architecture: RAG, hybrid retrieval and reranking

The principal architectural defense is retrieval-augmented generation. Rather than letting the model answer from training memory, a RAG system retrieves passages from a curated corpus of filings and constrains the answer to that text. The survey literature on hallucination treats retrieval grounding as the main mitigation because it forces the model to cite specific source documents.

For filings, a basic vector search is usually insufficient. The Fin-RATE authors recommend pipelines that combine keyword (sparse) and semantic (dense) retrieval, followed by a reranking model, before passages reach the LLM. The reasoning is straightforward: filings contain exact identifiers, such as line-item names, dollar amounts and section references, that keyword search handles well, and paraphrased concepts, such as a company describing the same risk in different words across years, that semantic search handles well.

A minimal pipeline

1. Ingest the filing and its metadata (accession number, form type, period of report, filing date).
2. Segment by item and section rather than by arbitrary token windows, so that a citation points to a recognizable location.
3. Index text for hybrid retrieval and keep XBRL facts in a structured store for reconciliation.
4. Retrieve and rerank passages relevant to a narrowly defined question.
5. Extract claims as structured records: statement, quotation, location, period.
6. Validate numeric claims against tagged XBRL values and flag mismatches.
7. Log the full chain so a reviewer can reproduce any output.

A complementary discipline for structured data is described in traceable SEC filing analysis with XBRL grounding, which treats tagged facts as the check on textual extraction.

4. Separating evidence from interpretation

A 2026 study on user-context spillover found that the same evidence from an SEC filing can lead to different conclusions depending on whether the model is prompted as an investor, an analyst or a general assistant. The recommended mitigation is to separate evidence extraction from personalized output generation.

In practice this means two stages with different rules:

• Stage one, extraction: neutral, persona-free, deterministic where possible, producing citations and quotations only.
• Stage two, interpretation: any scoring, ranking or narrative synthesis, operating on the frozen output of stage one and logged separately.

The separation also makes disagreement tractable. When two reviewers dispute a conclusion, they can examine whether the dispute lies in the evidence (stage one) or in the weighting (stage two).

5. EDGAR access: constraints that shape the pipeline

Any filing pipeline sits on top of EDGAR. The SEC's EDGAR APIs provide JSON access to submissions by company and to extracted XBRL data. Access is governed by a fair access policy of 10 requests per second. Practitioners often underestimate that the limit applies to the originating IP address, not to each user or process, so several parallel workers behind one address share a single budget. Systems that ignore this commonly encounter HTTP 429 responses.

Two implementation points follow:

• Use a shared token-bucket throttle across all workers, not a per-worker delay.
• Apply exponential backoff on 429 responses rather than immediate retries.

The SEC also publishes general guidance on accessing EDGAR data, including the requirement that registrants file electronically. Because EDGAR is the authoritative record, a pipeline should store the accession number and original URL for every ingested document, so that downstream citations resolve to the SEC's own copy.

6. Why narrative-versus-filing consistency is the harder problem

Most published LLM demonstrations summarize a single filing. The analytically valuable question is different: does a company's account of itself remain consistent across channels and over time?

The SEC has shown increasing interest in this. In January 2025, the agency brought a settled action against Presto Automation, which the D&O Diary reported may be the first AI-washing enforcement action against a reporting company. A practitioner analysis notes that the SEC alleged misleading statements about AI capabilities and also a lack of established processes for drafting, reviewing or approving periodic reports. The enforcement theme is therefore disclosure controls, not only wording.

Counsel commentary points the same direction. A Dentons discussion of AI and cyber disclosure observes that companies often underestimate the risk when public communications are inconsistent, for example between SEC filings and earnings calls or investor presentations. On the rules themselves, there is no standalone AI disclosure requirement; the SEC's Investor Advisory Committee recommendation frames the issue as materiality under existing Regulation S-K, including board oversight and the avoidance of generic, boilerplate statements.

For analysts, inaccurate or unsupported statements carry consequences beyond AI. Commentary on SEC enforcement for inaccurate statements in adjacent regulated sectors illustrates how exam and enforcement exposure attaches to what firms assert versus what they can evidence. The lesson for LLM-based review is that the unit of analysis should be the claim, tied to a date, a channel and a document.

Claim-tracking as a schema

A workable claim record includes:

• the verbatim statement and the speaker or filing section;
• the channel (10-K, 8-K, earnings call, press release, proxy statement);
• the date and fiscal period referenced;
• the commitment type (quantified target, qualitative aspiration, hedged expectation);
• the later disclosure that confirms, revises or omits it.

The linked guides on monitoring disclosure consistency across filings and hedging language detection expand on how commitment types and qualifiers are classified. Related patterns are covered in engineered disclosure patterns.

7. Our perspective

We approach LLMs in filing analysis as an extraction and indexing aid inside a methodology, not as the methodology. The scoring logic at Lacuna Index is built on a fixed, documented process applied identically to every covered company, using only public records: earnings calls, SEC filings, press releases, proxy statements and public statements. No insider access and no proprietary data are involved, and scoring is accompanied by governance logs so that a reader can trace any score to its evidence.

Three convictions follow from that design. First, cross-issuer comparability matters more than model sophistication: a score is useful only if the same rules produced it for every company in the sector. Second, the gap between narrative and delivery is measurable only when claims are stated with enough precision to be tested later, which is why unasked questions and engineered disclosures are as informative as the claims themselves. Third, an output that cannot be audited should not influence an investment or governance decision, regardless of how fluent it reads.

Our methodology walkthrough sets out the process in detail.

— Glen

8. Evaluating a filing-analysis workflow: criteria

Whether the workflow is built in-house or obtained from a research provider, the following criteria separate auditable systems from demonstrations.

For teams using filings in investigations, the guide to turning EDGAR filings into auditable evidence addresses evidentiary handling, and the piece on validating corporate claims in public disclosures addresses the testing step.

9. A worked example of a consistency check

Consider a hypothetical issuer, illustrative only. In an earnings call, management describes a program as on track to deliver a quantified result within two fiscal years. The following 10-K describes the same program in the business section using the same language, while the risk factors describe the underlying capability in conditional terms. A year later, the MD&A drops the quantified reference and substitutes a qualitative description.

A claim-tracking pipeline would record four items: the original quantified commitment (call, date), the consistent restatement (10-K business section), the conditional framing (risk factor, with a flag for hypothetical language), and the omission (later MD&A, with a flag that a prior commitment is no longer referenced). A single-document summarizer would see none of this, because each document is individually unobjectionable. The signal lies in the sequence, which is precisely the longitudinal reasoning where benchmark accuracy declines.

This is also why human review remains the final stage. A reviewer confirms that the omission is substantive and not a reclassification, and that conditional language is not simply a legitimate risk description.

10. Where Lacuna Index fits

For institutional investors, governance advocates, proxy advisors and financial journalists who need the output of this kind of analysis without building the pipeline, Lacuna Index applies a standardized method to public records and publishes the result as forensic reports with execution and narrative scores. Sector benchmarks are free to read, for example Financials, and per-company analysis is available to subscribers through the benchmarks section. Subscription plans, Cohort Pulse and Sector Sweep, are described on the pricing page.

FAQ

Can an LLM reliably read a 10-K?

For isolated extraction from a single filing, error rates can be low, but reliability falls on multi-year and cross-company tasks according to the Fin-RATE benchmark. Reliability should be established by checking outputs against the filing and XBRL data, not assumed from the model.

How does RAG reduce hallucination in filing analysis?

RAG retrieves passages from a curated set of filings and constrains the model to answer from them with citations, instead of from training memory. It does not eliminate error, since retrieval can miss the right passage, so hybrid retrieval, reranking and citation checks are normally added.

What rate limit applies when downloading from EDGAR?

The SEC's fair access policy sets a maximum of 10 requests per second, applied by IP address. Parallel workers behind one address share that budget, so a shared throttle and backoff on 429 responses are advisable.

Does the SEC require companies to disclose AI use separately?

As of 2026 there is no standalone AI disclosure requirement. The SEC's focus is on whether AI is material to operations, strategy or financial prospects under existing Regulation S-K, and on whether statements about it are accurate and not boilerplate.

Should model output be used directly in an investment decision?

The SEC and FINRA advise against relying solely on AI-generated information and recommend verification against original filings. Model output is best treated as an index into the filings, with a reviewer confirming each cited passage.

Sources

• Fin-RATE: benchmark for LLMs on SEC filings (arXiv)
• Investor.gov: Artificial intelligence and investment fraud alert
• Hallucination mitigation and retrieval grounding (arXiv)
• User-context spillover in LLM financial analysis (arXiv)
• SEC: EDGAR Application Programming Interfaces
• SEC: Accessing EDGAR Data
• The D&O Diary: SEC AI-washing action against Presto Automation
• JD Supra: SEC charges AI-washing at Presto
• Dentons: Securities disclosures in the age of AI and cyber risk
• SEC: Investor Advisory Committee AI disclosure recommendation

Recommended

• Traceable SEC Filing Analysis for U.S. Researchers With XBRL Grounding
• Monitor Disclosure Consistency Across Filings: 2026 Guide
• How to Validate Corporate Claims in Public Disclosures
• Prove Management Claims: Four Step U.S. Disclosure Controls Assessment