Automated earnings call narrative analysis converts unstructured transcript language into structured, auditable scores that quantify tone, uncertainty, guidance, and specificity, each one linkable to the exact quote that produced it. We use this approach to surface predictive signals that augment quantitative models and to give analysts evidence they can defend in front of a portfolio committee. Financial analysts, quant teams, and governance researchers rely on it when sentiment alone fails to explain market reaction.
TL;DR:
- Score uncertainty, guidance specificity, novelty, and topic mix alongside tone; research finds these features explain market responses better than sentiment alone.
- Keep prepared remarks separate from question and answer segments, and score full analyst exchanges so qualifiers remain attached to the claims being assessed.
- Retain each source passage, model version, and prompt with its score; linked quotations let reviewers verify explanations and reproduce results later.
- Test added value against a quantitative baseline using tests that hold out entire companies and compare sectors, then send borderline scores for analyst review.
- Compare call commitments with later filings and delivered results to identify overpromising, omitted commitments, or contradictions that scoring calls alone cannot detect.
Table of Contents
- Building the pipeline: from raw transcript to scored evidence
- Which narrative dimensions actually predict outcomes
- Choosing models and explainability methods that hold up in production
- Testing whether narrative signals actually add value
- How forensic analysis links narrative claims to public filings
- What the research gets right, and what practitioners still underrate
- Putting audit-traceable narrative scoring to work
- FAQ
- Sources
Building the pipeline: from raw transcript to scored evidence
A production-grade pipeline moves through distinct stages, and skipping any stage erodes the traceability that makes the output usable for investment or governance decisions.
- Source and transcribe: pull transcripts from the issuer's own investor relations page or a licensed provider, preserve speaker attribution and timestamps, and reconcile stated figures against the preceding press release.
- Segment: separate prepared remarks from the question-and-answer session before any scoring, since the two sections carry different informational weight.
- Chunk: split text into passages that preserve enough context (typically a full analyst exchange, not a single sentence) so a model never scores a claim that is stripped of its qualifier.
- Embed and index: convert chunks into vector representations and store them in a retrieval index alongside metadata (speaker, date, fiscal quarter).
- Query via retrieval-augmented generation: issue structured prompts that return both a score and the supporting quote span, never a score alone.
- Log provenance: record the exact chunk, model version, and prompt used for every score so the result can be reproduced later.
Regulation FD treats conference calls as a public-disclosure channel whenever they carry material information, and the SEC's guidance recommends reasonable advance notice and making transcripts or replays available, which is part of why these transcripts are reliably accessible for systematic analysis in the first place.
Pro Tip: Store the raw transcript chunk alongside every generated score, not just the score itself; without the underlying text, a reviewer cannot verify why the model produced the number it did.
Which narrative dimensions actually predict outcomes
Not every textual feature earns a place in a production model. The dimensions worth extracting share one trait: each has some empirical link to a realized outcome, not just a correlation with mood.
- Tone: the net positive or negative valence of language, typically the entry point for most systems but the weakest predictor on its own.
- Uncertainty: hedging language, conditional phrasing, and qualified commitments, which analysts tend to underweight relative to its informational value.
- Guidance specificity: whether forward statements include numeric ranges, timeframes, or concrete milestones versus vague aspiration.
- Novelty: how much a given call's language departs from the issuer's own prior-quarter script, flagging genuine news versus repeated boilerplate.
- Topic composition: the distribution of subject matter discussed, which recent research treats as a stronger signal than raw sentiment.
Large-sample research on earnings call narratives finds that topic-level attributes and textual attributes materially increase explained variance for market responses and implied-volatility changes, often outperforming sentiment scores alone, according to research on narratives contextualizing numeric disclosures.
Operationalizing these dimensions means choosing a scoring method: lexicon-based counts are fast but brittle; LLM rubric scoring (asking a model to rate a passage against a defined scale with a justification) offers more nuance at higher cost; and topic modeling or embedding clustering groups thematically similar language across calls without requiring predefined categories. Whichever method is chosen, the output should separate prepared-remarks scores from Q&A scores, since conflating them obscures which section actually drove the signal.
Choosing models and explainability methods that hold up in production
The choice between retrieval-augmented generation and direct LLM scoring depends on the question being asked. Direct scoring works when a passage is already isolated and the task is rating it against a rubric; RAG becomes necessary when the question spans multiple calls or requires pulling the specific passage that supports a claim before scoring it.
- Embedding model and similarity metric: domain-tuned embeddings outperform general-purpose ones on financial jargon, and cosine similarity remains the standard default for retrieval ranking.
- Chunk size: passages long enough to preserve a full analyst question and answer, short enough that a single topic does not get diluted across multiple chunks.
- Explainability: feature-attribution methods such as SHAP quantify which input tokens or phrases moved a score, which matters when a committee asks why a company received a low execution rating.
- Quote-level provenance: every explainability output should resolve to the literal quoted phrase, not a paraphrase, so a human reviewer can verify the attribution in seconds.
- Counterfactual morphing: systematically altering a passage's tone or specificity while holding other content fixed isolates which dimension actually drove a model's output, a near-experimental check before trusting a signal in production.
Research using counterfactual text-morphing and Shapley-style attribution finds that analysts tend to overreact to optimistic sentiment while underweighting uncertainty language, a pattern that automated systems can correct for by surfacing uncertainty explicitly rather than folding it into a single tone score, per research quantifying contextual narratives using language models.
Pro Tip: Treat counterfactual morphing as a validation step, not a production feature; run it periodically to confirm a dimension still carries predictive weight, especially after a major macro shock reshapes how analysts read language.
Trade-offs worth planning for up front include latency (RAG queries are slower than cached scores), update cadence (quarterly re-indexing versus continuous ingestion), and model drift, which tends to accelerate around earnings seasons with unusual macro volatility and requires scheduled re-calibration rather than a one-time calibration.
Testing whether narrative signals actually add value
A narrative score is only useful if it improves on what a quantitative model already knows, and that has to be tested directly rather than assumed.
- Measure incremental explanatory power: add narrative scores to a baseline model and track the change in R-squared for event-window returns or implied-volatility changes, rather than reporting raw correlations.
- Backtest with issuer-level holdouts: split training and test sets by company, not by random transcript chunk, since models that memorize firm-specific scripted language will overstate performance on repeated issuers.
- Check statistical significance across sectors: a signal that predicts outcomes in one sector may not generalize, so sector-level splits matter as much as firm-level ones.
- Set thresholds and human review triggers: flag scores near a decision boundary for analyst review rather than acting on them automatically.
Large-sample studies of narrative content find that firm-level evaluation designs, which hold out entire issuers rather than random transcript samples, are necessary to avoid overfitting to scripted or repeated language, a point emphasized in research on narratives contextualizing numeric disclosures. When presenting results to a portfolio manager, a short scorecard with linked quote evidence communicates more than a raw statistical summary ever will.
How forensic analysis links narrative claims to public filings
Scoring a transcript in isolation tells only part of the story. A forensic approach ties every narrative claim back to what the issuer actually filed, so a stated commitment can be checked against delivered results rather than left to stand on its own.
- Executive language from a call is matched to the corresponding SEC filing, press release, or proxy statement that either confirms or contradicts it.
- An audit-traceable output pairs the linked quote with its supporting filing, an execution metric derived from that filing, and a plain-language interpretation of the gap between the two.
- Common discrepancies surface as overpromising (guidance that outpaces delivered results), omission (a prior commitment that disappears from later calls without explanation), or contradiction (statements across calls that cannot both be true).
Our methodology walkthrough for readers who want the underlying scoring logic is detailed in our earnings call credibility audit framework, and we cover how selective disclosure patterns surface in public records in our guide to detecting selective disclosure.
What the research gets right, and what practitioners still underrate
The academic literature has made a convincing case that topic composition and uncertainty language carry more predictive weight than blunt sentiment scores, and production teams that still lean on a single tone number are leaving signal on the table. That much is well supported.
What gets underrated is the gap between a narrative claim and its filed outcome. Most narrative-analysis work treats a transcript as a closed system, scoring tone or guidance against the call itself rather than against what the company later delivers. A high-confidence call followed by a quietly revised filing is a different event than a high-confidence call followed by matched results, yet both would score identically under a transcript-only model.
The practical priority for anyone building this capability is sequencing: get provenance and quote-level traceability working before optimizing for scoring accuracy. A less accurate score with a verifiable quote behind it is more useful to an analyst than a marginally better score that cannot be checked.
— Glen
Putting audit-traceable narrative scoring to work
We built Lacuna Index to run the pipeline described above as a forensic product rather than a research exercise: every score we publish traces back to a quoted transcript passage and a corresponding public filing, scored through the same standardized methodology across every issuer we cover.
We offer subscription plans that provide institutional users with narrative-versus-delivery scoring across coverage groups and per-company forensic reports with execution scores linked to filings and quotes. Free sector benchmarks are available for readers who want to see the scoring in practice before subscribing, including our Financial Services benchmarks and Industrials benchmarks. For teams that need to convert narrative scores into client-ready spreadsheets quickly, Sheet Analyzer turns exported scores into formatted reports without manual rebuilding.
Review our pricing and plan details or read the full methodology walkthrough to see whether our approach fits your coverage needs.
FAQ
Are earnings calls public record?
Earnings calls are not formal SEC filings, but they fall under Regulation FD whenever an issuer uses them to share material non-public information. The SEC's guidance recommends reasonable advance notice and encourages issuers to make transcripts or replays publicly available.
What does an earnings call mean?
An earnings call is a scheduled conference call, typically held quarterly, where executives present financial results and respond to analyst questions. It usually follows a press release containing the headline figures, with prepared remarks addressing performance and a question-and-answer session that often carries more informative signal than the scripted portion, according to research on investor uncertainty in earnings calls.
Where can I read earnings calls?
Transcripts and audio replays are generally posted on a company's investor relations website, consistent with Regulation FD guidance encouraging public access. Third-party data providers and financial news platforms also republish transcripts, though the issuer's own site remains the primary source for the full session including Q&A.
How to analyze an earnings call?
Analysis typically starts by separating prepared remarks from the Q&A, then scoring narrative dimensions such as tone, uncertainty, guidance specificity, and novelty using either lexicon methods, LLM rubric scoring, or topic modeling. The most defensible approaches link every score back to the exact quoted passage that produced it, and, for forensic purposes, compare the claim against the company's own subsequent filings.
Sources
- Sec
- Narratives Contextualizing Numeric Disclosures: Insights From Earnings Calls
- Quantifying Contextual Narratives using Language Models