SEC EDGAR MCP Benchmark: 3 Providers, Same 3 Questions
We tested AlphaCreek, edgar.tools, and SEC API with the same model and 3 filing questions. Compare citations, failures, costs, and reproducible traces.
SEC EDGAR MCP Benchmark: AlphaCreek vs edgar.tools vs SEC API (July 2026)
Table of contents
- Short answer
- Providers compared
- Why we ran this SEC EDGAR MCP benchmark
- Methodology
- Citation quality — the clearest difference
- Retrieval style — TOC navigation vs whole-section reads
- Results at a glance
- Question 1 — NVIDIA risk factors
- Question 2 — TSLA strategy over five 10-Ks
- Question 3 — AAPL revenue segments
- What this means — choose by agent job
- Limitations and what we would test next
- Reproduce this benchmark
- FAQ
- Methodology and update context
- Final takeaway
- Get started with AlphaCreek
Short answer
We ran an SEC EDGAR MCP benchmark on July 9, 2026: the same model (gpt-5.4-mini), the same OpenAI Responses + MCP loop, and the same three filing questions against AlphaCreek, edgar.tools, and SEC API. The script and full report are on GitHub (run 084202).
AlphaCreek and edgar.tools succeeded on all three questions. SEC API succeeded on question 1 only; question 2 ended with session terminated and question 3 with HTTP 503. We cannot draw firm conclusions about SEC API on those tasks — the failures may reflect our free-tier setup, agent configuration, or transient service issues.
Where runs completed, answers were comparable and satisfactory, and LLM costs were in a similar range (~$0.04–0.10 per question). The meaningful difference was not answer quality or price. It was citations and retrieval style.
When the prompt asked to “cite the source for each material claim,” only AlphaCreek backed every claim with a link to the exact passage in the AlphaCreek SEC reader — each URL includes doc_id and node_id (for example, NVIDIA export-controls risk). edgar.tools linked to the sec.gov filing index; SEC API linked to the raw .txt submission — accurate filing identity, but no scroll-to-passage. AlphaCreek also read the table of contents first, then targeted nodes inside Risk Factors, Business, and segment tables. The other two MCPs typically pulled whole sections in one call.
Practical takeaway: If you are building a citation-backed SEC filing agent, compare MCPs on what “cite the source” actually means in the answer — not just tool count or latency. Reproduce our run on GitHub; for vendor selection (including EdgarTools open source), see Best SEC EDGAR MCP Servers for AI Agents.
| Question | AlphaCreek | edgar.tools | SEC API |
|---|---|---|---|
| Q1 — NVDA risks | Success | Success | Success |
| Q2 — TSLA strategy (5×10-K) | Success | Success | Failed (session terminated) |
| Q3 — AAPL revenue segments | Success | Success | Failed (HTTP 503) |
Providers compared
This benchmark compares three hosted SEC EDGAR MCP servers on the same agent loop and questions. We did not include EdgarTools open source (self-hosted) — see Best SEC EDGAR MCP Servers for AI Agents for that path.
| Provider | Product | MCP endpoint | Tools (this run) | Tier tested | One-line role |
|---|---|---|---|---|---|
| AlphaCreek | alphacreek.ai | https://mcp.alphacreek.ai/mcp | 4 | Free | Filing TOC navigation, node reads, passage citations |
| edgar.tools | edgar.tools | https://app.edgar.tools/mcp | 22 | Professional | Broader SEC analyst tools, section extracts, sec.gov links |
| SEC API | secapi.ai | https://api.secapi.ai/mcp | 87 | Free | Wide financial data platform — filings, facts, macro, factors, portfolio |
Docs and setup: AlphaCreek MCP docs · edgar.tools MCP · SEC API MCP workflows (if available; we used x-api-key auth on the free tier).
AlphaCreek is the publisher of this article and one of the three providers above. edgar.tools and SEC API were tested under the same open-source script without vendor involvement.
edgar.tools tier note: We used Professional because, in our testing, the free tier did not expose full 10-K / filing text through MCP (e.g. filing_section on risk factors or business overview). A fair filing-Q&A comparison required a tier that could return actual disclosure text. AlphaCreek and SEC API ran on free tiers in this run.
Why we ran this SEC EDGAR MCP benchmark
We were preparing Best SEC EDGAR MCP Servers for AI Agents and needed first-hand evidence. Product pages and READMEs describe tools and features; they do not show what happens when the same agent loop asks real filing questions and demands citations.
Desk research cannot answer:
- What “cite the source” means in practice per MCP
- Whether the agent navigates a filing or dumps whole Items
- Setup friction (OAuth, tool-name sanitization, tier limits)
- Failure modes under a generic, unconstrained agent loop
Our architecture posts argue that navigation beats naive RAG on SEC filings and that agents should read titled nodes, not arbitrary chunks. This benchmark tests whether that shows up in MCP tool traces and citation URLs. edgar.tools published a strong MCP-vs-web-search benchmark in June 2026; we extend that idea to hosted MCP vs hosted MCP with open scripts and passage-level citation comparison.
Methodology
Run settings
| Setting | Value |
|---|---|
| Date | July 9, 2026 |
| Model | gpt-5.4-mini |
| Orchestration | OpenAI Responses API + MCP Streamable HTTP |
| Script | benchmark_sec_edgar_mcp.py |
| Published report | benchmark_mcp_sec_providers_20260709_084202.md |
| Max tool rounds | 8 per question |
Provider tiers and auth
See Providers compared for product and MCP endpoint links.
| Provider | Tier used | Auth |
|---|---|---|
| AlphaCreek | Free | ALPHACREEK_API_KEY (Bearer) |
| edgar.tools | Professional | OAuth bearer token |
| SEC API | Free | SECAPI_API_KEY (x-api-key) |
This is not a perfectly symmetric tier comparison. edgar.tools ran on Professional; AlphaCreek and SEC API ran on free tiers. Quotas, tool gates, and rate limits may differ.
Why edgar.tools Professional: On the free tier, we could not retrieve full 10-K section text via MCP — the benchmark questions (risk factors, multi-year business narrative, segment tables) need readable filing content, not just filing metadata or briefs. We upgraded to Professional for this run so edgar.tools could participate on equal footing for substance; citation and tool-path results still reflect edgar.tools’ product design, not a claim that Professional is required for all MCP use cases.
Three benchmark questions (verbatim)
- What are NVIDIA's biggest risks in the latest 10-K? Cite the source for each material claim.
- How did TSLA's strategy evolve over the last 5 10-K reports?
- What is the latest revenue breakdown by product/segment for AAPL?
Question 1 alone included the per-claim citation instruction; questions 2–3 did not repeat it. The citation comparison below focuses on Q1 for that reason.
Metrics collected
Success, wall-clock time, OpenAI input/output tokens, estimated LLM cost, tools advertised, tool-call count, MCP output size, and citation URL patterns (AlphaCreek reader vs sec.gov index vs .txt).
Setup friction we hit
edgar.tools: OAuth, not the Settings API key. Documentation can read as if the API key under Settings works for MCP. It does not for this benchmark. Using the etk_... REST key produced an unauthorized / wrong-format error. MCP access required OAuth: python benchmark_sec_edgar_mcp.py --login-edgar-tools (browser PKCE flow), then EDGAR_TOOLS_MCP_TOKEN or the cached bearer token. If you reproduce the benchmark, budget time for OAuth — not Settings → API Keys.
SEC API: dotted tool names vs OpenAI. On the first run, OpenAI rejected SEC API’s tool list because names contain dots (entities.resolve, filings.latest, …). The Responses API requires ^[a-zA-Z0-9_-]+$. The benchmark script adds a sanitize/map layer: OpenAI sees safe names; session.call_tool() uses the original MCP names. Anyone wiring SEC API into OpenAI will likely need the same workaround.
Citation quality — the clearest difference
All three providers produced substantively correct answers on question 1 (NVIDIA risk factors). Export controls, customer concentration (22% / 14%), supply-chain dependence, cybersecurity, and competition appeared in each successful run. The prompt also asked for a source per material claim — and on the same NVDA export-controls risk, AlphaCreek linked to a passage-level reader URL, while edgar.tools and SEC API each repeated a filing index or .txt submission link for every claim. That is where the MCPs diverged; the per-provider screenshots below show each pattern.
AlphaCreek — exact passage in the SEC reader
AlphaCreek is the only provider that linked each claim to a unique passage URL in the AlphaCreek SEC reader. Each URL carries doc_id and node_id, opening the annotated filing at that titled node — the passage the model used, with structure and tables preserved.
Examples from the published run:
- Export controls / H20 charge: alphacreek.ai/sec/reader?doc_id=NVDA_10-K_000104581026000021&node_id=sp-1.2.28.1
- Competition risk: alphacreek.ai/sec/reader?doc_id=NVDA_10-K_000104581026000021&node_id=sp-1.2.9.1
- Customer concentration (22% / 14%): alphacreek.ai/sec/reader?doc_id=NVDA_10-K_000104581026000021&node_id=sp-2.3.17.1
- AAPL product revenue (Q3): alphacreek.ai/sec/reader?doc_id=AAPL_10-Q_000032019326000013&node_id=sp-1.1.5.4.3.2
- AAPL geographic segments (Q3): alphacreek.ai/sec/reader?doc_id=AAPL_10-Q_000032019326000013&node_id=sp-1.2.8.2
On question 2 (TSLA), AlphaCreek attached per-year, per-theme reader links — for example 2022 Business overview node and 2026 AI / Robotaxi framing.
AlphaCreek citations open the exact titled node in the SEC reader — not the whole filing. Example: NVIDIA export-controls risk (sp-1.2.28.1)
edgar.tools — filing index, not passage
edgar.tools cited the correct filing but repeated the same sec.gov filing index URL for every NVDA risk claim:
www.sec.gov/Archives/edgar/data/1045810/000104581026000021/0001045810-26-000021-index.htm
On question 3 (AAPL), it linked to the earnings-release index:
www.sec.gov/Archives/edgar/data/320193/000032019326000011/0000320193-26-000011-index.htm
That is valid provenance for analysts who know EDGAR. It does not satisfy “cite each claim” in an agent UX sense: the user lands on the filing detail page and must search Item 1A manually. On TSLA, edgar.tools did attach one index link per filing year — better granularity than NVDA, but still filing-level, not passage-level.
edgar.tools on Q1: correct filing, same NVDA 10-K index URL repeated for every risk claim.
SEC API — raw .txt filing
SEC API (question 1 success) cited the full-text .txt submission, repeated for each claim:
www.sec.gov/Archives/edgar/data/1045810/0001045810-26-000021.txt
A .txt file is searchable but not structured for quick verification: no HTML anchor, no section highlight, easy to open a 500-page blob with no obvious landing point. On questions 2 and 3, SEC API produced no final answer and no citation URLs before failure.
SEC API on Q1: correct filing identity, same NVDA .txt submission repeated for every claim.
Citation comparison table
| Provider | Citation target | Scrolls to passage? | Unique URL per claim? |
|---|---|---|---|
| AlphaCreek | AlphaCreek SEC reader (doc_id + node_id, e.g. NVDA sp-1.2.28.1) | Yes | Yes |
| edgar.tools | sec.gov filing index | No | No on Q1 (same index repeated) |
| SEC API | sec.gov filing .txt | No | No (same .txt repeated) |
AlphaCreek is the only MCP in this benchmark that backed up every material statement with a passage-level citation. The others pointed to the filing as a document, not to the sentence or table cell behind the claim.
Retrieval style — TOC navigation vs whole-section reads
Citation quality follows retrieval shape. AlphaCreek encodes navigation-first SEC filing retrieval; the other two optimize for section extraction.
AlphaCreek: TOC → targeted node reads
On every successful question:
get_latest_filingorlist_filings— choose the filingget_filing_toc— inspect the filing-specific map- Multiple
read_node_contentcalls on sub-nodes inside Item 1A, Item 1, or segment tables
Question 1 (NVDA): 8 tool calls — one TOC read, then six read_node_content calls on risk subsections (1.2.2, 1.2.12, 1.2.16, 1.2.28, 1.2.29, 2.3.17) instead of loading all of Item 1A at once.
Question 2 (TSLA): 11 calls — four TOC reads across four 10-Ks, then four read_node_content calls on Business (1.1.1) per year.
Question 3 (AAPL): 6 calls — latest 10-Q, TOC, then targeted reads on product and geographic segment nodes.
edgar.tools: discovery → whole filing_section
Typical path: company_filings / search_filings discovery, then filing_section for an entire section — e.g. risk_factors (~115K characters in one pull on NVDA), five business_overview pulls on TSLA, earnings_release with segment tables on AAPL.
SEC API: section keys at document granularity
Question 1 (success): entities.resolve → sections.get Item 1A mode: full (~124K characters, one call).
Question 2 (failure): nine sections.get calls (Item 1A / Item 7, compact, across years) → session terminated, no answer.
Question 3 (failure): filings.latest for 10-Q and 10-K → next step hit HTTP 503 (see below).
AlphaCreek navigates into subsections; edgar.tools and SEC API often read whole Items or sections in one shot.
Results at a glance
Full answers and tool traces: GitHub report 084202.
Success and cost (estimated LLM side only)
| Question | Provider | Success | Elapsed | Est. cost | Tool calls | Tools adv. |
|---|---|---|---|---|---|---|
| Q1 NVDA | AlphaCreek | Yes | 31.6s | $0.042 | 8 | 4 |
| Q1 NVDA | edgar.tools | Yes | 45.8s | $0.065 | 6 | 22 |
| Q1 NVDA | SEC API | Yes | 19.5s | $0.046 | 2 | 87 |
| Q2 TSLA | AlphaCreek | Yes | 70.7s | $0.101 | 11 | 4 |
| Q2 TSLA | edgar.tools | Yes | 41.0s | $0.088 | 12 | 22 |
| Q2 TSLA | SEC API | No | 18.6s | $0.036 | 9 | 87 |
| Q3 AAPL | AlphaCreek | Yes | 17.7s | $0.041 | 6 | 4 |
| Q3 AAPL | edgar.tools | Yes | 14.9s | $0.044 | 5 | 22 |
| Q3 AAPL | SEC API | No | 25.8s | $0.021 | 2 | 87 |
Balanced read of the numbers
- Answers: Where runs finished, substance was comparable and satisfactory — same NVDA risk themes, same TSLA AI/platform arc, same AAPL segment dollars ($56.994B iPhone, $30.976B Services, $111.184B total quarter).
- Costs: LLM costs clustered around $0.04–0.10 per question on successful runs — not order-of-magnitude apart.
- Citations: Only AlphaCreek met “cite each claim” with passage-level URLs (see Citation quality).
- SEC API: One clean success (Q1). Do not rank SEC API overall from Q2/Q3 failures alone — treat those runs as inconclusive pending rerun with tool filtering, paid tier, or vendor guidance.
Question 1 — NVIDIA risk factors
Prompt: What are NVIDIA's biggest risks in the latest 10-K? Cite the source for each material claim.
All three successful runs identified export controls (including the $4.5B H20 charge), customer concentration (22% and 14% direct customers), supply/demand mismatch, foundry dependence, cybersecurity, and competition.
| Provider | Tool path (summary) | Citation pattern |
|---|---|---|
| AlphaCreek | TOC + 6 risk sub-node reads | 10+ unique reader URLs |
| edgar.tools | 5 discovery calls + filing_section risk_factors | Same index URL per claim |
| SEC API | sections.get Item 1A full | Same .txt URL per claim |
SEC API was fastest (19.5s, 2 calls). edgar.tools spent several calls finding the right 10-K before pulling the entire Risk Factors section (~115K characters). AlphaCreek was slower but each risk linked to its own node.
Question 2 — TSLA strategy over five 10-Ks
Prompt: How did strategy evolve over the last five 10-K reports?
AlphaCreek and edgar.tools both delivered a coherent year-by-year narrative: EV + energy manufacturer → charging ecosystem / NACS → explicit AI and Robotaxi → “bring AI into the real world” with fleet-based economics.
- AlphaCreek: 11 tool calls; TOC per filing + Business node reads; multiple reader citation URLs per year (e.g. 2022, 2026 AI thesis).
- edgar.tools: 12 tool calls; five
business_overviewsection pulls; one sec.gov index link per filing year.
SEC API: Failed after nine sections.get calls (Item 1A / Item 7, compact, 2023–2026). Error: Session terminated. No final answer. We may have hit a timeout, quota, or configuration issue — not a definitive statement that SEC API cannot do multi-year strategy comparison.
AlphaCreek was the slowest successful run (70.7s, $0.101) but produced the finest-grained citations.
Question 3 — AAPL revenue segments
Prompt: Latest revenue breakdown by product/segment.
AlphaCreek and edgar.tools — both correct
Both returned fiscal Q2 2026 figures (quarter ended March 28, 2026):
- iPhone $56.994B, Services $30.976B, total $111.184B
- Americas $45.093B, Greater China $20.497B, etc.
edgar.tools was slightly faster (14.9s vs 17.7s) and routed through the Q2 2026 earnings release (filing_section, segment tables) — a pragmatic analyst shortcut.
AlphaCreek read the latest 10-Q segment tables and cited two reader nodes: product categories and geographic segments.
SEC API — HTTP 503 (inconclusive)
SEC API advertised 87 tools (filings, sections, macro, factors, portfolio, dilution, intelligence, …). The agent trace:
filings.latest— AAPL 10-Qfilings.latest— AAPL 10-K- Next step (
sections_searchfor segment keywords in console logs) →HTTP 503 Service Unavailableforhttps://api.secapi.ai/mcp
No answer was produced. This may be a transient outage, free-tier throttling, or a poor tool choice across an oversized catalog — we cannot isolate the cause without a filtered-tool rerun or SEC API support.
Note: An earlier experimental run hit
context_length_exceededafterintelligence.footnotesreturned ~1.25M characters — another reason to allowlist tools when exposing large MCP surfaces to general agents. The canonical published failure for this article is 503 on report084202.
What this means — choose by agent job
Answers and costs were broadly comparable where runs succeeded. Choose on citation model, retrieval style, and operational fit:
| If your agent needs… | This benchmark suggests… |
|---|---|
| Every claim linked to the exact filing passage | AlphaCreek — only provider that did this on every successful run |
| sec.gov filing index links + earnings-release shortcuts | edgar.tools |
| One-shot full Item 1A / Item 1 text | SEC API on Q1 — with tool filtering and caveats |
| Smallest tool surface / predictable TOC → node loop | AlphaCreek (4 tools) |
| Analyst-grade answers on standard filing Q&A | All three when runs succeed |
| Unrestricted 87-tool MCP in a generic agent | Caution — SEC API failed 2/3 questions in our free-tier run |
For deployment models, EdgarTools open source, monitoring, and enriched signals, see Best SEC EDGAR MCP Servers for AI Agents.
Limitations and what we would test next
- Single model (
gpt-5.4-mini); one run per question; three prompts only — not a full task suite. - Tier asymmetry: edgar.tools Professional (free tier lacked full 10-K text in our MCP tests); AlphaCreek and SEC API Free.
- Author affiliation: Krasimir Atanasov is AlphaCreek founder; AlphaCreek was tested under the same script as the other providers.
- SEC API Q2/Q3 inconclusive — rerun with tool allowlist, paid tier, and vendor best practices before judging capability.
- Citation analysis based on URL patterns and manual review, not blind human grading of answer correctness.
- edgar.tools OAuth was harder than docs implied; do not use Settings
etk_...for MCP. - Next steps: filtered SEC API tool set; EdgarTools OSS arm; second model; human citation audit; rerun Q3 after 503.
Reproduce this benchmark
git clone https://github.com/kpace/sec-edgar-mcp-benchmarks.git
cd sec-edgar-mcp-benchmarks
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
export OPENAI_API_KEY='...'
export ALPHACREEK_API_KEY='...'
export SECAPI_API_KEY='...'
python benchmark_sec_edgar_mcp.py --login-edgar-tools # OAuth — not Settings API key
python benchmark_sec_edgar_mcp.py
Reports are written to reports/. Compare your output to the July 9, 2026 run. Pull requests with additional providers, tiers, or questions are welcome.
FAQ
How did you benchmark SEC EDGAR MCP servers?
Same OpenAI Responses + MCP loop, model gpt-5.4-mini, three providers, three filing questions, July 9, 2026. Script: github.com/kpace/sec-edgar-mcp-benchmarks.
Which MCP had the best citations?
AlphaCreek: unique SEC reader URLs per claim (doc_id + node_id), scroll to exact passage — e.g. NVIDIA competition risk. edgar.tools: sec.gov filing index. SEC API: sec.gov .txt file (Q1 only).
Were answers and costs similar?
On successful runs, yes — comparable substance and LLM costs roughly $0.04–0.10 per question.
Why did SEC API fail on questions 2 and 3?
Q2: session terminated after nine section reads. Q3: HTTP 503 after filings.latest calls. Inconclusive — may be service, tier, or our agent configuration.
Why did the edgar.tools API key from Settings not work?
MCP requires an OAuth bearer token, not the etk_... REST key from Settings. Use --login-edgar-tools in the benchmark script.
Why did SEC API break on the first OpenAI call?
Tool names contain dots (filings.latest). OpenAI requires alphanumeric/underscore names. The benchmark script maps sanitized names for OpenAI and original names for MCP.
Is this investment advice?
No. Infrastructure and agent-workflow comparison only.
Methodology and update context
- Benchmark date: July 9, 2026
- Canonical report: benchmark_mcp_sec_providers_20260709_084202.md
- Repository: kpace/sec-edgar-mcp-benchmarks (MIT)
- Prior art: edgar.tools MCP vs web search benchmark (different scope: MCP vs web, not MCP vs MCP)
- Provider docs: AlphaCreek MCP, edgar.tools MCP, SEC API MCP workflows
- Last updated: July 9, 2026
This article compares technical infrastructure for SEC filing access and financial AI agents. It is not investment advice.
Final takeaway
This SEC EDGAR MCP benchmark shows that hosted MCP servers can return similar answers at similar cost — but they are not interchangeable when the product requires verifiable citations.
Only AlphaCreek linked each material claim to an exact passage in the annotated SEC reader. edgar.tools and SEC API pointed to filing-level sec.gov resources (index or .txt) that leave verification work to the user. AlphaCreek read the TOC and targeted nodes inside sections; the others often pulled whole sections in one tool call.
SEC API succeeded once and failed twice in our free-tier run — inconclusive, not a final verdict. Reproduce the test on GitHub. For architecture context, see AI agents over SEC filings: RAG vs navigation and how to build an SEC filing agent without naive RAG. For vendor selection, read Best SEC EDGAR MCP Servers. Use AlphaCreek when passage-level citations are part of the product requirement.
Get started with AlphaCreek
AlphaCreek gives agents hosted MCP access to structured SEC filings, filing table-of-contents maps, targeted node reads, and citation URLs to exact passages.
Build with AlphaCreek MCP · View MCP developer docs · AI agents over SEC filings: RAG vs navigation · How to build an SEC filing agent without naive RAG · Best SEC EDGAR MCP servers