Industry Study
SuperMoney’s AI Scores 99.4% on Its CFP® Knowledge Benchmark
Last updated 08/19/2026 by
SuperMoney Team
Edited by
Andrew Latham
The bar for financial AI should be measurable. So we measured ours.
Sense AI, the AI financial assistant inside SuperMoney’s agentic finance app, scored a median of 99.4% on a 1,000-question benchmark built against CFP Board’s 2026 Principal Knowledge Topics blueprint. Seven full runs. A standard deviation of 0.11 percentage points. The number that matters most: 988 of the 1,000 questions were answered correctly in every single run.
Figure 1. Item-level reliability. 988 of 1,000 questions were answered correctly on every one of the seven runs.
Why we benchmarked against CFP Board’s exam blueprint
Large language models produce confident financial answers whether or not they’re correct. In personal finance, confident and wrong is the most expensive combination there is. Get Roth conversion timing or estate titling wrong, and a family pays for it in real dollars, sometimes for years.
The industry has mostly responded in one of two ways. Some products restrict AI to disclaimers and boilerplate. Others wrap a raw model in safety scaffolding and call it rigor without ever measuring whether it works.
We took a third path. We measured Sense AI against the published knowledge topics that CFP Board’s certification exam is written around, and published the methodology. If a system is going to help people understand financial decisions, it should be able to demonstrate command of that body of knowledge.
To be clear about what this is and is not: our benchmark tests the knowledge dimension, and only that. The exam is just one of four requirements for certification, alongside coursework, thousands of hours of real client experience, and a fiduciary ethics commitment. We are not claiming Sense AI would pass the CFP® exam or match a CFP® professional.
What Sense AI is
Sense AI is the conversational AI financial assistant embedded across SuperMoney, an agentic finance app. It helps members understand their own money (spending, credit, debt, savings, investments, retirement, tax exposure, insurance, estate) with grounded, member-specific explanations rather than generic content, and it’s the reasoning layer behind the app’s agentic features.
The core components:
- Language model. Sense AI is model-agnostic by design. The retrieval, routing, tooling, and guardrail layers are independent of the underlying LLM, which we can swap as frontier models improve, without rebuilding the system around it. The configuration evaluated in this benchmark ran Anthropic’s Claude Opus 4.7 at medium reasoning effort.
- Retrieval-augmented generation. A curated knowledge base of SuperMoney-authored financial planning modules, embedded and stored in PostgreSQL with pgvector. The most relevant modules are retrieved and passed into the prompt at inference time, so answers are grounded in a knowledge base we control rather than the model’s memory alone.
- Agent routing. Each query is classified by type (investment calculation, debt strategy, budgeting, general planning) and routed to a specialist agent with its own system prompt and tools.
- Tool use. Specialist agents call financial calculators, member-data lookups, and third-party integrations.
- Guardrails. A two-stage guardrail layer filters both member input and model output for safety and scope.
The same reasoning engine powers three member surfaces: the conversational assistant, per-insight explanations attached to notifications and cards, and periodic financial summaries. We evaluated the conversational path because it has the widest query space, which makes it the hardest test.
Evaluation methodology
The question bank
1,000 multiple-choice items, four options each (A through D), one designated correct answer per item, calibrated to CFP Board’s 2026 Principal Knowledge Topics blueprint. Coverage across all eight domains:
| Domain | Questions |
|---|---|
| A – Professional Conduct & Regulation | 83 |
| B – General Principles of Financial Planning | 157 |
| C – Risk Management & Insurance Planning | 121 |
| D – Investment Planning | 159 |
| E – Tax Planning | 134 |
| F – Retirement Savings & Income Planning | 177 |
| G – Estate Planning | 104 |
| H – Psychology of Financial Planning | 65 |
Every item carries a difficulty label (basic, intermediate, advanced) and a Bloom’s-taxonomy cognitive level (recall, application, analysis), so performance can be sliced and reported by segment, not just in aggregate.
Why we wrote our own questions
A fair question. Why build your own bank instead of using the real thing? Because using the real thing isn’t legally possible, for us or anyone. The CFP® exam is confidential; every candidate signs a pledge prohibiting the sharing of exam questions with any entity other than CFP Board, and CFP Board treats the contents of both the exam and its own practice exam as its exclusive property. CFP Board’s exam terms also expressly prohibit using its exam preparation materials in ways that violate its copyright. That’s why any published AI benchmark in this space, ours included, must be built from independently written questions calibrated to the public blueprint rather than the exam itself.
Self-authored questions raise an obvious objection. How do we know the bank is as rigorous as a real exam? To address that, we purchased a commercially available CFP® practice exam from an established test-prep provider and used it, solely for internal calibration, as an independent reference. Those questions are the provider’s intellectual property, so they are not part of the published benchmark or the score; their role was to confirm that our bank’s difficulty and domain coverage hold up against an authoritative third-party standard.
What we actually tested
Many published AI benchmarks test a lab configuration that customers never touch. We tested production, full stop.
| Component | Setting |
|---|---|
| System under test | Sense AI |
| Language model (test configuration) | Claude Opus 4.7, medium reasoning effort. Sense AI is model-agnostic; this is the model that was in production at test time. |
| Retrieval | Production knowledge base, pgvector |
| Routing | Production agent-routing configuration |
| Endpoint | /api/chatbot/external_chat. The same API behind every member surface |
| Session | Isolated per question, no shared state |
| Guardrails | Enabled, production sensitivity |
One configuration choice worth explaining: reasoning effort was set to medium, not high, and not because it was cheaper. During model selection, high effort made the guardrail layer more conservative, declining a meaningful share of exam-style items as out of scope (3.7% to 11.2% across trial runs), while medium effort produced far fewer declines and higher accuracy. More thinking time made the system more cautious, not more correct. In the published configuration, medium effort with the evaluation-context instruction in place, out-of-scope declines were eliminated: the assistant returned an answer to every in-scope item across all 7,000 queries. Medium was also the production setting members received at the time of testing.
Run protocol
Each run pushed all 1,000 questions through the production API sequentially, with concurrency capped at 5 in-flight requests. Every question got a fresh session ID. No conversation memory, no shared state, no exposure to prior questions.
That isolation was deliberate. Running the bank through a chat UI would have accumulated hundreds of prior questions and answers in one context window, effectively letting the model study during the exam. Per-question API isolation reproduces realistic single-query inference conditions, which is stricter than what a real member session looks like.
Each response was parsed for a designated answer line and scored against the key, with domain, difficulty, and cognitive-level metadata attached for per-slice reporting. Responses were retried up to three times if they didn’t produce a parseable answer line.
We completed seven independent runs of the full bank: 7,000 question-answers administered in total. Six of the 7,000 (0.09%) still returned no parseable answer, all on calculation-heavy items where the model’s working ran long without a clean final answer line, and were excluded from the denominator rather than scored incorrect. No item was ever marked wrong for a non-answer, so per-domain denominators count valid answers. Analysis was completed in July 2026, based on seven runs.
Results
| Metric | Value |
|---|---|
| Median accuracy | 99.40% |
| Mean accuracy | 99.37% |
| Standard deviation across runs | 0.11 percentage points |
| Range | 99.20% to 99.50% |
| 95% confidence interval on the mean | [99.27%, 99.46%] |
Figure 2. Seven independent runs landed within a 0.3-point band.
Performance by domain
Three domains scored perfectly across all seven runs: Risk Management & Insurance Planning (847 of 847), Retirement Savings & Income Planning (1,238 of 1,238*), and Psychology of Financial Planning (455 of 455). The weakest domain, Tax Planning, still came in at 97.97%.
*Retirement had 1,239 question-answers administered (177 questions × 7 runs); one returned no parseable response and was excluded from scoring under the rule described in the run protocol. Sense AI answered all 1,238 valid responses correctly. The same exclusion rule explains why a few other domain denominators sit slightly below questions × 7.
Figure 3. Accuracy by knowledge domain across all valid question-answers. The axis starts at 95% so you can see the differences; every domain sits between 97.97% and 100%.
Item-level reliability
988 of the 1,000 questions returned a valid, correct answer in every one of the seven runs (see Figure 1). That stable core is the difference between “we got a good run once” and “the system knows this material.” The 12 items that varied across runs are exactly where our hardening work goes next.
How we know it holds up
- Configuration parity. The benchmark hit the same API, retrieval, routing, guardrails, and tool wiring that members hit. There was no benchmark-specific build.
- Grounded retrieval. Every response is generated with SuperMoney-curated planning modules retrieved into context. The model doesn’t answer from parametric memory alone.
- Specialist routing. A single monolithic prompt would be a compromise across every question type. Queries are classified and routed to an agent tuned for that domain.
- Model-agnostic architecture. The system layers (retrieval, routing, tools, guardrails) sit above the language model, not inside it. The benchmark measures the whole system, and the score has a built-in upgrade path: when a stronger model ships, we can swap it in and re-run the same 1,000-question bank under the same protocol. This test used Claude Opus 4.7; the methodology doesn’t change when the model does.
- Isolation and repetition. Fresh session per question, seven independent runs, variance measured and published.
- Open to scrutiny. The questions are ours, so we can show our work. A representative sample of the question bank is available on request, and we’re glad to walk researchers and journalists through the methodology in more depth.
What Sense AI is for
Sense AI is built as a DIY educational tool: a coach and research companion that helps members understand their own finances and run their own money with confidence. And when a member does bring in a professional (for a complex estate, a business sale, a messy tax year), they walk in informed, ask sharper questions, and can tell good advice from a sales pitch. Three representative uses in production:
- Personal finance education. “What do general guidelines suggest about retirement savings rates for someone at my income and age?” Framed with standard planning principles and the member’s linked-account data for context, as education rather than a personalized recommendation.
- Educational explanations. “How does a Roth conversion work, and what factors generally affect whether it makes sense?” Explained using general, current federal tax rules, with the member’s own data for context. Sense AI does not tell members what to do or provide personalized tax, investment, or legal advice; for a decision on your own situation, consult a licensed CFP® professional, CPA, or attorney.
- Insight coaching. Per-insight explanations that turn a static notification or card into an interactive conversation about what a number means and why it matters.
Key takeaways
- Sense AI scored a 99.4% median across seven independent 1,000-question runs of a benchmark built on CFP Board’s published exam blueprint, with a standard deviation of just 0.11 points.
- The benchmark ran through the production API with production retrieval, routing, and guardrails. No lab-only configuration.
- 988 of 1,000 questions returned a valid, correct answer on every single run.
- Every question ran in an isolated session with no memory of prior questions, which is stricter than a real member conversation.
- Three of eight knowledge domains (Insurance, Retirement, Psychology) scored a perfect 100% across all runs.
- A representative sample of the question bank is available on request; the questions are SuperMoney’s own, written against CFP Board’s public blueprint because the actual exam is confidential and its practice materials are copyrighted.
Disclosures
Sense AI is an educational tool. SuperMoney is not a registered investment adviser, and nothing in this post or produced by Sense AI is personalized investment, tax, or legal advice or a recommendation to take or refrain from any action.
Certified Financial Planner Board of Standards, Inc. owns the certification marks CFP®, CERTIFIED FINANCIAL PLANNER®, CFP® (with plaque design), and CFP® (with flame design) in the United States, which it awards to individuals who successfully complete CFP Board’s initial and ongoing certification requirements. SuperMoney and Sense AI are not affiliated with, endorsed by, or certified by CFP Board, and nothing in this report implies that Sense AI holds CFP® certification or any other professional credential.
The benchmark question bank was written by SuperMoney and calibrated to CFP Board’s published 2026 Principal Knowledge Topics blueprint. It is not CFP Board’s certification exam or practice exam, neither of which is available for this purpose: the exam is confidential under a candidate pledge, and CFP Board’s practice materials are its copyrighted property. Performance on this benchmark does not predict performance on the actual CFP® exam.
Benchmark accuracy measures performance on multiple-choice, exam-style questions under controlled conditions. Real member conversations are open-ended, and results on this benchmark do not represent, and may not predict, the accuracy of any individual response in production. Like all AI systems, Sense AI can make mistakes, and the quality of its answers depends in part on the accuracy and completeness of the information members provide or link. Past performance, including performance on standardized test questions, does not guarantee future results.
Members are responsible for their own financial decisions and should consult a licensed CFP® professional, CPA, attorney, or other qualified advisor before making material personal decisions. Sense AI is a complement to professional advice, not a replacement for it.
Conclusion
Sense AI answers questions built on CFP Board’s exam blueprint at 99.4% median accuracy, measured on the production system real members use and verified across seven independent runs. The tested configuration ran Claude Opus 4.7, though the architecture is model-agnostic and the same protocol will re-measure the system as models evolve.
Two design choices carry the weight: configuration parity, so the tested system is the shipped system, and repetition under isolation, so the score reflects stable knowledge rather than a lucky run. That’s the standard we think every AI making claims about financial competence should be held to.
Share this post:
Add
as a preferred source on GoogleTable of Contents