CAI-2 AI Practitioner (Data & Business Analysis): assessment blueprint

Review and limits. The course content has completed cross-model editorial review. The task descriptions and operational pass rule below describe the configured exam. Domain weights, timings and the pass standard remain provisional: no completed job-task analysis or empirical standard-setting study is claimed.

What CAI-2 Data & Business Analysis shows

A CAI-2 AI Practitioner (Data & Business Analysis) credential shows that the holder produced a business or data analyst's AI-assisted work to the exam's published rubric in a time-boxed exam. The tasks ask them to turn a request into an answerable question with tight metric definitions, keep personal and confidential data out of the wrong tools, clean messy data with AI planning while the spreadsheet or SQL does the work, check AI-written formulas and queries before trusting them, avoid the statistical traps that make analysis misleading, keep their work reproducible, and report charts, tests and recommendations honestly, briefing, checking and refining an AI assistant's work in a recorded conversation.

CAI-2 Data & Business Analysis is a practitioner credential. It does not certify data science, statistical modeling or machine-learning engineering.

CAI-2 is performance-based and open-AI-by-design: you earn it by doing realistic analysis work with AI in a time-boxed exam, not by recalling facts. It is genuinely failable: completing the course does not earn it, and a pasted answer or a single prompt doesn't meet the minimum for your recorded conversation.

CAI-2 builds on the CAI-1 skills (briefing, verification, data handling). CAI-1 is recommended but not required.

Competency domains and weights

| Domain | Weight | A competent candidate can… | | --------------------------------------------------- | ------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | D1. Questions, metrics and data handling | 7.5% | turn a vague request into a question tied to a decision, with a measure, population, period and comparison; write metric definitions that handle tax, currency, refunds, internal activity and incomplete windows; hold to a planned metric; keep personal and confidential data out of unapproved tools by aggregating, removing identifiers and merging small cells. | | D2. Data preparation and reproducibility | 10% | profile data before cleaning; handle duplicates, mixed date formats, currencies, units, outliers and missing values with rules that can be rerun; tell errors from genuine unusual values; keep a cleaning log with rows, rules and control totals; keep source data untouched and every figure traceable to a formula or query that someone else can rerun. | | D3. Analysis, formulas and statistical judgment | 40% | test AI-written formulas and SQL (ranges, references, joins, date boundaries, NULLs) before trusting them; compute pooled rates and averages from totals; recognize Simpson's paradox, small samples, chance findings among many comparisons, selection and survivorship, base rates, and correlation presented as cause; keep information not available at decision time out of the analysis; read A/B tests without early stopping, switched metrics or novelty effects, with a simple interval. | | D4. Communicating results | 27.5% | design charts that don't mislead (zero-based bars, labeled axes, monthly not cumulative, even intervals, descriptive titles); write a one-page memo that leads with the answer, states key figures with their definitions and uncertainty, avoids cause words the evidence doesn't support, and makes a decision someone can act on. | | D5. AI collaboration and workflow | 15% | give an assistant the context, structure and rules it needs without the data it doesn't; work in checkable steps; keep calculations in tools; challenge, verify and correct its output; keep decisions human; design repeatable reporting workflows with checks and records. |

The launch exam uses the two configured tasks below, weighted equally. Each domain weight is the mean of its share in those two task rubrics. The broader course includes additional practice; Messy extract is outside this launch exam.

| Domain | Analysis critique | Test readout | Mean (exam weight) | | ------ | ----------------- | ------------ | ------------------ | | D1 | 0 | 15 | 7.5% | | D2 | 20 | 0 | 10% | | D3 | 35 | 45 | 40% | | D4 | 30 | 25 | 27.5% | | D5 | 15 | 15 | 15% | | Total | 100 | 100 | 100% |

Traceability: tasks → criteria → domains

| Task | Criterion (weight within task) | Domain | | ----------------------------------- | ------------------------------------------------------------------------------------ | ------ | | 1. Critique and rebuild an analysis | Rebuilt figures are correct and reproducible (20) | D2 | | | Detects the planted flaws (25), Keeps accurate statements (10) | D3 | | | Chart specification is corrected (10), Rewritten summary is reliable (20) | D4 | | | AI-collaboration quality (15) | D5 | | 2. Test readout to decision memo | Holds to the planned metric and decision rule (15) | D1 | | | Judges whether the test result can be trusted (25), Segment analysis is correct (20) | D3 | | | Decision memo is honest and usable (25) | D4 | | | AI-collaboration quality (15) | D5 |

Task types

The course and the exam use the same task families (assessment-as-learning).

| Task type | What you do | How it's scored | Main domains | | ------------------------- | ----------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- | ------------ | | AI practice (sandbox) | Do real analysis work with an AI assistant: plan, clean, query, answer, report. | Rubric, including the transcript | D1–D3, D5 | | Critique | Find the planted errors in AI-written formulas, SQL, findings or charts, and justify each against the data. | Rubric: one checklist item per planted issue, plus corrections and false-alarm avoidance | D2–D4 | | Scenario | Make a sequence of decisions in a realistic analysis workflow. | Deterministic, from the quality of each choice (course only) | D1–D5 | | Work product | Produce an analysis plan, decision memo or workflow design. | Rubric with 4–7 analytic criteria | D1, D4, D5 |

How you earn CAI-2 Data & Business Analysis

Pass the CAI-2 Data & Business Analysis exam. Course completion is recommended, and the module checkpoints (80% mastery threshold, unlimited retakes) are the best rehearsal, but they are not required. Experienced analysts can take the exam directly (the challenge path).

The exam

Format: two performance tasks in the browser-based exam workspace, in configured order:
1. Critique and rebuild an AI-written quarterly analysis, with an issue log, rebuilt figures, a rewritten summary and a corrected chart specification. About 45 minutes.
2. Decision memo from an A/B test, with a calculations appendix. About 50 minutes.
Messy extract remains additional course practice and a possible later exam family.
Time: a single 3-hour time-boxed window for both tasks. (Provisional; to be confirmed in the beta.)
Task contexts: this exam uses fixed, reviewed task materials and data. Candidates may receive the same company, figures and constraints. Sharing task content is prohibited.
What's graded: your work products for each task, plus your AI-collaboration transcript for that task.
Integrity: honor-code attestation on submission.

Scoring

Each task is scored against a published-format rubric of 4–7 analytic criteria, including an AI-collaboration quality criterion worth 15% of the task. Each criterion is scored 0–4 against behavioral level descriptions, derived from a checklist of observable features.
Every checklist decision must cite verbatim evidence from your submission or your transcript. Claims your work doesn't support earn no credit.
Rubric scoring. Your answers are scored against the rubric's behavioral level descriptions. Before a task variant is used, the workspace assistant's own answer to it is measured and stored for reference. That stored answer does not decide whether a variant is used.
Criterion scores combine into a weighted composite per task (0–4), and the two task composites are averaged with equal weight.

Pass standard (provisional)

An overall composite of at least 2.5 of 4, with no task below 2.0.
Your recorded conversation with the workspace assistant must also meet a minimum on every task: at least 2.0 of 4 on AI-collaboration quality.
This remains a provisional operational standard. No completed job-task analysis or empirical standard-setting study is claimed. The task-level rubric cutScore still controls individual review/routing. The award rule above is computed from final authorized scores.

How decisions are made

AI grading uses three separate scoring passes on a fixed, versioned grading configuration. The passes may share a model.
A person reviews every fail and every flagged attempt before a decision is released. Borderline results and perfect scores get a second check by a different AI Model and go to a person unless both agree. A person also reviews a random sample of clear passes. Every credential decision is logged, and your result shows whether a person confirmed it.
Feedback generated by AI is labeled as AI-generated.
Appeals: you can appeal any result. The first appeal on each attempt is free. While Certavant has one human reviewer, an independent AI Model re-grades your work blind and a person countersigns the outcome. The outcome letter says if that person also confirmed the original decision. Appeals are decided within 5 business days.

What has been checked

Editorial review checks the course and task content for accuracy, alignment, accessibility and bias. It does not establish a validated pass standard.
Scoring checks compare the grading configuration with panel-adjudicated scores on fictional responses to the selected task contexts. They measure agreement with those AI-scored references, not agreement with human experts or coverage of other task variants.
Job-task analysis and standard setting remain provisional. The operational pass rule is not an empirical finding about workplace performance; it is reviewed against real submissions after launch.

AI-use policy for the exam

CAI-2 is open-AI-by-design: using AI well is part of the skill being assessed.

You may:

use the AI assistant provided in the exam workspace, as much as the turn limit allows. The limit is shown at the start of the exam;
use a spreadsheet or calculator, your own notes, the course materials and any reference sources.

How your AI use is assessed:

The workspace assistant is the sanctioned AI channel. Every message you send and every reply is recorded server-side, per task, and attached to your submission as a transcript appendix.
The transcript is graded under AI-collaboration quality: whether you supplied the context and rules, worked in steps, kept calculations in your own tools, checked and challenged the output, corrected its errors, kept unnecessary personal data out of it, and made the decisions yourself. It is 15% of each task's score, and it has its own minimum on every task, so each task needs a recorded conversation with the assistant that meets it.
Answers are scored against the rubric, and your recorded conversation must meet the minimum on every task. A pasted answer or a single prompt doesn't meet it.

You may not:

use any AI tool other than the workspace assistant during the exam. You confirm this in the attestation when you submit;
have another person do any part of the work, or share task content with anyone;
paste real confidential, client or personal data into the assistant. The tasks provide all the information you need, and some task materials deliberately include fictional personal data that the task does not need;
include text intended to influence the grader, such as instructions to the AI grader or hidden text, in your submission or your conversation with the assistant. This is an integrity violation. It results in a fail and can lead to a ban and revocation of any credential.

The course tutor is not available during the exam.

The credential

Verifiable and shareable (Open Badges 3.0), with an "Add to LinkedIn profile" option. Displayed as CAI-2 AI Practitioner (Data & Business Analysis).
Evidence-embedded: the credential can link to the work you submitted, with privacy settings you control (public, employer-link-only, or hidden).
Stamped with an expiry date at issue (12–18 months; the final term is to be confirmed) and renewable by assessment.

Content freshness

Almost all of the course and exam measure durable skills (D1–D5) that change slowly: question framing, metric definitions, cleaning rules, statistical judgment, reproducibility and honest reporting. Spreadsheet and SQL function behavior is stable but differs between products and versions, so the course names the products it assumes and the exam states the syntax it uses. Rules on personal data and anonymization differ between jurisdictions; the course teaches widely shared principles, cites UK guidance where it states a rule, tells learners to check with their data-protection team, and the exam materials state every rule a task needs.