Overview

You deployed AI three months ago. People are using it. But are you actually getting the results you promised? Time savings. Quality improvements. Better employee experience. You need to know.

This lesson teaches you measurement methodology, how to build credible before/after comparisons, control for confounds, and claim impact honestly. You'll learn the difference between correlation and causation. You'll understand when you can confidently say "AI saved us 30% time" vs. when you need to hedge. And you'll design measurement approaches that leadership trusts.

The risk isn't measuring too much. It's measuring wrong, claiming savings you didn't actually achieve, or missing impact because you didn't look for it the right way.

Why This Matters for HR Leaders

Bad measurement leads to two bad outcomes: false claims or missed impact.

False claims: You announce "AI recruiting saved us 25% time" based on one recruiter's rough estimate. Leadership allocates budget based on this. You expand to other functions. Then auditing shows the actual savings were 10% (still good, but you oversold). Credibility eroded. Future asks get scrutinized.

Missed impact: You measure only adoption rate ("90% of recruiters are using it"). You miss that quality declined 5% or that diverse candidate flow dropped. These are real impacts you should know about.

Good measurement gives you:
- Credible results that leadership trusts. "We measured this rigorously. These numbers are real."
- Early warning signals. "Quality's trending down. Let's investigate before it becomes a problem."
- Investment justification. "Year 1 cost was $150K. ROI is $X by month 18. Worth the investment."
- Continuous improvement data. "This use case is working well. That use case isn't. Let's adjust."

Measurement Methodology: Four Approaches

Not all measurement approaches are equal. Different approaches have different strengths and weaknesses. Choose based on what you're measuring and your ability to isolate variables.

Approach 1: Before/After (Simple but Risky)

Method: Measure current state before you implement AI. Wait 30-90 days. Measure again after. Compare.

Example - Recruiting:
- Before AI (month -1): Average time-to-fill across 20 hires = 45 days
- After AI (day 90): Average time-to-fill across 20 new hires = 32 days
- Conclusion: AI saved 13 days (29% improvement)

Strengths:
- Simple to understand and communicate
- No complex statistical analysis needed
- Uses actual operational data

Risks (the big one: confounds):
- What if you also hired 2 experienced recruiters? Their work alone might reduce time-to-fill.
- What if the labor market improved? More candidates available = faster fills.
- What if you increased salary offers? Better pay attracts better candidates faster.
- What if you changed job descriptions to be easier to fill? Less competition = faster hires.
- Any of these changes alone could explain the improvement. Did AI cause it, or did other factors?

Use when:
- You're confident no major changes happened between before and after
- You're measuring steady-state operations (recruiting happens constantly; you're comparing similar periods)
- You document what else changed and try to estimate its impact

Template:
- Baseline metric (before)
- Measurement period and conditions (what dates? what jobs? what market conditions?)
- Post-AI metric (after)
- Known confounds documented (what else changed?)
- Conservative estimate of AI's impact after accounting for confounds

Approach 2: Control Group (Better, But Complex)

Method: Split your users into treatment (uses AI) and control (doesn't). Run them in parallel. Compare outcomes.

Example - Recruiting:
- Group A (treatment): 3 recruiters use AI screening for 2 months. Hire 20 people. Track time-to-fill, cost-per-hire, hire quality.
- Group B (control): 3 other recruiters use manual screening for same 2 months. Hire 20 people. Track same metrics.
- Compare: Did Group A outperform Group B?

Strengths:
- Controls for confounds (both groups experienced same market, same hiring volume, same hiring manager quality)
- Isolates AI's impact
- Defensible scientifically

Risks:
- Control group might resent being excluded ("Why do we have to do it the hard way?")
- Might not be able to run truly parallel processes (if you have 5 recruiters, can you really pull 3 away to be "control"?)
- Takes longer (need long enough period to see real impact)
- Control group members might secretly use AI anyway

Use when:
- You can afford to run two parallel processes
- You have enough volume to split meaningfully
- You can manage the people dynamics (explain why control group matters)

Template:
- Treatment group definition (who uses AI? how many?)
- Control group definition (who doesn't? matched on relevant variables)
- Metrics tracked (same for both groups)
- Timeline (how long will you run this parallel?)
- Results (did treatment outperform control?)

Approach 3: Matched Pair (Hybrid)

Method: Match similar hiring cycles or projects. One uses AI, one doesn't. Compare.

Example - Recruiting:
- AI group: 2 recruiters filling senior engineer roles (using AI screening). Track time-to-fill, cost, quality.
- Control group: 2 different recruiters filling similar senior engineer roles (manual screening). Same time period. Same market conditions.
- Match on: job level, hiring volume, candidate pool, recruiter experience
- Compare outcomes

Strengths:
- Uses real operations (not artificial test)
- Controls for role/market differences
- Manageable logistics

Risks:
- Perfect matching is hard (different recruiters have different skills; even if roles are similar, circumstances differ)
- Smaller sample sizes (you're comparing 2 recruiters to 2 recruiters, not 10 to 10)
- Confounded by recruiter skill differences

Use when:
- You want real-world measurement without artificial control groups
- You can match on key variables (job level, timing, market conditions)
- You have enough cycles to find matches

Approach 4: Trend Analysis (Continuous)

Method: Measure your outcome continuously (weekly or monthly). Plot over time. Look for change when you implement AI.

Example - Recruiting:
- Track time-to-fill weekly for 12 weeks
- Weeks 1-4: baseline (before AI)
- Week 5: implement AI
- Weeks 6-12: measure with AI
- Did time-to-fill drop when you implemented AI? If it drops exactly at Week 5 and stays lower, that's evidence AI caused the change.

Strengths:
- Uses all available data
- Clear inflection point (when AI launched)
- Monitors for degradation over time

Risks:
- Other changes might happen at Week 5 (new recruiter hired, market shift, job description changed)
- Requires consistent, clean data
- Need enough data points to see signal through noise

Use when:
- You have clean, continuous data (recruiting happens every week; you have reliable metrics)
- You can identify when AI launched precisely
- You can document what else changed that week

Specific Measurement Approaches by Outcome

Measuring Time Saved (Productivity)

What you're measuring: How much time does the AI actually save? Per task? Per person? Per week?

Approach 1: Time Study (Simple)

Have 10 recruiters manually time themselves on 100 resumes (old way). Then have them time themselves on 100 resumes with AI (new way).

• Manual: average 8 min/resume
  - With AI: average 3 min/resume
  - Time saved: 5 min/resume or 62%

Multiply: 100 resumes × 5 min saved × 20 recruiters × 50 weeks/year = 83,000 hours saved per year (or about $2.5M in recruiter time at loaded cost of $30/hour)

Caveat: This is lab conditions. Real recruiting is messier. But it gives you a baseline.

Approach 2: Tool Logging

Many AI tools log time spent. If your tool logs time, use that data.

• Total time-on-tool from logs per recruiter per week
  - Average per task
  - Compare to historical average (how long did this task take before?)

Caveat: Tool logs might not capture all time. Recruiters might review AI recommendations outside the tool. Logging accuracy depends on user behavior.

Better approach: Combine tool logs + manager estimates. "The tool logs say 15 hours per week on screening. Does that feel right?" Cross-check.

Approach 3: Workload Capacity

Measure how much work people can complete with AI vs. without.

• Before AI: One recruiter can screen 50 resumes per week (500 per year)
  - With AI: One recruiter can screen 150 resumes per week (7,500 per year)
  - Throughput increase: 200%
  - Impact: You can fill roles faster, or hire same volume with fewer recruiters

This is real impact. You're not measuring time saved; you're measuring capacity gained.

Recommendation: Use Approach 3 (workload capacity). It's cleaner than time studies. And it maps directly to business value ("We can fill roles faster" or "We don't need to hire more recruiters").

Measuring Quality of Hire

What you're measuring: Does AI hiring actually select better people? Are they staying? Are they performing?

Approach 1: Hiring Manager Satisfaction (Quick)

Post-hire survey: "How satisfied are you with this hire?" (1-5 scale)

Compare pre-AI hires (month -1) to post-AI hires (day 90).

• Pre-AI: average satisfaction 3.8/5
  - Post-AI: average satisfaction 4.1/5
  - Improvement: modest

Caveat: Subjective. Manager might like hire for wrong reasons (they hired a friend, hire is just charismatic). Not reliable for quality measurement.

Better: Add specific questions: "Did this person have the skills needed?" "Is this person hitting performance targets?" "Would you rehire this person?"

Approach 2: Retention & Performance (Slow but Credible)

Track 1-year retention for cohorts: pre-AI hires vs. post-AI hires.

• Pre-AI cohort (100 people hired month -3 to 0): 85 still employed at 12 months = 85% retention
  - Post-AI cohort (100 people hired month 3 to 6): 90 still employed at 12 months = 90% retention
  - Improvement: 5 percentage points

Also track performance ratings: Do post-AI hires have higher performance ratings at 6 and 12 months?

Caveat: Takes 12 months to see full data. Hard to attribute causation (many factors affect retention). But if you see both higher retention AND higher performance, evidence is stronger.

Better: Track retention + performance + cost-per-hire. If post-AI cohort has better retention, better performance, and comparable or lower cost, you have strong evidence of quality improvement.

Approach 3: Time-to-Productivity

How long until a new hire is productive?

• For engineers: days to first code review
  - For sales: days to first deal
  - For operations: days to first process completion

Compare pre-AI to post-AI cohorts.

• Pre-AI: average 60 days to first code review
  - Post-AI: average 45 days to first code review
  - Improvement: 25% faster ramp

Caveat: Varies by role and by person. Hard to isolate AI's impact. But combined with retention/performance data, it adds another signal.

Recommendation: Use Approach 2 (retention + performance). It takes longer but it's the most credible. Quality is ultimately whether people stay and perform. That's what matters.

Measuring Employee Experience

What you're measuring: How do your HR team members (and job candidates) feel about the experience?

Approach 1: Internal Staff Survey

Post-implementation survey for your HR team: "How has AI changed your experience?" (1-5 scale for multiple questions)

• "The hiring process is faster" (1-5)
  - "The hiring process is fairer" (1-5)
  - "I trust the AI's recommendations" (1-5)
  - "I feel supported in my job" (1-5)
  - "Overall satisfaction with recruiting" (1-5)

Target: average 4.0 or higher

Caveat: Subjective. Self-reported experience ≠ objective reality. People who had bad experiences more likely to respond. Selection bias.

Better: Combine with objective metrics. "Staff say process is 30% faster; tool logs confirm 40% time reduction. Experience aligns with reality."

Approach 2: Candidate Experience

Post-interview survey for candidates (both hired and rejected): "How was your recruiting experience?" (1-5 scale)

• "The process was fair" (1-5)
  - "The process was timely" (1-5)
  - "I understood how I was evaluated" (1-5)
  - "I would recommend applying here again" (1-5)

Caveat: Candidates rejected by AI might rate experience lower (they're upset about outcome, not process). And you might have higher non-response rates from rejected candidates (they didn't get the job; less incentive to respond).

Better: Track application-to-offer time for all candidates (objective). If it's faster, candidates experience speed regardless of whether they report it. Add subjective feedback to provide context.

Approach 3: Diversity Outcomes

% of candidate pool from underrepresented groups before and after AI.

• Pre-AI: 20% of candidates from underrepresented backgrounds. 15% hired.
  - Post-AI: 20% of candidates from underrepresented backgrounds. 12% hired.
  - Change: Diversity outcomes got worse.

Critical: If AI worsens diversity outcomes, it's not working, even if time-to-fill improved. This is a red flag that AI has bias.

Recommendation: Use Approach 1 (internal staff survey) + Approach 3 (diversity outcomes). Track both quickly. Post-90 days, you'll know: Are your people happy? Are diversity metrics stable?

Handling Confounds: Documenting Alternative Explanations

You implement AI. Time-to-hire drops 30%. But what else could explain this?

List of common confounds:
- Recruiter changes (hired new recruiters, got more experienced people)
- Job market (candidate supply increased, easier to fill)
- Job description (changed to be easier to fill, requiring fewer qualifications)
- Salary changes (increased offers, attracted better candidates)
- Recruiting process changes (phone screen moved earlier, weeding out faster)
- Hiring volume (fewer hires = faster average time-to-fill)
- Role mix (hiring different roles now, which happen to fill faster)

How to handle:

• 
  Document what else changed. Sit down month 1. "What changed between month -1 and month 3 besides AI?" Write it down. Recruiter additions? Market conditions? Job descriptions? Be honest.
• 
  Model the impact separately. "Extra recruiter probably accounts for 10% of improvement (we have 20% more capacity). Market conditions might account for 5%. AI accounts for remaining 15%." You don't have to be precise, but make your assumptions visible.
• 
  Be conservative in claims. "Time-to-hire improved 30%. We estimate 50% of that was AI. Other 50% was market conditions and process changes. So AI accounts for about 15 percentage points of the 30% improvement." This is credible and honest.
• 
  Use control group if possible. The cleanest way to account for confounds is parallel measurement. If you can't control, document what happened. Leadership will trust you more if you acknowledge uncertainty than if you claim false precision.

The Measurement Plan Template

Here's the structure for a solid measurement plan. Fill this out per outcome you're measuring.

MEASUREMENT PLAN: Time-to-Fill

What we're measuring:
Average days from job posting to offer acceptance

Why it matters:
Speed is a competitive advantage. Slower hiring means we lose candidates to competitors.

Baseline (current state, month -1):
- Average time-to-fill: 45 days (average of last 100 hires across all roles)
- Sample: recruiting team of 20, hiring across 50 open roles average
- Market: 3% unemployment, candidate-driven market

Measurement approach:
- Before/after comparison (month -1 vs. day 90)
- Sample: 100 hires pre-AI, 100 hires post-AI
- Measurement timing: Day 90 post-implementation

How we'll measure:
- Data source: ATS (applicant tracking system)
- Definition: "Days from job posting to offer acceptance"
- Calculation: median and mean (mean can be skewed by outliers)

Confounds we'll track:
- Recruiter additions/departures
- Job level mix (entry-level roles fill faster than exec roles)
- Market conditions (if market improves, time-to-fill improves regardless of AI)
- Salary changes (if we increased offers, that could drive faster fills)

Success target:
- Time-to-fill: 32 days (33% improvement)
- But we'd accept 35-38 days (22-15% improvement) depending on confounds

Acceptable variance:
- If we get 25 days: likely AI contributed, but other factors helped too. Claim 60-70% to AI.
- If we get 35 days: modest improvement, but in range. Claim 40-50% to AI.
- If we get 45 days: no improvement. AI might not be working, or other factors worsened it.

Deliverable: Your Measurement Plan

Create one of these per outcome. 2 pages each.

Page 1: Outcome Definition & Baseline
- What are we measuring? (specific metric)
- Why does it matter? (business context)
- Baseline (current state, documented)

Page 2: Measurement Approach & Success Criteria
- Measurement method (before/after, control group, matched pair, trend analysis)
- Sample (who, how many, what time period)
- How we'll measure (data source, definition, calculation)
- Confounds we'll track
- Success targets (what's good?)

Key Takeaways

• Before/after is simple but risky. Confounds are everywhere. If you use before/after, document confounds and be conservative in claims.
  - Control group is better but harder. Parallel measurement (treatment vs. control) isolates AI's impact. Use if you can.
  - Combine objective and subjective data. Tool logs + manager estimates. Time-to-fill + hiring manager satisfaction. Retention + performance ratings.
  - Quality and speed are both important. Don't optimize for time-to-hire if quality drops. Measure both.
  - Document uncertainty. "AI accounts for 50% of improvement; other factors account for 50%" is more credible than "AI saved 30%."
  - Measurement prevents overstating impact. Better to conservatively claim 15% than to aggressively claim 30% and lose credibility.

FAQ

Q: How long until we see business impact?

A: Productivity (time saved): immediately (within 30 days). Quality (retention, performance): 90+ days. True ROI (payback): 180+ days.

Q: What if our control group finds that AI doesn't actually help?

A: Then you've learned something important. The tool isn't working as expected. You investigate why: accuracy too low? Wrong use case? Bad training? This is valuable feedback. Better to know now than to expand broken AI.

Q: Should we measure candidate experience if we're using blind resume review?

A: Yes, but differently. Measure: speed (application-to-interview time). Measure: fairness (do candidates from different backgrounds get similar interview rates?). Measure: clarity (do candidates understand the process?).

Q: Can we use historical data as our "before"?

A: Risky. Historical data is from different recruiters, market conditions, hiring volumes. Better to get a month of baseline data during the same period you're measuring post-AI. But if you must use historical, document all confounds clearly.

What's Next

You've measured outcomes. You have data on time saved, quality, adoption, employee experience. Now you need to report this to leadership in a way they understand and trust. Next lesson: Reporting HR AI ROI to CHRO, CFO, and Board.

Your measurement tells you what happened. Your reporting tells the story.

You have metrics defined. Now you need to actually measure them. This sounds simple but it's not. How do you measure "time saved"? Do you count recruiter hours? Include hiring manager review time? What about the time hiring managers spend managing rejected candidates?

This lesson teaches you measurement methodology. You'll learn to set up before/after comparisons, control for confounds, and measure outcomes with confidence. You'll understand when you can claim impact and when you need to be cautious.

Why This Matters for HR Leaders

Bad measurement leads to false claims. "We saved 30% time" might be true, or you might be counting wrong. Bad measurement also leads to wasted effort: collecting data that doesn't answer your question.

Good measurement gives you credible results that leadership trusts.

Measurement Methodology

Approach 1: Before/After (Simple but Risky)

Method: Measure current state (before AI). Implement AI. Measure after (30/60/90 days).

Example - Recruiting:
- Before: Average time-to-fill = 45 days
- After (Day 90): Average time-to-fill = 32 days
- Conclusion: AI saved 13 days (29% improvement)

Strength: Simple. Clear. Easy to explain.

Risk: Confounds. What if you also hired more experienced recruiters? What if hiring volume increased? What if the market for candidates improved?

Use when: You're confident no major changes happened between before and after.

Approach 2: Control Group (Better)

Method: Split users into treatment (uses AI) and control (doesn't use AI). Compare outcomes.

Example - Recruiting:
- Group A (treatment): 3 recruiters use AI screening (20 hires over 90 days)
- Group B (control): 3 recruiters use manual screening (20 hires over 90 days)
- Measure: time-to-fill, cost-per-hire, quality of hire for both groups
- Compare: Did Group A outperform Group B?

Strength: Control for confounds. You know it's the AI, not other changes.

Risk: Requires splitting your team. Control group might feel like they're being tested unfairly.

Use when: You can afford to run two parallel processes.

Approach 3: Matched Pair (Hybrid)

Method: Match similar hiring cycles/roles. One uses AI, one doesn't. Compare.

Example - Recruiting:
- AI group: 2 recruiters filling senior engineer roles (AI-screened)
- Control group: 2 different recruiters filling similar senior engineer roles (manual screening)
- Match on: job level, hiring volume, market conditions
- Compare outcomes

Strength: Control for role differences. Uses real operations (not artificial test).

Risk: Hard to match perfectly. Differences in recruiter skill might confound results.

Use when: You want real-world measurement without artificial control group.

Approach 4: Trend Analysis (Continuous)

Method: Measure outcome continuously over time. Look for change when you implement AI.

Example - Recruiting:
- Track time-to-fill weekly for 12 weeks
- Week 1-4: baseline (before AI)
- Week 5: implement AI
- Week 6-12: measure trend with AI
- Did time-to-fill drop when you implemented AI?

Strength: Uses real data. Clear inflection point when AI launched.

Risk: Other changes might also happen at Week 5 (e.g., new recruiter hired, market shift).

Use when: You have clean, continuous data.

Specific Measurement Approaches by Outcome

Measuring Productivity (Time Saved)

Method 1: Time Study
- Have 3 recruiters manually screen 20 resumes (old way). Time each one.
- Have same 3 recruiters screen 20 resumes with AI (new way). Time each one.
- Calculate average time savings.
- Example: Manual = 8 min/resume. AI = 3 min/resume. Savings = 5 min/resume or 62%.

Caveats:
- Small sample (only 20 resumes). Might not be representative.
- Test environment (artificial; real recruiting is messier).

Better: Ask 10 recruiters to track their time on 100 resumes (AI-assisted). Compare to historical average (8 min/resume). If actual is 3 min/resume, you've saved time.

Method 2: Tool Logging
- If the tool logs time spent (many do), use that data.
- Total time-on-task from tool logs. Average per task.

Caveats:
- Tool might not capture all time (e.g., recruiters review screened candidates outside the tool).
- Logging accuracy depends on user behavior.

Better: Combine tool logs with manager estimates: "Roughly how much time does AI save you per week?" Average of both approaches.

Measuring Quality of Hire

Method 1: Hiring Manager Satisfaction
- Post-hire survey: "How satisfied are you with this candidate's performance?" (1-5 scale)
- Compare pre-AI hires to post-AI hires (same hiring manager, same role).

Caveats:
- Subjective. Manager might like candidate for wrong reasons.
- Only works if same hiring manager evaluates both.

Better: Add specific questions: "Did this hire have the right skills?" "Is this person hitting performance targets?"

Method 2: Retention & Performance
- Track 1-year retention: % of hires still employed at 12 months
- Track performance: Manager ratings at 6 months and 12 months
- Compare cohorts: pre-AI hires vs. post-AI hires

Caveats:
- Slow (you need 12 months of data).
- Hard to attribute causation (many factors affect retention and performance).

Better: Track both retention and performance. If post-AI cohort has better retention AND higher performance, you have stronger evidence.

Method 3: Time-to-Productivity
- How long does it take a new hire to become productive? (e.g., first month revenue, first project completion)
- Compare pre-AI to post-AI cohorts

Caveats:
- Varies by role. Engineering ≠ sales ≠ operations.

Better: Use role-specific productivity metrics. For engineers: days to first code review. For sales: days to first deal.

Measuring Employee Experience

Method 1: Employee Survey
- Post-implementation survey: "How has AI changed your experience?" (1-5 scale for multiple questions)
- Questions:
- "The hiring process is faster" (1-5)
- "The hiring process is fairer" (1-5)
- "I trust the AI's recommendations" (1-5)
- "Overall satisfaction with hiring" (1-5)

Caveats:
- Subjective. Self-reported experience ≠ objective reality.
- Selection bias (people with strong opinions more likely to respond).

Better: Combine with objective metrics. "Employee survey says process is 30% faster. Tool logs confirm 40% time reduction."

Method 2: Candidate Experience
- Post-interview survey (for candidates, whether hired or not): "How was the hiring process?"
- Questions:
- "Process was fair" (1-5)
- "Process was timely" (1-5)
- "I understood how I was evaluated" (1-5)

Caveats:
- Candidates rejected by AI might rate experience lower (bias).

Better: Track application-to-offer time for all candidates (objective). Add subjective feedback to context.

Method 3: Diversity Outcome
- % of candidate pool from underrepresented groups before and after AI
- % of hired candidates from underrepresented groups before and after AI
- Do ratios improve, stay same, or worsen?

Critical: If AI worsens diversity outcomes, it's not working, even if other metrics are good.

Handling Confounds & Alternative Explanations

You implement AI. Time-to-hire drops 30%. But what if:

• You hired 2 more recruiters (that would also drop time-to-hire)
  - Job market improved (more candidates, faster to fill)
  - You changed the job description (easier to fill)
  - You increased salary (attracted better candidates faster)

How to handle:

• 
  Document what else changed. "We hired 1 recruiter. Market conditions X. Salary Y." Everything that could affect the metric.
• 
  Model the impact separately. "Extra recruiter probably accounts for 10% of the improvement. AI accounts for 20%."
• 
  Be conservative in claims. "Time-to-hire improved 30%. We estimate AI accounted for 20% of that improvement, other factors 10%." Better to be accurate than overstate.
• 
  Use control group if possible. The best way to account for confounds is parallel measurement (treatment vs. control).

The Measurement Plan Template

What we're measuring: Time-to-fill, cost-per-hire, quality of hire

Baseline (current state):
- Time-to-fill: 45 days (average of last 100 hires)
- Cost-per-hire: $2,400 (recruiting budget / # hires)
- Quality: 80% 1-year retention rate

Measurement approach:
- Before/after comparison (AI screening vs. manual screening)
- Sample: 50 hires pre-AI, 50 hires post-AI
- Timeline: Day 90 post-implementation

How we'll measure:
- Time-to-fill: Days from job posting to offer acceptance
- Cost-per-hire: (Recruiting costs) / (# offers accepted)
- Quality: Track 1-year retention rate for post-AI hires

Confounds we'll track:
- Recruiter changes
- Job level mix (are we filling different roles?)
- Market conditions
- Salary changes

Success target:
- Time-to-fill: 30 days (33% improvement)
- Cost-per-hire: $2,100 (12% improvement)
- Quality: 82%+ 1-year retention (maintain or improve)

Deliverable: Your Measurement Plan

Create a 2-page document per outcome:

Page 1: Outcome Definition
- What are we measuring? (e.g., "time-to-fill")
- Why does it matter?
- Baseline (current state)

Page 2: Measurement Approach
- Method (before/after, control group, trend analysis)
- Sample (who, how many)
- Timeline
- Confounds we'll track
- Success target

Key Takeaways

• Before/after is simple but risky (confounds). Control group is better but harder.
  - Combine objective (tool logs, time tracking) and subjective (surveys, interviews).
  - Document confounds. Be honest about what else might explain results.
  - Be conservative in claims. "AI accounts for 60% of improvement, other factors 40%" is more credible than "AI caused 100% improvement."
  - Quality matters as much as speed. Don't optimize for time-to-hire if quality drops.

FAQ

Q: How long until we see business impact?

A: Productivity (time saved): immediately. Quality (retention, performance): 90+ days. Business ROI: 180+ days.

Q: What if one group using AI outperforms due to recruiter skill, not AI?

A: Use matched-pair approach (similar recruiters, similar roles). Or track individual recruiter performance (with AI vs. without).

Q: Should we measure employee experience for internal staff vs. job candidates?

A: Both. Internal: are recruiters adopting and satisfied? External: are candidates having better experience?

What's Next

You've measured outcomes. Now you need to report them to leadership. That's the next lesson: Reporting HR AI ROI to the CHRO, CFO, and Board.

Your measurement tells you what happened. Your reporting tells the story.