Overview

You launched your resume screening tool 30 days ago. Your PM says it's "going well." Your CEO asks: "Are we winning?" You don't have a clear answer.

That's a problem. "Going well" is not a success metric. You need explicit, measurable criteria that answer: Are we winning? Should we keep going? Is this worth the investment?

This lesson teaches you to define success for AI initiatives, not at the end when you're evaluating what happened, but upfront so you know what winning looks like before you start. You'll learn the difference between leading indicators (tells you fast if you're on track) and lagging indicators (the ultimate measure of success). You'll build an OKR framework for AI initiatives. And you'll understand when to kill a failing pilot vs. when to pivot and try again.

Why This Matters for HR Leaders

Success criteria serve three functions:

1. They clarify what you're actually trying to achieve. "We're implementing resume screening because we want to save time and improve quality." But which matters more? If you save 40% of time but quality drops 10%, is that a win? You need to know upfront.

2. They protect you from false positives. A tool can look successful because people like it (high satisfaction) but not actually be achieving business results. Explicit success criteria force you to measure actual impact, not just vibes.

3. They give you an off-ramp. Every initiative should have a clear point (usually 30-90 days) where you decide: keep going, pivot, or kill. Without that decision point, failing initiatives drag on for months consuming resources and generating false hope.

Leading vs. Lagging Indicators for HR AI

Every AI initiative has two categories of metrics: leading indicators (early warning signs) and lagging indicators (the ultimate truth).

Leading Indicators tell you fast (within 2-4 weeks) whether the initiative is on track. They guide daily decisions. "Are people using the tool?" "Is it fast enough?" "Do they like it?"

Lagging Indicators tell you whether it actually moved the needle on business outcomes. They take longer to measure (30-90 days) but are the final truth. "Did time-to-fill actually drop?" "Did quality improve?" "Did we save money?"

You need both. Leading indicators tell you whether to keep going. Lagging indicators tell you whether it was worth it.

Phase 1 Initiatives: Quick Wins

Example: Resume Screening AI

Leading Indicators (measure daily/weekly):
- Adoption: % of recruiting team using the tool daily (target: 80% by day 30)
- Ease of use: Average rating on "easy to use" survey (target: 4.0/5.0 by day 7)
- Tool performance: Average screening time per candidate (target: 60% of manual time by day 14)
- Error rate: % of tool-screened candidates flagged as incorrect (target: <5% by day 30)
- Support burden: # of help requests from users (target: <2 per week by day 30)

Lagging Indicators (measure at 30/60/90 days):
- Time savings: Actual hours saved on screening per week (target: 10+ hours/week)
- Quality impact: Hiring manager satisfaction with screened candidates (target: ≥85%)
- Process speed: Time from posting to initial candidate review (target: 30% reduction)
- Retention of hires: Conversion of screened candidates to offers (target: ≥85%)

Decision points:
- Day 7: Leading indicators on "ease of use" and tool performance
- Green: Proceed to broader pilot
- Yellow: Adjust training or workflow; reassess at Day 14
- Red: Consider vendor alternatives or process change
- Day 30: Leading indicators on adoption and error rate
- Green: Full team rollout
- Yellow: Extended pilot with process changes; reassess at Day 45
- Red: Kill initiative or pivot to different use case
- Day 90: Lagging indicators on time savings and quality
- Green: Declare success; optimize and measure ongoing
- Yellow: Savings are lower than expected; identify causes; consider reducing scope
- Red: Initiative didn't deliver; reallocate resources

Phase 2 Initiatives: Process Integration

Example: AI-Assisted Performance Review Insights

Leading Indicators (measure weekly):
- Manager adoption: % of managers viewing AI insights (target: 50% by week 4, 75% by week 8)
- Feedback quality: Manager rating on "insights are relevant" (target: ≥3.5/5.0 by week 4)
- Time engagement: Average time spent reviewing insights (target: 10+ min per manager)
- Technical performance: Tool uptime and data accuracy (target: 99%+ uptime, 95%+ accuracy)

Lagging Indicators (measure at 30/60/90 days):
- Impact on conversations: % of managers who report that insights improved their review conversations (target: ≥60%)
- Quality improvement: Performance calibration time reduction (target: 20% faster calibration)
- Manager confidence: Manager rating on "I feel more prepared for reviews" (target: ≥4.0/5.0)
- Retention: Voluntary attrition of high-performers in pilot population (target: <5% for reviewed group)

Decision points:
- Week 4: Leading indicators on adoption and feedback relevance
- Green: Expand pilot to second department
- Yellow: Adjust training/messaging; reassess at Week 6
- Red: Process isn't right; consider redesigning before broader rollout
- Week 8: Leading indicators on sustained adoption
- Green: Plan broader rollout
- Yellow: Adoption plateauing; investigate barriers; address before rollout
- Red: Managers don't see value; kill or pivot
- Month 3: Lagging indicators on quality and retention
- Green: Declare success; invest in optimization
- Yellow: Some positive impact but lower than expected; measure longer-term retention
- Red: No impact on outcomes; reallocate resources

Phase 3 Initiatives: Strategic Decisions

Example: Succession Planning AI

Leading Indicators (measure weekly/biweekly):
- Data quality: % of required fields populated and accurate (target: ≥95%)
- Leadership engagement: % of senior leaders reviewing AI recommendations (target: ≥80%)
- Model validation: % of AI recommendations validated by senior leaders as reasonable (target: ≥80%)
- Governance: Issues logged and resolved within SLA (target: 100% escalations resolved within 48 hours)

Lagging Indicators (measure at 90/180 days):
- Succession readiness: % of key roles with identified successors (target: increase from 40% to 75%)
- Leadership development: # of identified high-potential leaders enrolled in development programs (target: +20)
- Retention of identified talent: Attrition rate of AI-identified high potentials vs. control group (target: 5% lower attrition)
- Executive confidence: CHRO and CEO rating on "AI succession insights are actionable" (target: ≥4.0/5.0)

Decision points:
- Week 4: Leading indicators on data quality and leadership engagement
- Green: Proceed with pilot
- Yellow: Data issues identified; fix before broader rollout; reassess at Week 6
- Red: Leadership not engaged; solve trust issue before continuing
- Week 8: Leading indicators on model validation
- Green: Prepare for broader pilot
- Yellow: Model needs refinement; work with vendor on training data; reassess at Week 10
- Red: Model output isn't valid; may not be solvable with current approach
- Month 3: Lagging indicators on succession readiness
- Green: Plan broader rollout; build into succession planning process
- Yellow: Positive direction but slower than expected; measure at 6 months
- Red: No impact on succession readiness; evaluate whether different approach needed

OKR Framework for HR AI Initiatives

Use Objectives and Key Results (OKRs) to structure success criteria. OKRs separate the "what we're trying to achieve" (Objective) from the "how we'll know we got there" (Key Results).

Example OKR for Resume Screening:

Objective: Speed up recruiting and improve hire quality through AI-assisted candidate screening

Key Results (at 90 days):
1. 80%+ adoption by recruiting team (leading: adoption, lagging: sustained use)
2. 30% reduction in time-to-screen per candidate (lagging: time savings)
3. 85%+ hiring manager satisfaction with screened candidates (lagging: quality)
4. <5% error rate in AI screening (leading: quality, caught before affecting hiring)

If Key Results are hit: Initiative is a success. Optimize and scale.

If 1-2 Key Results are missed: Partial success. Extend pilot, identify issues, and reassess.

If 3+ Key Results are missed: Initiative failed. Kill or pivot.

Example OKR for Compensation Equity Analysis:

Objective: Identify and remediate pay inequities using AI analysis

Key Results (at 90 days):
1. Analysis completed for 100% of employee population (leading: data quality)
2. 12+ statistically significant pay inequities identified (lagging: business outcome)
3. CFO and CHRO executive sign-off on remediation plan (leading: stakeholder buy-in)
4. Remediation plan approved by legal team (leading: compliance)

If Key Results are hit: Initiative successful. Execute remediation plan.

If Key Results 1 and 4 are hit but not 2-3: Proceed carefully. You may have cleaner data than you thought, but need to validate findings.

If Key Result 1 is missed: Data quality issue. Halt initiative until resolved.

Building Your Success Criteria Template

For each AI initiative, use this template:

Initiative: [Name]

Objective (what we're trying to achieve in plain language):
"[Function] will [achieve outcome] using AI, enabling [business impact]."

Example: "Recruiting will screen candidates faster while improving quality, enabling us to compete with speed-hiring competitors and land top talent."

Phase 1 Metrics (measure continuously, 7-30 days):

Metric
Leading/Lagging
Target
Owner
Frequency

Adoption rate
Leading
80% by day 30
[Name]
Daily

Ease of use rating
Leading
4.0/5.0 by day 7
[Name]
Daily

Tool performance (time per unit)
Leading
60% of manual
[Name]
Daily

Help/support requests
Leading
<2/week by day 30
[Name]
Weekly

Error rate
Leading
<5% by day 30
[Name]
Weekly

Phase 2-3 Metrics (measure at day 30, day 60, day 90):

Metric
Leading/Lagging
Day 30 Target
Day 60 Target
Day 90 Target
Owner

Time/cost savings
Lagging
,
20%
30%
[Name]

Quality impact
Lagging
,
Visible
Measurable
[Name]

Stakeholder satisfaction
Leading
75%
85%
90%+
[Name]

Business outcome
Lagging
,
,
Achieved
[Name]

Day-30 Decision Gate:

• Green: Proceed to broader rollout/pilot
  - Yellow: Extend pilot with specific improvements; reassess at Day 45
  - Red: Kill initiative or pivot to different use case

Day-90 Decision Gate:

• Green: Declare success; move to optimization
  - Yellow: Partial success; extend measurement period; plan for Year 2
  - Red: Failed initiative; reallocate resources

The Decision Tree: Keep, Pivot, or Kill

When you hit a decision gate and the metrics are mixed, use this tree:

Decision Point Reached (Day 30, 60, or 90)

↓

Are leading indicators green?
(Adoption ≥60%? Technical performance good?)

├─ YES → Move to next question
│ ├─ Are lagging indicators tracking?
│ │ ├─ YES → Keep going (green)
│ │ └─ NO → Pivot (address root cause)
│ │ └─ Is root cause fixable in 30 days?
│ │ ├─ YES → Pivot and reassess
│ │ └─ NO → Kill
│ │
└─ NO → Are we failing because of process/training issues?
├─ YES → Pivot (improve training/change management)
│ └─ Reassess in 30 days
└─ NO → Kill (not a fixable problem)

Specific Examples:

Example 1: Resume screening, Day 30
├─ Leading: Adoption at 75%, tool performance good
├─ Lagging: Time savings only 15% (target 30%)
└─ Decision: PIVOT
└─ Root cause: Recruiters still doing manual review because they don't trust AI
└─ Fix: Pair screening AI with bias audit; show recruiters the AI reasoning
└─ Reassess: Day 60

Example 2: Performance AI, Day 60
├─ Leading: Adoption at 45% (target 75%)
├─ Process: Managers say "insights aren't relevant to our calibration process"
└─ Decision: KILL or PIVOT
├─ Option 1: Kill (spend on something managers actually want)
└─ Option 2: Pivot (redesign to fit manager workflows better)

Example 3: Succession planning, Day 30
├─ Leading: Data quality issues (only 80% of required fields populated)
├─ Technical: Can't run reliable model without clean data
└─ Decision: PAUSE and FIX
└─ Fix data quality in next 30 days
└─ Reassess: Day 60

Common Mistakes in Setting Success Criteria

Mistake 1: Impossible targets
"We'll reduce time-to-fill from 45 days to 20 days." That might be possible, but not in 90 days with a new tool. Set realistic targets.

Mistake 2: Too many metrics
You end up tracking 20 things; you can't prioritize. Pick 4-6 metrics that matter.

Mistake 3: Metrics that don't match the initiative
If you're testing resume screening for speed, measuring "quality" is secondary until later. Don't weight them equally upfront.

Mistake 4: No leading indicators
You wait 90 days to find out the initiative failed. Leading indicators tell you at day 30 whether to keep going.

Mistake 5: Targets set by vendor, not by you
Vendors will say "our customers see 50% time savings." Your company is different. Set your own targets based on your baseline.

Mistake 6: Moving the goalposts
"We targeted 80% adoption, got 65%, so we're changing the target to 65%." Don't do this. If you miss, that's data. Learn why.

>
CALLOUT BOX: The Success Criteria Conversation

Before you start the implementation, have a working session with your core team:

• "What does success look like for this initiative?" (Objective)
  - "How will we know we got there?" (Key Results)
  - "What's the earliest we could know?" (Leading indicators at 7-14 days)
  - "When do we make the keep/pivot/kill decision?" (Decision gates at 30/60/90 days)
  - "What could go wrong, and what would that look like?" (Risk scenarios)

Write it down. Share with leadership. Reference it throughout implementation.

Case Study: Using Metrics to Make Hard Decisions

A 2,000-person company implemented an AI-driven attrition prediction model. Here's how they used success criteria to manage it.

Target: Identify 80% of employees likely to leave within 90 days, enabling proactive retention conversations.

Leading Indicators (Week 2):
- Model accuracy on historical data: 78% (target: 85%)
- Manager engagement: 40% of managers accessed predictions (target: 60%)

Assessment: Model accuracy is slightly below target, but workable. Manager engagement is lower than hoped.

Action: Extended training and communication to managers. Reassessed at Week 4.

Leading Indicators (Week 4):
- Model accuracy: Improved to 82% (getting closer)
- Manager engagement: 55% (moving in right direction)
- Error case analysis: Several false positives; identified pattern in data

Assessment: Progress on both fronts. Accuracy improvement suggests model refinement is working.

Action: Continued with pilot. Monitored false positive rate carefully.

Lagging Indicators (Day 90):
- Identified 23 at-risk employees
- Proactive conversations happened with 18 (78%)
- Retention: 14 of 18 stayed (78% retention rate)
- Counterfactual: Typical attrition would have been 5-7 of 18 leaving
- Impact: Saved ~10 employees, estimated $2M in replacement costs

Final Assessment: Initiative successful. Deployed to all managers.

The key: They had clear metrics, decision gates, and the discipline to pivot (extend training) when needed, rather than abandoning the initiative or declaring victory prematurely.

Deliverable: Your Success Criteria Document

Create a one-page document per initiative:

Section 1: OKR Summary
Objective + 4-6 Key Results

Section 2: Metric Details
A table with leading and lagging indicators, targets, and frequency

Section 3: Decision Gates
When and how you'll decide to keep, pivot, or kill

Section 4: Risk Scenarios
"If this goes wrong, it will look like X. We'll address it by doing Y."

What to Do Monday Morning

• 
  Write the Objective for each initiative. Not "implement the tool." Something like: "Speed up recruiting while improving quality" or "Identify and fix pay inequities."
• 
  Define 4-6 Key Results for each. Use the templates I provided. Mix leading and lagging.
• 
  Set realistic targets. Based on your baseline, industry benchmarks, and vendor estimates. Ask: "Is this achievable in 90 days?"
• 
  Schedule decision gates. Day 30, 60, 90. Mark them on your calendar. Don't skip them.
• 
  Share with your team and leadership. Everyone should know what success looks like.
• 
  Build daily/weekly tracking. Leading indicators you can check frequently (not just at decision gates).

Key Takeaways

• Leading indicators tell you fast whether to keep going. Lagging indicators tell you whether you succeeded. You need both.
  - Success criteria should be set before you start, not after. Otherwise you're justifying whatever happened.
  - Four to six metrics is plenty. More than that and you lose focus.
  - Decision gates are non-negotiable. Day 30, 60, 90, at each one, you decide: keep going, pivot, or kill.
  - Moving the goalposts is death by a thousand cuts. If you miss a target, that's data. Learn why. Don't just adjust the target.
  - Partial success is better than complete failure, but worse than real success. Have a conversation: does partial success justify continuing, or should we reallocate?

FAQ

Q: What if we miss a target at Day 30 but leadership wants to keep going?

A: That's fine, but be explicit about it. "We targeted 80% adoption, got 65%. We think this is fixable with better training. We're extending the pilot 30 days." Document the decision and the reason. Reassess at Day 60.

Q: What if a lagging indicator won't be measurable until Day 180?

A: You can't wait that long for a decision. Identify an earlier proxy (leading indicator). "We won't measure 'new hire retention at 1 year' until 12 months, so we're measuring 'hiring manager satisfaction' at 90 days as a leading indicator."

Q: Should success criteria be the same across all initiatives?

A: No. Phase 1 initiatives (quick wins) should have faster payback and higher adoption targets. Phase 3 initiatives (strategic) can tolerate longer timelines and focus more on governance and quality than speed.

Q: What happens if an initiative hits some Key Results but not others?

A: That's a pivot, not a kill. "We hit adoption and speed targets but missed quality targets. Let's extend the pilot, fix the quality issue, and reassess." It's still worth solving.

Q: Do we report success criteria to the board?

A: Yes, if it's a significant investment (≥$100K). Frame it as "here's what we're trying to achieve, here's how we'll measure success, and here's where we stand." It shows rigor and accountability.

What's Next

You've set success criteria at the initiative level. Now you need to align those initiatives with your enterprise AI strategy. Your HR AI work doesn't happen in a vacuum. It connects to IT, Legal, and Finance. That's the subject of the next lesson: Aligning HR AI Strategy with Enterprise AI Strategy.

Your success criteria tell you whether individual initiatives work. Enterprise alignment tells you whether they fit the bigger organizational picture.