When the URAC reviewer arrived, the director of pharmacy thought the hard part would be the workflow. She had spent months building the AI-assisted prior-authorization process, embedding the verification step, instrumenting the audit trail. What she had not prepared for was the question the reviewer asked in the first hour, sitting in the break room watching a technician submit a PA: "How do you know this person is competent to use this tool?" The director started to answer, "They're trained, they're careful, they've been doing this for years," and then stopped, because she heard how it sounded. It was an assurance, not an answer. She could not produce a record of what that technician had been trained on, when, against what standard, or how anyone had confirmed the training took. She had a workforce she believed was competent and not one shred of evidence that it was. The reviewer was not doubting her people; the reviewer simply could not credit a competence that could not be shown. This lesson is about the competency half of the URAC (Utilization Review Accreditation Commission) user track, the part that asks not "do you have good tools" but "can you prove the people using them are prepared," and how a pharmacy leader builds the competency and documentation that turns a believed-competent workforce into a demonstrably competent one.

Why Competency Is a Precondition, Not a Nicety

It is tempting to treat staff competency as a soft, secondary expectation, the human-resources tail of an AI program, nice to have once the real work of building tools and workflows is done. The L1 documentation lesson named why this is exactly backwards, and at the strategist level the point sharpens into an operating principle: competency is not a nicety alongside the tools; it is a precondition for deploying the tools safely at all. The reasoning is specific to how AI fails. A generative model produces fluent, confident output whether it is right or wrong, and that confident fluency is precisely what disarms an unprepared user. A pharmacist who does not understand that an AI tool can fabricate a coverage criterion with the same assured tone it uses when it is correct will not bring the skepticism that catches the fabrication. A technician who does not grasp the cardinal rule, that AI supports the pharmacist's judgment and never replaces it, may treat an AI suggestion as a decision rather than a prompt.

This produces a counterintuitive but load-bearing truth for a leader allocating attention: an incompetent user of a capable AI tool can be more dangerous than no tool at all. The manual process the AI replaced was slow, but it did not generate plausible, confident, wrong answers and hand them to someone with no defense against them. Deploying a capable tool to staff who do not understand its failure modes does not add a safety layer; it adds a fluent source of potential error in the hands of people who do not know to distrust it. For a pharmacy leader, this reframes the competency program from a compliance checkbox into a genuine patient-safety control, on the same footing as the verification standard itself, because the verification standard only works if the people executing it understand what they are verifying against and why. Competency is the human foundation the entire verification discipline rests on, and a leader who underinvests in it has built a program that looks complete on the org chart and is hollow at the point where a person meets a tool.

An incompetent user of a capable AI tool can be more dangerous than no tool, because confident, fluent output gives a false sense of reliability that an unprepared person cannot defend against. Competency is not a nicety; it is a precondition for safe deployment.

The Two-Part Test: Real and Documented

The competency expectation has two parts, and a leader has to satisfy both, because each fails differently. The first part is that the competency is real: the staff genuinely understand the tools, the failure modes, the verification discipline, and the cardinal rule. The second part is that the competency is documented: the pharmacy can show who was trained, on what, when, and how the training was confirmed. These are distinct, and a pharmacy can hold one without the other, to its detriment.

Consider the two failure modes. A pharmacy with real competency but no documentation has genuinely prepared staff and no way to prove it, which is the director's situation in the opening: the people are ready, but to a reviewer the readiness is invisible, indistinguishable from an untrained workforce, because the reviewer can only credit what is shown. That pharmacy fails the review despite doing the substance right, which is maddening but correct from the reviewer's position. The opposite failure is a pharmacy with documentation but no real competency: a stack of completion certificates from a module nobody absorbed, training that was clicked through rather than learned, a paper trail that attests to a competence that does not actually exist. That pharmacy may pass a shallow review and fail in the real world, when an underprepared pharmacist misses the fabricated criterion the documentation claimed they were trained to catch. The leader's obligation is to build for both: training rigorous enough that the competency is genuinely real, and documentation complete enough that the real competency is demonstrable. Neither alone is sufficient, and a program optimized for the paperwork at the expense of the substance is arguably worse than one with neither, because it manufactures false confidence in both the pharmacy and the reviewer.

What Pharmacy AI Competency Actually Covers

To build competency rather than the appearance of it, a leader has to be concrete about what an AI-using pharmacist or technician actually needs to understand, because vague "AI awareness" training produces vague, unreliable competence. The content is specific to the pharmacy context and follows directly from the program's spine. Staff need to understand what AI is and is not, enough to know that a generative model predicts plausible text rather than retrieving verified truth, which is why it can hallucinate. They need to understand the specific failure modes that matter in pharmacy: the invented coverage criterion, the wrong renal dose, the missed or fabricated drug interaction, the diagnosis the record does not support, because these are the errors that reach patients, and a person who cannot name them cannot watch for them.

They need to understand and internalize the cardinal rule operationally, not just as a slogan: that an AI-surfaced clinical signal is a prompt to think, never a verdict to accept, and that the human who verifies and signs owns the clinical call no matter what the AI produced. They need to know the verification standard for their actual workflow, what specifically must be checked against the chart, the formulary, the payer criteria, before an AI-touched output takes effect, because competency that does not translate into the concrete verification steps of the job is decorative. And they need to understand the documentation and PHI (protected health information) obligations around their AI use: that AI-touched clinical work must produce a record, and that patient information fed to or produced by AI tools carries the same protection it always did. A leader who maps the competency content to these concrete, pharmacy-specific understandings builds a workforce that is competent at the things that actually keep patients safe, rather than one that has absorbed generic AI vocabulary and remains defenseless at the dispensing window. The depth should be calibrated to role: a pharmacist owning clinical sign-off needs the full verification and cardinal-rule depth, while a technician needs enough to understand the boundary of their role and to escalate appropriately, but every role needs real, not nominal, understanding of the parts that touch their work.

The Competency Evidence File

The documentation half of the expectation resolves, operationally, into a competency evidence file: the durable record that lets the pharmacy answer the reviewer's question with proof rather than assurance. A leader should design this file to answer, for any individual using AI in a clinical capacity, four questions cleanly. Who is this person and what is their role, so the depth of expected competency is clear. What were they trained on, the specific content, mapped to the failure modes and the verification standard, not just a course title. When did the training occur, and when was it last refreshed, because competency is not a one-time event. And how was the competency confirmed, whether through an assessment, a documented sign-off, a demonstrated verification, some evidence beyond mere attendance that the training actually produced understanding.

That last element separates a real competency file from a weak one. A file that records only that a person attended a session proves attendance, not competence, and a sharp reviewer knows the difference. A file that records that a person completed training and passed an assessment, or demonstrated the verification discipline in a documented way, attests to competence rather than just exposure. This is also where completing a structured, role-grounded AI program like this one becomes directly valuable as evidence: it is not a generic awareness module but a genuine curriculum with assessment, which is precisely the kind of confirmable competency development the expectation calls for, and the completion records are competency evidence in exactly the form a reviewer wants to see. The leader's task is to ensure the evidence file is built as staff are trained, accumulating in real time, rather than reconstructed frantically before a review from sign-in sheets and memory, because a file assembled under deadline pressure is both incomplete and visibly so, and an evidence file that looks back-filled undermines the very credibility it was meant to establish.

Competency as an Ongoing Program, Not an Event

The most common structural error in staff AI competency is to treat it as a one-time event: a training session at deployment, a stack of certificates, and a sense that the box is checked permanently. This fails because the conditions that competency answers to do not stand still. The AI tools change, gaining capabilities and new failure modes. The staff turn over, so a workforce trained eighteen months ago is partly a different workforce today. The risks evolve as the pharmacy extends AI to new uses. And competence itself decays without reinforcement, the verification discipline that was sharp at training loosens under months of routine if nothing maintains it. A one-time competency program is therefore a competency program that is already out of date by the time it is needed, attesting to a state of preparation that no longer fully exists.

Operationally, this means a leader has to build competency as a living program with a maintenance rhythm rather than a launch event. New staff are trained before they use the clinical AI tools, not after, because an untrained user of a capable tool is the dangerous configuration the competency expectation exists to prevent. Existing staff are refreshed on a defined cadence and, importantly, retrained when the tools or the verification standard change materially, because a competency record for the old tool does not attest to competence on the new one. The competency evidence file is maintained continuously, so it always reflects the current workforce against the current tools rather than a snapshot from deployment. This ongoing structure is itself persuasive evidence: a reviewer who sees a competency program with a maintenance cadence, current records, and retraining triggered by tool changes sees a pharmacy that treats competence as the living precondition it is, which is far more convincing than a folder of two-year-old certificates. The leader who builds competency this way can answer the reviewer's opening question, "how do you know this person is competent," with the only answer that holds: not "I trust them," but "here is what they were trained on, when, how we confirmed it, and how we keep it current," which is the difference between a workforce believed to be competent and one demonstrably so.

The Leader's Payoff: Competence That Is Both Safe and Provable

It is worth stepping back to see why this work pays off twice, because a leader allocating scarce time and budget needs to know the competency program is not just a compliance cost. The first payoff is safety, and it is the real one: a genuinely competent workforce catches the AI errors that an underprepared one would pass through to patients, so the competency program is, before it is anything else, a patient-safety investment that prevents the fabricated criterion and the wrong dose from reaching the people they would harm. A leader who built nothing but the workflow and the tools, and skipped the competency, would have a fast, well-instrumented process operated by people who do not reliably catch its errors, which is a faster path to the same harm.

The second payoff is demonstrability, and it is what the user track specifically rewards. The same competency program that makes the workforce genuinely safer also produces the evidence that the workforce is safe, which is what an accreditor, a board, or a partner asks to see. This is the recurring alignment the whole program turns on: the work that makes AI use genuinely safer is the same work that makes it demonstrably safe, so the competency investment is never a compliance distraction from patient care; it is patient care, captured in a form that can be shown. A leader who internalizes this stops experiencing the documentation as overhead bolted onto the training and starts seeing it as the inseparable second face of the same investment: build the competency so your people catch the errors, document it so you can prove they can, and the result is a pharmacy that is not only safe at the point where a person meets a tool but able to demonstrate that safety to anyone with the standing to ask. That dual result, safe and provable, is exactly the position the URAC user track is designed to reward, and the competency program is how a leader reaches it.

Key Takeaways

• Competency is a precondition for safe AI deployment, not a nicety: an incompetent user of a capable tool can be more dangerous than no tool, because confident, fluent output gives a false sense of reliability that an unprepared person cannot defend against, so competency sits on the same footing as the verification standard.
• The competency expectation has two parts that fail differently: competency must be real (staff genuinely understand tools, failure modes, and the verification discipline) and documented (the pharmacy can show who was trained, on what, when, and how it was confirmed). Documentation without real competence is arguably worse than neither, because it manufactures false confidence.
• Pharmacy AI competency covers concrete, role-specific understanding: what AI is and is not, the specific failure modes (invented criterion, wrong renal dose, missed or fabricated interaction, unsupported diagnosis), the operational cardinal rule, the actual workflow verification standard, and documentation and PHI (protected health information) obligations, not generic "AI awareness."
• The competency evidence file should cleanly answer who, what, when, and how-confirmed for each clinical AI user; recording attendance alone proves exposure, not competence, while an assessment or a documented demonstration attests to real understanding.
• Completing a structured, role-grounded AI program is exactly the confirmable competency development the expectation calls for, and its completion records are competency evidence in the form a reviewer wants.
• Competency is an ongoing program with a maintenance rhythm, not a one-time event: tools change, staff turn over, risks evolve, and competence decays, so new staff are trained before they use clinical AI, existing staff are refreshed and retrained when tools or standards change, and the evidence file is kept current.
• Build the evidence file as staff are trained, accumulating in real time, never reconstructed under deadline pressure, because a back-filled file is both incomplete and visibly so, which undermines the credibility it was meant to establish.
• The competency program pays off twice: it makes the workforce genuinely safer (catching AI errors before they reach patients) and demonstrably safe (producing the evidence the URAC user track rewards), which is the program's recurring alignment, the work that makes AI safer is the same work that makes it provable.