The disproportionality run finished overnight, and at the top of the morning queue sits a single red row: for a marketed biologic, the preferred term immune thrombocytopenia has crossed the threshold, with a proportional reporting ratio of 4.2 and an Empirical Bayes Geometric Mean lower bound above the alert line, computed across EudraVigilance for the last quarter. The safety scientist pastes the statistics, the case count, and the drug-event pair into the enterprise model and asks for the three documents the signal-management process requires: the medical assessment, the signal-validation memo, and the signal-management plan. Ninety seconds later all three appear, fluent and structured, and the medical assessment opens with a sentence stating that the data confirm a causal association between the product and immune thrombocytopenia. That word, confirm, is the entire danger of this workflow. A disproportionality signal is a statistical alert that a human validates, not a confirmed risk, and a model that writes confirm has skipped the single most important act in pharmacovigilance. This lesson shows how AI drafts the documentation around a PRR, ROR, or EBGM signal from VigiBase, FAERS, or EudraVigilance, and why the validation judgment, the decision that a statistical flag is or is not a real signal, stays human.

What a Disproportionality Signal Actually Is, and What It Is Not

Disproportionality analysis asks a narrow statistical question: for a given drug-event pair, is the event reported more often with this drug than you would expect if there were no association, given the background reporting rate of that event across the whole database. The Proportional Reporting Ratio and the Reporting Odds Ratio are the simpler frequentist measures built from the two-by-two contingency table of the drug, the event, and everything else in the spontaneous reporting system. The Empirical Bayes Geometric Mean, produced by the Multi-item Gamma Poisson Shrinker, is a Bayesian measure that shrinks unstable estimates from small cell counts toward the null, which is why it is preferred for the rare-event corners where a PRR of infinity can be produced by two reports. All three are computed over a spontaneous reporting database, principally the WHO global database VigiBase, the FDA's FAERS, or the EMA's EudraVigilance.

What every one of these measures shares is the property that makes this lesson necessary: a disproportionality statistic is a measure of reporting, not of risk. It is elevated when an event is reported disproportionately often, and reporting is shaped by forces that have nothing to do with causality, notably notoriety bias, where publicity about a suspected reaction drives a wave of reports, and the Weber effect, where reporting peaks in the first years after launch. A high PRR can mean a real adverse drug reaction, or it can mean a journalist wrote a story, a competitor ran a campaign, or the indication itself causes the event. The statistic cannot distinguish these. It is a smoke detector that does not know the difference between a fire and burnt toast, and the validation step is the human walking into the kitchen to look.

This is why the language of the documents matters at the level of individual verbs. A statistical alert has crossed a threshold; that is a fact about the arithmetic. Whether the alert represents a genuine new safety concern is a separate determination, made by a qualified human weighing the clinical features of the cases, the biological plausibility, the temporal relationship, the dechallenge and rechallenge information, the confounding by indication and comedication, and the existing knowledge in the label and the literature. A model that compresses these two things into the word confirm has erased the validation step in a single token, and the safety scientist who lets it stand has signed an attestation the data do not support.

Three Documents, Three Different Risk Objects

The signal-management process produces three artifacts around a flagged drug-event pair, and they fail under AI in three distinct ways. The medical assessment is the clinical evaluation of the signal: what the cases look like, whether the association is plausible, what the disproportionality statistics show, and what the assessor concludes about whether this is a validated signal. It is the document most likely to inherit the model's overclaiming, because the model's training corpus is full of confident causal language and short on the hedged, evidence-bounded phrasing that a real assessment uses. The assessor must rewrite every causal verb the model produces into the calibrated register the evidence supports, replacing confirms with is consistent with or warrants further evaluation wherever the data have not earned the stronger claim.

The signal-validation memo is the narrower document that records the decision: was the statistical signal validated as a signal requiring further action, or refuted, and on what basis. Its danger is the opposite of the medical assessment's. Where the assessment overclaims, the validation memo can underdocument, because the model, asked to record a decision, will produce a clean conclusion sentence without the audit-grade chain of reasoning that connects the contingency-table counts, the case review, and the existing label to the validation outcome. A validation memo that states a signal was validated without showing the work is undefendable at inspection, and the model's tendency to write the conclusion and skip the derivation is exactly the failure to guard against.

The signal-management plan is the forward-looking document: given a validated signal, what actions follow, additional data collection, a targeted analysis, a label change proposal, a risk-minimization measure, or continued routine surveillance, and on what timeline with what owner. Its danger is that the model invents plausible actions and timelines that are not grounded in the company's actual procedures, producing a memo that proposes a fifteen-day expedited label submission when the situation calls for routine monitoring, or vice versa. The plan reads authoritative because action language is well represented in the corpus, and the assessor must check every proposed action and clock against the actual signal-management standard operating procedure rather than against the model's general sense of what one does about a safety signal.

The Statistics Are Inputs the Model Transcribes, Not Conclusions It Computes

A foundational architectural point governs every number in these three documents: the model does not compute the disproportionality statistics, and it must never appear to. The PRR, ROR, and EBGM values come from a validated signal-detection system or a statistical environment that ran the contingency tables over the reporting database, and they enter the documentation as transcribed values from a named, dated, reproducible analysis. A model asked to state the PRR for a drug-event pair, without the computed value loaded, will produce a plausible number, and a fabricated PRR of 3.8 carries exactly the false authority of a real one. The discipline is identical to the periodic-report numbers rule: every statistic in the assessment traces to a cell in the loaded signal-detection output, and any number that cannot be traced is wrong until proven right.

There is a subtler version of this error that the model is especially prone to: the misinterpretation of a correctly transcribed statistic. The model may carry the right EBGM lower bound into the assessment and then characterize a value just above the alert threshold as a strong signal, when the methodology treats it as a weak one barely clearing the line, or it may describe a PRR built on three cases as robust when the small cell count makes it fragile. These are not transcription errors; they are interpretation errors, and they are more dangerous because the number is correct, so a reviewer checking only the figures will pass them. The assessor must verify not only that each statistic matches the source but that the qualitative characterization of its strength matches the methodology that produced it.

The Database Shapes the Signal, and the Model Does Not Know Which One It Read

The three reporting databases are not interchangeable, and a competent assessment names the source and accounts for its idiosyncrasies, which the model will not do unless forced. VigiBase, the WHO global database maintained in collaboration with the Uppsala Monitoring Centre, aggregates reports from more than a hundred national programs, which gives it breadth and also heterogeneity in reporting quality and duplication across borders. FAERS, the FDA's system, is shaped by US reporting patterns, mandatory manufacturer reporting, and a strong notoriety effect driven by direct-to-consumer dynamics. EudraVigilance, the EMA's system, carries the structure of EU expedited and literature reporting and its own duplication and stratification considerations. A disproportionality value is only interpretable against the database that produced it, and a PRR computed in FAERS does not transfer to a statement about the EU experience.

The model, handed a number and a drug-event pair, will write a generic assessment that does not name the database or its biases, because in its training corpus signals are discussed in the abstract far more often than they are tied to the quirks of a specific system. The assessor must insert the source-aware reasoning the model omits: which database produced this value, what its known biases are for this kind of event, whether the case count is inflated by duplicates across reporting streams, and whether the elevated reporting plausibly reflects notoriety or the Weber effect given the product's launch timing. This is the confounding analysis that separates a real assessment from a restated statistic, and it is precisely the part the model cannot supply because it does not know which database it is looking at or what forces shaped the rows it never saw.

Why the Validation Judgment Stays Human

Signal validation is the determination of whether there is sufficient evidence demonstrating the existence of a new potentially causal association, or a new aspect of a known association, to justify further action. It is the hinge of the entire pharmacovigilance system, the point at which a statistical flag becomes, or does not become, a thing the company acts on, and it is irreducibly a human judgment for the same reason the benefit-risk integration is. It requires weighing evidence of different kinds that no statistic combines: the disproportionality value, yes, but also the clinical coherence of the individual cases, the strength of the temporal association, the presence or absence of dechallenge and rechallenge, the plausibility of an alternative explanation in the underlying disease or a comedication, and the prior probability set by what is already known about the drug class.

The model performs this weighing fluently and wrongly in a characteristic direction. Asked to assess a flagged pair, it tends toward validation, because the framing of the task, here is a signal, assess it, primes the conclusion that there is something to assess, and because confident causal narratives dominate its training data. The result is a model that validates marginal signals it should refute and writes robust clinical detail for cases it has not actually weighed, manufacturing the appearance of a careful assessment from a statistical alert and general medical knowledge. This is the inverse of a system that should be skeptical by default, treating each flag as presumptively burnt toast until the case review earns the upgrade to fire.

So the workflow assigns the model a bounded role. It drafts the structure of all three documents, transcribes the loaded statistics into the correct fields, assembles the case-level facts from the loaded line listing, and lays out the considerations a validation must address, the plausibility, the temporality, the confounders, the existing knowledge, as a structured pre-read. It does not reach the validation conclusion, and where it produces a draft conclusion, that conclusion is treated as a hypothesis for the human to test against the evidence, never as a finding to retain. The safety scientist performs the validation as a human act, calibrates every causal verb to the evidence, and owns the determination under their name. The model can describe a signal; only a human can validate one.

Calibrated Language as the Control That Carries the Whole Workflow

The defining skill in this workflow is verb discipline, and it is worth making explicit because it is the cheapest and most reliable control available. Pharmacovigilance writing uses a graded vocabulary of association that maps to a graded strength of evidence: an association may be reported, observed, suggested, consistent with, supportive of, or, only rarely and only with strong evidence, established or confirmed. The model writes across this entire range with the same even confidence, reaching for the strong end because the strong end is more common in its corpus, and the assessor's core task is to push every claim back down the ladder to the rung the evidence actually supports. A disproportionality statistic alone supports observed and suggested; it does not, by itself, support confirmed.

This verb discipline is not stylistic fussiness; it is the substance of the regulatory record. A medical assessment that says the data confirm a causal association commits the company to a position its own evidence does not support, and that commitment propagates into the validation memo, the management plan, and potentially a label change or a regulatory communication, each inheriting a certainty the smoke detector never had. The reviewer at an EMA or FDA inspection reads these documents as a chain of claims that must each match their evidence, exactly the posture the submission reviewer takes toward a Clinical Overview, and a single overclaiming verb at the head of the chain undermines every document downstream of it. The calibrated verb is therefore the load-bearing control, and capturing the run, the model, the loaded statistics, the prompt, and the human-verified final text, is what lets an inspector confirm that the calibration was a human act and not a model accident.

Documenting the Run So the Validation Is Attributable

Like every AI-assisted regulated artifact, the three signal documents inherit the obligation to be reconstructable, and the stakes here are specific. If a validated signal later proves to have been a notoriety-driven artifact, or a refuted signal later proves to have been real, the question at review is whether the validation was a sound human judgment on the available evidence or an unreviewed model conclusion that someone signed. Only the run record answers that. The defensible record captures, per document, the model and version, the system prompt, the exact signal-detection output and case line listing loaded, the prompt, the temperature, the timestamp, and the final human-verified text, linked to the controlled version in the safety document system, alongside the human assessor's recorded rationale for the validation decision itself.

The point that ties this lesson to the rest of the pharmacovigilance chapter is that the validation rationale is the irreplaceable human contribution, and the documentation must make it visible as such. A signal-validation memo whose reasoning was generated by the model and merely accepted by the human is automated negligence with good formatting, the same failure named in the literature-surveillance lesson, now operating at the most consequential decision point in the system. The model accelerates the production of the documents around the judgment; it cannot make the judgment, and the record must show, claim by claim and verb by verb, that a named human did. That is what converts a statistical alert and three fluent drafts into a defensible signal-management decision.

Key Takeaways

• A disproportionality signal, a PRR, ROR, or EBGM alert from VigiBase, FAERS, or EudraVigilance, is a measure of reporting, not of risk. It is a statistical flag a human validates, not a confirmed association, and it is shaped by notoriety bias and the Weber effect, so a high value can mean a real reaction or a journalist's story. The validation step is the human walking into the kitchen to distinguish fire from burnt toast.
• The three artifacts fail in three different directions. The medical assessment overclaims causality, the validation memo underdocuments the reasoning chain, and the signal-management plan invents actions and clocks ungrounded in the company SOP; each requires a different correction by the assessor.
• The model transcribes the statistics; it never computes them, and a correct statistic can still be misinterpreted. Every PRR, ROR, and EBGM traces to a cell in the loaded signal-detection output, and the assessor verifies not only that the number matches the source but that its characterized strength matches the methodology.
• Signal validation stays human because the model is biased toward validation. The task framing and the corpus prime confident causal narratives, so the model validates marginal signals it should refute; the system must be skeptical by default, treating each flag as presumptively burnt toast until the case review earns the upgrade.
• Calibrated verbs are the load-bearing control. Pharmacovigilance language grades association from observed to confirmed, the model reaches for the strong end, and the assessor pushes every claim down to the rung the evidence supports; capture the run so an inspector can confirm the calibration and the validation rationale were human acts.