The post-authorisation safety study (PASS) is a pharmacovigilance activity often required as a post-marketing commitment to establish a safety profile or address specific safety concerns. An imposed PASS must be submitted in common technical document format.

Comparability of observational studies to clinical trials is limited by a number of factors related to the differences in design and conduct of these studies. These include selection bias, which is harder to control in the observational setting, and typically a relatively higher extent and quality of data collection in the clinical setting. The PASS also places a strong focus on risk without collecting new formal benefit information. These factors present medical writers with some new (and not so new) challenges.

Offsetting the challenges, the PASS creates opportunities to assess the “real world” prescribing of a drug, to compare the real target population with the label population, and, because of the large scale of such trials, to assess safety across multiple sub groups with greater certainty than possible in a clinical trial.

INTRODUCTION

The non-interventional, post-authorisation safety study (NI-PASS) is an increasingly common pharmacovigilance measure, carried out after a medicine has been authorised, to obtain further information on a medicine’s safety. That information may constitute detection of a new, or quantification of an existing safety hazard, or confirmation of a known safety profile.1

While observational studies have a long pedigree, the value of conducting PASS has gained increasing regulatory attention, and the European Medicines Agency has published a template (similar to that for clinical trials) to aid harmonisation of reporting of PASS.2

A PASS is requested for about half of new substances;3 given the scale of these studies (usually much larger patient populations are enrolled than in clinical trials), it has particular value in identifying rare AEs4 and in providing reassurance about established safety knowledge.

When a PASS is requested by regulatory authorities, regulatory submission is expected, in the usual common technical document (CTD) format. This inevitably leads to sponsors wishing to draw comparisons between their pivotal clinical trials and the PASS. This article looks at some of the challenges to comparing data between these very different types of studies, and how the (usually limited), high external validity observational data can complement the (usually thorough), lower external validity clinical data.

DATA AVAILABILITY AND SAFETY ENDPOINTS

The observational setting is limited compared to a clinical trial in terms of the data that can be generated. The principal limitations relate to the fact that interventions other than those that would occur during routine treatment or clinical practice are not permitted in the observational setting. This includes any kind of testing (labs, X-rays, vital signs), or even more intensive questioning or study visits other than those that would be part of routine care. Where the product label suggests additional monitoring, data can and should be collected as indicated. Treatment is solely at the discretion of the investigator, and the freedom of dosing, duration of treatment, stopping and starting, changes of dose, and even changes of treatment can con found interpretation of results within the PASS, while providing useful “real world” information.

While data extraction from medical charts is permissible, monitoring is likely to be less intensive than in a clinical trial, making clarification of missing data challenging. The duration of many trials may also make it difficult or impossible to clarify data at a distance in time. Even basic data such as patient age, sex, and disease history, let alone more critical information such as adverse events (AEs) or causes of death, are far more likely to be absent than in a clinical trial.

The extent of missing data must be considered when making any comparison with prior clinical trials, and the number of missing data points should be quantified wherever possible. Imputation methods must be described in detail, along with any sensitivity analyses. For most soft endpoints (such as biomarkers or quality of life measurements), or those at high risk of reporting bias (such as patient-reported outcomes, including AEs), comparability between a PASS and a clinical trial is often limited, while harder endpoints (such as survival) may be more reliable.

Endpoints requiring measurements or patient questioning are likely to take place less frequently in the observational real world setting than in clinical trials, limiting the value of comparisons. Additionally, the extent and reliability of data collection is usually lower in the PASS. For example, if adverse events are recorded systematically, typical differences to the clinical trial include a longer interval between patient contacts, longer duration of the study (increasing reporting fatigue, higher risk of loss to follow-up), and a focus on particular or established, rather than unexpected, safety issues. Details such as start and stop dates, severity, or countermeasures are more likely to be vague or missing entirely than in a closely monitored clinical trial. These factors conspire to reduce data availability and limit the comparability of data between the observational and clinical trial settings.

If an overt comparison of AE rates between observational and clinical data is included in 2.7.4, remember that regulators are well aware of these systematic effects. A lower AE incidence rate in the PASS than the clinical trial may not be very informative, but a notably higher AE incidence will probably need explaining; this would of course also apply should a higher AE rate for the primary endpoint (if single event or class of event) is observed in the PASS than in clinical data.

For larger PASS, subgroup data may take more prominence than in typical pivotal-trial based submissions. Studies are almost never powered for subgroups, and formal conclusions cannot be drawn, but the number of patients can provide particularly strong reassurance, or evidence for higher adverse event rates in particular groups.

DEMOGRAPHICS

Especially where the screening failure rate is low, the selection of patients and treatments by investigators, which would render a clinical trial useless, is one of the most important pieces of real-world data to emerge from a PASS. Demographics and background characteristics thus take on a much more important role in the PASS submission than the typical clinical submission, which can often be summed-up as “treatment groups were well-balanced”.

This still needs cautious interpretation, as selection bias can change with increasing experience of a product, whether because the product becomes established or more (or less) affordable or because new safety information causes investigators to restrict use. Furthermore, clinical investigators tend to be more experienced and up-to-date than the medical community in general. The type of patients selected and the quality of treatment at a centre of excellence may well be closer to the “real world” than in a clinical trial but still not be representative of the real world.

The real world usually differs from the clinical trial population in a number of ways. Inclusion and exclusion criteria for clinical trials have a tendency to select patients who are exemplary for the target indication but lack severe comorbidities.5 Where the target indication is quantified or graded, the range of severity is likely to be higher, including both sicker and less sick patients, in the PASS than the clinical trials. In terms of comorbidities, again, the selection for clinical trials tends to reduce the proportion of patients with other diseases, while the PASS should have no such restrictions beyond those in the label. This results in a wide variety of confounding factors and the need to consider their impact on the main safety results. Differences between groups in multiple-arm PASS should be discussed and sources of bias that may explain the differences mentioned. Extent of comorbidity and disease severity are worthwhile considering for subgroup definitions, at the latest during drafting of the statistical analysis plan.

EFFICACY OR EFFECTIVENESS

By definition, a PASS is preceded by a Phase III submission, and the Phase III studies typically inform the design of the PASS. Inclusion and exclusion criteria should be minimal and are usually broad enough to capture every patient who receives the treatment at study sites. In some cases, particularly where there are multiple study arms, some effort will be made to recruit similar subjects across arms (reinforced by the product label), or at least to restrict the study to the particular indication. Some outcomes may be recorded that lend themselves well to comparison with the previous Phase III studies, in particular analysis based on spontaneously reportable events that are at low risk of being missed or falsely recorded.

Hard endpoints, such as death, recurrence of the disease under treatment, or hospitalisations, can often be evaluated on the basis of routine data collection, without prejudicing the observational status of the study. If comparable to efficacy endpoints from the clinical trials, these can be detailed in Module 2.7.3, provided it is made clear that, formally, the results arise from safety analyses in the PASS. Because PASS studies are not conducted to investigate efficacy, no efficacy claims should be made, even for endpoints that lend themselves well to this and show similar effectiveness to the clinical setting. Comparisons of effectiveness to clinical efficacy data are subject to the same caveats as all other endpoints, due to the considerable differences in study conduct.

In terms of terminology, effectiveness is preferred over efficacy for results of observational studies. When comparing data directly, other potentially useful terminological distinctions could include study, and patient (for the PASS) versus trial and subject for the clinical trial. These, however, will not excuse an otherwise inadequate distinction between the data sources. Imposing such subtle differences of course generates additional writing and QC effort.

SELECTION BIAS

Many tools used to reduce bias in clinical trials, such as blinding or randomising, are not available in the observational setting. Potential sources of bias need to be considered very carefully, and discussed in detail, in any submission of data derived from a PASS.

The PASS is particularly prone to selection bias and especially to bias in the allocation of patients to treatment groups within the study. One non-interventional study of an anticoagulant6, showed a clear but unexpected difference in all-cause mortality between treatment groups, in favour of the investigational treatment. However, there were important differences between the treatment groups, with patients receiving the investigational treatment less likely to have cancer at baseline, and being younger on average than comparator patients receiving standard treatment.

Prescribing practices for a new medicine change over time, particularly in the first years when experience and knowledge are being gained, and later studies may show different biases than early studies. Even when established imbalances can be traced to particular reasons for clinical decision making, these should be considered anew with each new study.

Selection bias also applies at the point where investigators are considering whether to include patients in the PASS. This can be mitigated by asking investigators to consider for inclusion all (consecutive) patients who are being considered for any of the treatments permitted by the observational plan, reducing the risk that investigator concerns about compliance, likely response to treatment, etc., influence the outcome.

CONCLUSIONS

In contrast to most submissions of clinical data the focus for a PASS is on risks, not benefits. Nevertheless, the PASS creates opportunities for interpretation of data beyond the study focus on one or two safety endpoints, in particular in terms of likely real-world usage of the product, how the treated study population differs from that defined in prior clinical trials, and how the safety profile compares across a range of subgroups.

The biases inherent in the design of a PASS differ considerably from those encountered in clinical trials, raising challenges for direct comparisons between the study types, particularly when the PASS has more than one treatment group, where selection bias may confound comparisons even within the PASS treatment groups. Data quality issues may also complicate interpretation, and direct comparisons to clinical data should be made very cautiously. Nevertheless, submission of a PASS offers an opportunity to create a robust safety profile for real-world use of a drug at a relatively early stage in the product lifecycle.

ACKNOWLEDGEMENTS

I am grateful to the many colleagues at Trilogy, past and present, who have worked with me on PASS submissions, study reports, and narratives, all of whom have shaped the way these documents are prepared. I am also most grateful to the clinicians, statisticians, project managers, and other colleagues on our client teams.

REFERENCES

1. European Medicines Agency. Postauthorisation safety studies (PASS). http://www.ema.europa.eu/ema/index.jsp?curl=pages/regulation/document_listing/document_listing_000377.jsp&mi=WC0b01ac058066e979.
2. Morley G. Non-interventional postauthorisation safety studies (NI-PASS): a different type of report. Med Writ 2014; 23:273-6.
3. Bouvy JC, De Bruin ML, Hoekman J, Stolk P. The costs and effects of postauthorisation safety studies for new active substances. Value in Health 2014; 17:A445.
4. Cohet C, Rosillon D, Willame C et al. Challenges in conducting post-authorisation safety studies (PASS): a vaccine manufacturer’s view. Vaccine 2017; 35:3041-9.
5. Kardos P, Worsley S, Singh D, Román- Rodriguez M, Newby DE, Müllerová H. Randomized controlled trials and real-world observational studies in evaluating cardiovascular safety of inhaled bronchodilator therapy in COPD. Int J Chron Obstruct Pulmon Dis 2016; 11:2885-95.
6. Ageno W, Mantovani LG, Haas S et al. Safety and effectiveness of oral rivaroxaban versus standard anticoagulation for the treatment of symptomatic deep-vein thrombosis (XALIA): an international, prospective, non-interventional study. Lancet Haematol 2016; 3:12-21.

AUTHOR INFORMATION

James Visanji, PhD, MCIL, is a manager at Trilogy Writing and Consulting in Frankfurt, and has recently written several NI-PASS reports and associated submission dossiers.