Why is this trial due to report?

Now that we’ve launched our FDAAA TrialsTracker, we plan on occasionally taking a closer look at some of the trials that go unreported. Our first blog was about a trial examining 2 drug combinations for managing pain during labor (NCT01846221).

So why do we think this specific trial is due to report? While we go through how we established our criteria and set up our database in detail in our preprint paper on Biorxiv, we wanted to walk you through exactly what fields matter on ClinicalTrials.gov and how we used them to build our tracker. We think we were able to do this well, but are open to feedback and will amend our paper and methods as necessary.

In this blog post we are going to go through a single trial’s registry data, to ascertain whether it is required to report results, and if so, when. You might want to open the trial’s registry entry here so you can follow along for yourself as we go through the data below.

One quick note: All these images were taken directly from the public trial record via ClinicalTrials.gov on February 22, 2018. If they look different for you, it probably means the entry has since been updated. If it does change, you can scroll to the bottom of the “Study Details” tab to see when the trial was last updated; there you can also find a link to a record of all changes.

The regulation concerning reporting results (Final Rule) defines “ACTs” (or Applicable Clinical Trials) and “pACTs” (or probable Applicable Clinical Trials), the types of trials that are required to report results. In order to turn these requirements into code based on the data fields provided by ClinicalTrials.gov, we turned to the Protocol Registration and Results System (PRS) User’s Guide. The PRS is the system through which trial information is entered and managed for eventual display on ClinicalTrials.gov. This guide describes how to use the PRS and provides instructions and information about various PRS functions. We are specifically interested in section 8.1.2 titled “FDAAA 801 Problems.” This section guides users on how the PRS system will identify issues related to the results reporting requirement and outline the criteria used to determine ACTs and pACTs.

As you can see, much of the criteria are the same for both types of trials. The differences are because new data fields and clearer criteria were outlined in the Final Rule. So how do we know which criteria to use for this trial? Let’s start with the final bullet: “Study Start Date.” This field provides a clear distinction regarding which criteria to apply. Trials started on or after January 18, 2017 (the effective date of the Final Rule) will be held to the ACT criteria; anything before will be held to the pACT criteria. Below are the key study dates from ClinicalTrials.gov for this trial:

So we know, based on a “Start Date” in 2014 that this trial should be held to the standard of a pACT. Next, let’s look at completion date. The PRC User’s Guide says that any trial with a “Primary Completion Date” (or if no “Primary Completion Date is provided, a “Study Completion Date” ) on or after January 2008 is a pACT. While ClinicalTrials.gov may want to internally track all trials that meet this criteria of a pACT, we believe that this is not sufficient if your goal is to identify pACTs specifically required to report under the Final Rule. The following table, which appears on page 65121 of the Final Rule, makes clear that the reporting requirement of the Final Rule only applies to those trials that end on or after the effective date of the Final Rule (42 CFR Part 11 is just the another name for the Final Rule).

So one of our criteria was that the study have a “Primary Completion Date” on or after January 18, 2017. Checking the key dates from above again, we can see that this study completed the day after the Final Rule went into effect. Great! So now that we know we are all set on the relevant dates, we can begin to look at the other fields.

• Study type is “Interventional?” Check.
• Study Phase is not “Phase 1?” Check, it is Phase 4.
• Primary Purpose is not “Device Feasibility?” Check, it is “Treatment.”

This leaves two final criteria. First, “Intervention Type:” The PRC User’s Guide tells us that a trial which includes any of these intervention types would be a pACT: “Drug, Device, Biological/Vaccine, Radiation, Genetic, Combination Product, or Diagnostic Test.” The “Interventions” are clearly described as Drugs so we know that criteria has been met.

Lastly, the trial needs to meet one of the following two conditions:

1. Facility Location(s): At least 1 U.S. location or locations not specified, or
2. U.S. FDA IND or IDE: Is “Yes”

We will first look at “facility location”. The documentation says that your trial is a pACT if it has sites in the US, or if no location is specified. We wanted to be as conservative as possible, to ensure we do everything in our power to only identify truly due trials. So, for our tracker, to meet the “facility location” criteria, we only included trials that explicitly identify a U.S. study location. For this trial, it’s clear that this condition is met:

The next criterion is interesting: “U.S. FDA IND or IDE” must be “Yes”. Here, there is an interesting problem. ClinicalTrials.gov hold this data field: but they do not make it publicly available. We think this is very problematic. So, we cannot use this field. That means again that our algorithms and approach are conservative: we do everything we can to ensure that we only identify trials that are due; and we accept that we might miss some due trials, because of data being withheld by ClinicalTrials.gov.

That is the final criteria to identify this trial as a pACT…. Or is it? If we go all the way back up to the ACT criteria (even though this trial is a pACT) we can see a new field that you might think should only appear in an ACT’s record: “FDA Regulated Drug or Device.” Most trials started before January 18, 2017 will not yet have anything in this field, as it did not exist prior to the Final Rule’s effective date. Still, some older trials may have updated their entries since then to include this information. So just to be safe, we do a quick additional check to make sure the FDA-Regulated Drug or Device field is either “Yes” or missing.

Checking For Results

Now, we move on to checking for results, because the “Primary Completion Date” we identified earlier was over a year (a requirement of the law) + 30 days (to account for any administrative delays) ago. How does our software check for results? This is a three stage process. We check to see if it has results reported in the data fields denoting the presence of these results; we check to see if it has “pending” results, that have been submitted but not yet posted online; finally, we check in the relevant data fields to see if there is any indication that the trial has received a special certificate allowing them to delay the reporting of results.

The third tab at the top of our trial’s record is currently titled “No Results Posted”, as you can see in the screenshot below. That tab would contain any information about the results, if they were submitted and undergoing review, and any allowable delays:

As you can see, there has apparently been no results information submitted and no delays granted. And that’s it! Using the public data, and resources provided directly by ClinicalTrials.gov, we are able to see that this trial: (1) meets all the listed criteria of a pACT; (2) is due; and (3) there is no indication that any results have been posted. This is why it is listed as “overdue” in our tracker.

We may do another one of these posts for an ACT in the future. However, any trials meeting the ACT criteria would have started in mid-January 2017, and remember trials have a year plus 30 days to report results, so it may be some time before any ACTs appear in our tracker. In the meantime, we hope the FDAAA TrialsTracker acts as a valuable resource to promote transparency and accountability in clinical research.

We hope you found this walk-through interesting and useful. As always, with all work from the Bennett Institute, we are keen to hear positive or negative feedback that we can act on, feature requests, bug reports, new ideas, and more. It helps if you can be specific and precise. Cheers!