Our EU TrialsTracker has been live for 4 months providing data on who is and isn’t meeting EU trial reporting guidelines. The tracker reached a major milestone last week when our data was used by the House of Commons Science and Technology Committee to inform UK Universities of their current reporting performance and remind them of their ethical and legal obligations to report their sponsored trials.
As some of our regular users may have spotted, we’ve recently make some changes to the EU TrialsTracker. We wanted to take some time to describe these updates and let you know why we made them.
Date issues
First is a small change under the hood. A sponsor recently told us that for two of their trials, dates were appearing on the tracker that did not match the EUCTR trial record. After investigating, we found a bug that caused completion dates that once existed on the EUCTR to persist in our data even after they were removed. This was an incredibly rare occurrence: affecting only 4 trials out of over 8000 in our entire “results due” cohort. Still, we were happy to make the correction to our code and grateful for the feedback from the sponsor. All of the code and data for the EU TrialsTracker (and all other Bennett Institute projects) is available openly and freely on our GitHub for review and reuse complete with a record of every update and change made to our data and code. If you ever spot an issue or have any questions, please let us know: bennett@phc.ox.ac.uk.
Changes to labelling of trials
Next, some more noticable changes. We’ve updated some of the labels applied to trials throughout the tracker. One important note, none of these changes had any impact on the headline reporting statistics for “due” trials either overall or for individual sponsors.
Terminated Trials: We’ve added a tag throughout all categories to indicate when a trial is terminated, or in EUCTR parlance “Prematurely Ended,” in all locations. The 2012 EU Guidelines that inform the tracker unambiguously require trials terminated for any reason to report results (see section 4.3 “Timing” of the above link). This makes sense. Trials that collect any data should be made to share that data even if the trial doesn’t reach completion.
Terminated trials - a limitation: A number of sponsors told us there was an issue here: certain trials listed as “Prematurely Ended” never actually happened. They were registered but never enrolled any patients or collected any data, so they had nothing to report. The EUCTR has no way to identify a trial that was registered but never began. Some older trials had ways around this, but newer trials have no way to officially indicate that the trial never took place, or report this in lieu of results. Contrast this to ClinicalTrials.gov, which has a separate trial status for Terminated (ended early but enrolled any patients) and “Withdrawn” (cancelled without enrolling any patients). We will still evaluate terminated trials as due to report, but those “Terminated” for all locations will have a tag until the EUCTR system is improved to better handle this issue.
Exempt Trials: Some trials in our “Not Due” category would never become due because they are identified as a Phase 1 trial that is not part of a Paediatric Investigation Plan. These types of trials are not covered by the reporting guideline and are typically not made public on the EUCTR. Yet some trials meeting this description are available. There may be some reason why these trials are made public and required to report, however to remain conservative, we now identify these trials as “Exempt” and they remain in in the “Not Due” category.
Inconsistent Data: We’ve made a few changes to the “Inconsistent Data” category. This tab includes trials that we cannot assess because of data issues originating from the sponsor, national authorities, or the EUCTR itself.
Our first change is a small one. When a trial with inconsistent data has results, we now note this in the label for all trials. Previously, this information was only presented for some trials.
The second change is a bit more complicated:
Trials with a non-EU/EEA protocol are now identified separately from those with no trial status for an EU/EEA-based protocol. Previously, trials with non-EU/EEA protocols were either included in the “No Trial Status on Register” tag, or counted as “Ongoing”.
While certain trials are required to include information about the non-EU/EEA parts of their study, the EMA has limited ability to collect and present data on non-EU/EEA research. This means these protocols have no ability to submit trial status or completion dates and therefore we cannot confidently assert that a trial has completed in all locations. This new tag helps distinguish these trials, with issues impossible to address due to the design of the EUCTR, from trials with correctable issues.
These trials present a unique issue for the EU TrialsTracker. One of the reasons we created the EU TrialsTracker is so that sponsors can use it to improve their trial reporting and data quality. Having inconsistent data on the tracker that is impossible to address is frustrating for both us and the sponsors who want to improve. Yet we are also committed to presenting data on all trials on the EUCTR. There is no question these trials are required to report, we just can’t say confidently when they should report.
One positive is that this population of trials actually reports results at an incredibly high rate: According to data from our most recent scrape of the EUCTR (10 Jan 2019), 84% (1006 out of 1203) of all trials with a non-EU/EEA protocol have results available.. If you restrict that sample to only trials that would otherwise meet our “results due” criteria, it is 97% (56 out of 58) compared to just 56% for all other “due” trials. There may be other data issues with some of these trials that would keep them in the “Inconsistent Data” category, but this is nonetheless encouraging.
To address this problem we considered simply assessing trials as if they had no non-EU/EEA protocol, or even dropping these trials from the tracker completely. For now, neither of those options seemed like good ones. However, as always, we are open to feedback! Please let us know (bennett@phc.ox.ac.uk) if you have any thoughts, comments, or ideas on the best way to present and catalogue this data. Please keep in mind, we are a tiny team with minimal resourcing for this work. Major changes to the Tracker cost us time and money and won’t happen right away.
If you know of any funding opportunities in the trials transparency and research integrity spaces, please do let us know! You can read our appeal for funding here.