- Posted
- Categories
-
- Research Integrity
Research Integrity Update - Feb 2024
Details on two new papers from the Bennett Institute Research Integrity Team.
Details on two new papers from the Bennett Institute Research Integrity Team.
The first session in the Bennett Conference discussed past and present work of the Institute on research integrity and policy issues.
A whistlestop tour of the whole history of our group, from OpenPrescribing via TrialsTracker to OpenSAFELY and on through Open Science, policy work, and more
The Clinical Trial Information System (CTIS) is the European Union’s new registry set to fully replace the existing EU Clinical Trials Registry next year. Nick DeVito decided to take a spin through the registry and record some of his initial thoughts.
This repository contains the data cleaning notebook, all necessary datasets, and the code for running the COVID-19 TrialsTracker website at covid19.trialstracker.net. Docker files are included to ensure a consistent environment for reproducibility.
Most people share their end of year roundup during late December when everyone is too full of cake to read. Now you’re back in the saddle, here’s our roundup of everything the Bennett Institute threw out into the world over the previous 12 months! OpenPrescribing OpenPrescribing.net went from strength to strength, with over 135,000 unique users last year. We now have over 80 measures of prescribing safety, efficacy and cost-effectiveness and have been working on new types of measures and alerts to identify “outlier” prescribing, such as with zuclopenthixol.
This is the repository containing everything you need to recreate our analysis published in The Lancet assessing compliance with the Final Rule of The Food and Drug Administration Amendments Act (FDAAA) (2007). The code can also be easily adapted for future analyses of interest using ClinicalTrials.gov data.
Our EU TrialsTracker has been assessing compliance with EU trial reporting guidelines for over a year now. Our data has been used by Parliamentary Committees and trial sponsors to improve the reporting of clinical trials registered on the EU Clinical Trials Register (EUCTR). We previously shared some of the data issues that make complete assessments of trials on the EUCTR difficult. One particularly troubling issue is dealing with trials that never began.
Last week, the Commons Science and Technology Select Committee convened an inquiry following-up their previous clinical trials transparency efforts. The Select Committee has been very active in the trials transparency space over the past two years. Earlier this year they sent letters to every public university and NHS trust in the UK that sponsors clinical trials on the EU register. These letters reminded sponsors of their trial reporting responsibilities and provided them with reporting performance figures based on data from our EU TrialsTracker.
Earlier this year we noticed a disturbing trend among UK trials registered on the EUCTR. The amount of newly approved and registered trials on the EUCTR dropped drastically following the 2016 Brexit vote. The chart from that blog post (from May 2019) is included below. When we reached out to the MHRA to ask about this, they stated that there was actually no decline in UK trials post-Brexit but rather administrative delays that kept trials from appearing on the public-facing EUCTR website.
We recently noticed a very dramatic drop in the number of UK clinical trials post-Brexit on the EU Clinical Trials Register (which would be a catastrophe for British science) and wrote to MHRA to get their view. It turns out that the drop is largely due to bad data management: this is worrying, but not the same thing as a drop in the number of UK clinical trials. The Short Story We’ve been conducting research on trends in clinical trials globally with one very striking recent finding: examining clinical trials on medicinal products, using the EU Clinical Trials Register (EUCTR), the number of trials with a competent authority decision from the UK has fallen dramatically.
It’s been a busy start to the year for the FDAAA TrialsTracker! As the FDAAA TrialsTracker celebrates its 1 year anniversary, we now show over 1000 trials currently overdue to report under the FDA Amendments Act of 2007 (some background on our tracker and the FDAAA 2007 here). Additionally, we estimate that the FDA could have collected over $2 billion in fines if they were enforcing these requirements. To date, we have no evidence they have issued even a single dollar in fines.
As you know, here at the Bennett Institute we are working on the RetractoBot project to reduce the number of cited retracted papers. Citations of retracted RCTs are particularly dangerous because such trials provide strong and unbiased evidence of treatment’s safety and efficacy (hierarchy of evidence). Moreover, results of RCTs are often pooled in systematic reviews and meta-analyses, which are used to synthesise the available evidence on a given subject or to justify clinical guidelines.
Nearly all trials of medicinal products conducted in Europe since 2004 are required to post their results directly onto the EU trials register within one year of completion. Since September, our EU TrialsTracker has monitored all clinical trials in the EU to check whether they are compliant. Recently we produced a specific report on data on trial reporting at UK Universities: the House of Commons Science and Technology Committee is currently using this data to alert Universities and monitor their current reporting performance.
Our EU TrialsTracker has been live for 4 months providing data on who is and isn’t meeting EU trial reporting guidelines. The tracker reached a major milestone last week when our data was used by the House of Commons Science and Technology Committee to inform UK Universities of their current reporting performance and remind them of their ethical and legal obligations to report their sponsored trials. As some of our regular users may have spotted, we’ve recently make some changes to the EU TrialsTracker.
Yesterday, we shared an overview of trial reporting performance for all UK Universities on our EU TrialsTracker and FDAAA TrialsTracker. Today, we are pleased to see this data being put to good use. Following-up on the October 2018 report on clinical trials transparency, the chair of the House of Commons Science and Technology Committee, Norman Lamb MP, has sent letters to 41 universities with trials registered on the EU Clinical Trials Register (EUCTR).
Our EU TrialsTracker has now been live for four months. As of 10 January 2019, we have identified 8,062 trials registered in Europe that are unambiguously due to report results under EU guidelines; a total of 4,323 (53.6%) trials have reported results to the registry. We have also seen some institutions — for example Kings College London — improve their trial reporting performance dramatically and rapidly. Our BMJ paper showed that non-commercial sponsors (mostly universities) were substantially worse at reporting results to the registry, when compared with drug companies.
In our third full year of existence we produced even more exciting outputs and continued to grow. We welcomed Lydia Berry, back from maternity leave; Dave Evans, Consultant Programmer, who joined the OpenPrescribing technical team; and Brian MacKenna, an Honorary Research Fellow Pharmacist and member of the NHS England Medicines and Diagnostics Policy Unit. We also welcomed Darren Smyth, a UK and European Patent Attorney - our work so far includes our pregabalin papers (here and here), and he has also contributed to our EUCTR work.
Earlier this week, the Media, Freedom & Information Access Clinic at Yale Law School filed a lawsuit against the heads of the Department of Health and Human Services, the NIH, and the FDA over their interpretation and implementation of trial reporting provisions in the FDA Amendments Act of 2007 (FDAAA 2007). The lawsuit was filed on behalf of Charles Seife, a journalist and NYU professor, and Dr. Peter Lurie, President of the Center for Science in the Public Interest.
The EU TrialsTracker, which provides the results reporting status of every trial on the EU Clinical Trials Registry, launched last week alongside a paper in the BMJ. We already shared some of the media coverage from the launch of the tracker and comments from Norman Lamb MP, Chair of the House of Commons Science and Technology Committee and Fergus Sweeney, Head of Inspections, Human Medicines Pharmacovigilance and Committees at EMA.
Earlier this year we launched our FDAAA TrialsTracker, providing a live look at whether individual sponsors and trialists are meeting their responsibility to report the results of clinical trials on ClinicalTrials.gov. Now we have launched the EU TrialsTracker. This new tracker shows the results status for every trial on the EU Clinical Trials Registry (EUCTR). We are launching the tracker alongside a paper in the BMJ providing a detailed overview of our methods, an analysis of the factors associated with non-reporting and a discussion of the data issues that make assessing compliance difficult on EUCTR.
Our FDAAA TrialsTracker has been updating each working day since February showing which trials on ClinicalTrials.gov have reported, in compliance with the FDA Amendments Act of 2007. Our newly launched EU TrialsTracker and paper in the BMJ show exactly which trials on the EUCTR have reported according to EU guidelines. This week we’re delighted to tell you that our TrialsTracker work has been awarded a Cochrane-REWARD Prize for Reducing Research Waste.
This repository contains all the data extraction and front-end code for EU Trials Tracker.
This article in Quartz, by Jane C Hu, reports on our BMJ paper assessing compliance with requirement to report results on the EU Clinical Trials Register and the associated EU TrialsTracker.
On February 19, 2018 we launched the FDAAA TrialsTracker, a tool that automatically monitors whether clinical trials are reporting their results to ClinicalTrials.gov in accordance with US law. We bring public accountability by ranking sponsors, and help researchers to comply with the law by showing them which of their trials are overdue. We also calculate the total fines the FDA could have imposed to date on non-compliers. This week marks two important milestones for the FDAAA TrialsTracker.
In February, we posted our first Unreported Trial of the Week here in the Bennett Institute blog highlighting a trial from Columbia University. Since then, that trial has submitted results to ClinicalTrials.gov and Unreported Trial of the Week has moved to the BMJ! Each week we profile a new unreported clinical trial that has not reported results in accordance with their legal requirements. You can see all four trials we have profiled thus far as well as the introductory post at the BMJ.
Our FDAAA TrialsTracker provides a public list of all trials required to report under the FDA Amendments Act of 2007 (FDAAA). We update every weekday with live data showing who has, and has not, reported their results in accordance with the law. You can read our full methods in our preprint paper. One unreported trial last week stood out: this trial was sponsored by the FDA, the agency in charge of enforcing the reporting requirements of the FDAAA.
Here at the Bennett Institute we are a truly multidisciplinary team: clinicians, academics and software engineers, working together to produce tools, as well as papers. This is glorious fun, and productive. From our time working together as a team we now have software engineers who know about clinical trials, prescribing data, and more; and we have researchers and clinicians who know a bit about building websites. This may or may not thrill you, but I will share it: our academics write code that runs live on our websites!
When we launched our FDAAA TrialsTracker we wanted to produce a tool that would improve clinical trial reporting, rather than another repetitive academic journal paper that simply documents the extent of the problem. This reflects our ethos in the Bennett Institute: clinicians, academics and software engineers, working together to produce tools, as well as papers. Two weeks after launch we have had extensive media coverage, and a lot of great user feedback.
“If you’re a huge company trying to develop a new drug or treatment, the temptation to hide ‘bad’ results might be a strong one. Until now, you would be able to hide those results with few consequences.” In this Wired article, Abigail Beall, discusses the Alltrials campaign and the FDAAA TrialsTracker
When you produce online tools from data, you often get useful feedback that helps you improve the outputs. (Send us feedback any time!). Additionally, when you use data, you learn about interesting glitches in it, some of which can be entirely undocumented. Here we share one example of helpful feedback, and how we used it to improve our tool. First some background. Trial reporting is a huge problem in medicine: the results of clinical trials are routinely withheld from doctors, researchers, and patients.
Now that we’ve launched our FDAAA TrialsTracker, we plan on occasionally taking a closer look at some of the trials that go unreported. Our first blog was about a trial examining 2 drug combinations for managing pain during labor (NCT01846221). So why do we think this specific trial is due to report? While we go through how we established our criteria and set up our database in detail in our preprint paper on Biorxiv, we wanted to walk you through exactly what fields matter on ClinicalTrials.
“Institutions that fail to report the results of their drug and medical trials will be named on a new website.” In this BBC article, Chris Foxx, highlights the FDAAA TrialsTracker. Quoting Bennett Institute Director, Ben Goldacre, “I’m not interested in naming and shaming people in order to criticise them. This project is being done to nudge institutions to prioritise trial reporting.”
This week, we launched our FDAAA TrialsTracker which gives you a live look at whether individual sponsors and trialists are meeting their responsibility to report the results of clinical trials on ClinicalTrials.gov. A lot of work went into the tracker and making sure we got it right. You can read all about our methods, in detail, here but the short version goes like this… Certain trials registered on ClinicalTrials.gov, the US trial registry run by the National Institutes of Health (NIH), are required to report their results.
In our second full year of existence we produced even more exciting outputs and continued to grow. You can see the team on our website: Emma-Jane has been our project manager since the Spring; we have two new part-time coders working for us; and researchers Alex, Nick, Daniel and Karolina all started this year. We’ve delivered a range of great new features on OpenPrescribing, got great progress on numerous pre-launch projects, and have a nice portfolio of papers submitted.
We were at Oxford University’s Curiosity Carnival, an event featuring researchers from all disciplines enthusiastically explaining their work to members of the public in an enormous variety of forms, all around the city. We entered the Great Research Bake-Off and took on the challenge of representing some of the key issues around research integrity through the medium of cake. We displayed an array of fairy cakes, each representing clinical trials. To produce a good cake, or trial, a good methodology is fundamental.
In our first full year of existence we’ve grown fast, recruiting lots of new staff. You can see us all on our new website: Lydia our project manager started in the Spring, as did Seb, our new coder; Lisa, our community engagement manager, started over the summer; Helen, our new researcher, started in December, with two more researchers starting in January. For our OpenPrescribing work we’ve been joined by Richard as our pharmaceutical advisor.
This repository contains all the analysis and front-end code for trialstracker.ebmdatalab.net which is a simple application that tracks major trial sponsors with unreported trials on ClinicalTrials.gov.