- Posted
- Categories
-
- Research Integrity
Help Us Improve Code Sharing in Health Research
We are starting a UKRI-funded project on improving code sharing and want your help!
We are starting a UKRI-funded project on improving code sharing and want your help!
After a bit of a hiatus, the FDAAA TrialsTracker is back with nerdy details about what went wrong and how we fixed it.
Details on two new papers from the Bennett Institute Research Integrity Team.
The Clinical Trial Information System (CTIS) is the European Union’s new registry set to fully replace the existing EU Clinical Trials Registry next year. Nick DeVito decided to take a spin through the registry and record some of his initial thoughts.
It has been a busy month for paper publication at The Bennett Institute. We have written a brief description of the most recent papers below. Please sharewith colleagues and get in touch if you have any relevant observations! Remember you can read all our academic papers related to OpenPrescribing on our research page.
Hospital medicines data: We are frequently contacted at OpenPrescribing about when we are going to make a hospital version. Unlike primary care, access to hospital medicines data is restricted. The BMJ have just published our Bennett Institute article about why we think The NHS deserves better use of hospital medicines data.
This week the British Journal of General Practice published our latest paper on unsafe prescribing of methotrexate. We found that the prevalence of unsafe methotrexate prescribing (10mg tablets) has reduced but remains common, with substantial variation between practices and CCGs. In the paper we also discuss recommendations for better strategies around implementation.
Anyone can view the live data on unsafe methotrexate prescribing at openprescribing.net/measure/methotrexate, to support audit and review in your local organisation. Read the paper here and you can watch Dr. Kevin Barret (twitter @drkbarret) describe here how his practice used OpenPrescribing to identify a breach of their practice policy on methotreaxte prescribing and fix it!
OpenSAFELY is a new secure analytics platform for electronic health records in the NHS, created to deliver urgent results during the global COVID-19 emergency. OpenSAFELY is a collaboration between the Bennett Institute, the EHR group at London School of Hygiene and Tropical Medicine and TPP who produce SystmOne. OpenSAFELY is now successfully delivering analyses across more than 24 million patients’ full pseudonymised primary care NHS records. The first analysis from OpenSAFELY is Factors associated with COVID-19-related hospital death in the linked electronic health records of 17 million adult NHS patients with more answers to important questions expected shortly.
Most people share their end of year roundup during late December when everyone is too full of cake to read. Now you’re back in the saddle, here’s our roundup of everything the Bennett Institute threw out into the world over the previous 12 months!
OpenPrescribing.net went from strength to strength, with over 135,000 unique users last year. We now have over 80 measures of prescribing safety, efficacy and cost-effectiveness and have been working on new types of measures and alerts to identify “outlier” prescribing, such as with zuclopenthixol. In 2019 we also: launched new dashboards and bespoke alerts for every single primary care network (PCN), sustainability and transformation partnership (STP), and NHS region; upgraded the Analyse page to support more organisations with bespoke medicines queries; and developed tools to help solve the problem of Ghost Branded Generics which cost the NHS an extra £11.6million per annum but has massively reduced during 2019.
We have now launched our long-awaited Primary Care Networks (PCNs) dashboard, made possible thanks to the the membership list being published by NHS England last week.
We have PCN prescribing dashboards for every single NHS PCN and their member general practices. (Don’t know your PCN’s name? You can find it on your practice dashboard). This allows anyone to explore NHS prescribing patterns in their PCN and how this compares to others across England — supporting safer, more efficient prescribing.
Our EU TrialsTracker has been assessing compliance with EU trial reporting guidelines for over a year now. Our data has been used by Parliamentary Committees and trial sponsors to improve the reporting of clinical trials registered on the EU Clinical Trials Register (EUCTR).
We previously shared some of the data issues that make complete assessments of trials on the EUCTR difficult. One particularly troubling issue is dealing with trials that never began. Sometimes trials get all the necessary approvals to begin, and therefore are automatically registered on the EUCTR. In rare instances, and for any number of reasons, they then simply never begin; they are cancelled before they enroll a single participant or collect any data. Obviously, these trials have no results to report.

Our newest paper is now out in the BMJ! Here we show huge variation in adoption of warranted changes in prescribing behaviour, using some exciting new openly available change detection methods.
This month we have launched a range of new measures. As always, measures are prioritised on your dashboard by potential for improvement so you can quickly and easily spot where your organisation is an outlier. You can also sort by potential savings or group measures by clinical condition.
Following many requests from users, we have created links from our measures to the corresponding search on the Analyse page. This is particularly useful if you want to see multiple practices or CCGs on one chart, view the results on a map, or see exactly which products are included in the measure.
You will notice this is not yet available for all of our measures (which now number close to 100), due to the complex way some measures are constructed. This currently affects just 12 of our standard measures and 4 of the Low-Priority prescribing measures, and we hope to keep increasing the options in the coming months.
This week we have launched a new browser for the NHS Dictionary of Medicines and Devices, better known as dm+d. dm+d is the standard dictionary for the medicines and devices used across the NHS. At last count there were over 150,000 packs of medicines and devices described. You can read more about the dm+d in [this detailed blog]/blog/2019/08/what-is-the-dm-d-the-nhs-dictionary-of-medicines-and-devices/).
We have been using the dm+d browser internally for a while, and having found it very useful we believe it will come in handy for others too, so we have now made it available publicly for anyone to use. Read more about the tool here.
Yesterday (27 June), NHS England completed its consultation on an additional proposed set of products being considered for a recommendation against regular use in primary care. Therefore, on the site we have now included these new items in the set of low-priority measures. The newly added products include bath and shower emollients, higher cost insulin pen needles and dronedarone. See how your practice or CCG is performing on these measures here, or navigate to the Low Priority measures from your favourite organisation’s dashboard.

One of our recent papers, published in the Journal of Antimicrobial Chemotherapy, highlighted variability in the speed at which different CCGs switched from prescribing trimethoprim to nitrofurantoin (as recommended by PHE for uncomplicated UTI).
It appears that the practices which changed the most were in CCGs that had taken some action to promote the new guidelines, such as a change in formulary. What are the implications? Read more on this in our blog and you can of course see any region’s performance on this measure on OpenPrescribing.
We recently noticed a very dramatic drop in the number of UK clinical trials post-Brexit on the EU Clinical Trials Register (which would be a catastrophe for British science) and wrote to MHRA to get their view. It turns out that the drop is largely due to bad data management: this is worrying, but not the same thing as a drop in the number of UK clinical trials.
We’ve been conducting research on trends in clinical trials globally with one very striking recent finding: examining clinical trials on medicinal products, using the EU Clinical Trials Register (EUCTR), the number of trials with a competent authority decision from the UK has fallen dramatically. As you can see on the graph below: from a long-term trend of around 150 new trials per quarter, after 2016 there is a rapid drop to fewer than 50, and now fewer than 10.
At the Bennett Institute we value openness and transparency as we believe open discussion of ideas and methods are the key ingredients for high quality data analysis. We think it is unhelpful that so much NHS data analysis is outsourced, or done behind closed doors. This blog sets out the steps we take with OpenPrescribing.net to ensure all our analysis and methodology is freely available for inspection, review, and importantly re-use, by anyone. We hope other teams will adopt these approaches so we can all collaborate to optimise analysis, generate insights and improve care for patients in the NHS.
It’s been a busy start to the year for the FDAAA TrialsTracker! As the FDAAA TrialsTracker celebrates its 1 year anniversary, we now show over 1000 trials currently overdue to report under the FDA Amendments Act of 2007 (some background on our tracker and the FDAAA 2007 here). Additionally, we estimate that the FDA could have collected over $2 billion in fines if they were enforcing these requirements. To date, we have no evidence they have issued even a single dollar in fines. But along with the bad comes the good. Over the past year we’ve also heard from many trial sponsors and other organisations and individuals who are using our TrialsTrackers to ensure trials are reported on time and in accordance with the law.
We have introduced a new email alert service, built upon our recently-launched NHS Price Concession calculator, which shows the cost impact of concessions for every single practice, CCG and for all of England combined.
If you sign up for these new alerts, an email will pop into your inbox soon after a new price concession is announced (approximately once per week) to update you on the impact these costs are predicted to have on your practice/CCG. Read more in our blog, or sign up by going to the price concessions tool from your practice or CCG dashboard.
As you know, here at the Bennett Institute we are working on the RetractoBot project to reduce the number of cited retracted papers.
Citations of retracted RCTs are particularly dangerous because such trials provide strong and unbiased evidence of treatment’s safety and efficacy (hierarchy of evidence). Moreover, results of RCTs are often pooled in systematic reviews and meta-analyses, which are used to synthesise the available evidence on a given subject or to justify clinical guidelines.
Nearly all trials of medicinal products conducted in Europe since 2004 are required to post their results directly onto the EU trials register within one year of completion. Since September, our EU TrialsTracker has monitored all clinical trials in the EU to check whether they are compliant. Recently we produced a specific report on data on trial reporting at UK Universities: the House of Commons Science and Technology Committee is currently using this data to alert Universities and monitor their current reporting performance.
Yesterday, we shared an overview of trial reporting performance for all UK Universities on our EU TrialsTracker and FDAAA TrialsTracker. Today, we are pleased to see this data being put to good use.
Following-up on the October 2018 report on clinical trials transparency, the chair of the House of Commons Science and Technology Committee, Norman Lamb MP, has sent letters to 41 universities with trials registered on the EU Clinical Trials Register (EUCTR).
The letter reminds each institution of their trial reporting responsibilities under EU and US law. It gives them their current reporting performance, taking current figures directly from our EU TrialsTracker (BMJ paper here). It asks each university to verify that systems are in place to ensure compliance with these requirements, and opens the possibility that continued poor performance could lead to being called in front of the committee in the near future.
Our EU TrialsTracker has now been live for four months. As of 10 January 2019, we have identified 8,062 trials registered in Europe that are unambiguously due to report results under EU guidelines; a total of 4,323 (53.6%) trials have reported results to the registry. We have also seen some institutions — for example Kings College London — improve their trial reporting performance dramatically and rapidly.
Our BMJ paper showed that non-commercial sponsors (mostly universities) were substantially worse at reporting results to the registry, when compared with drug companies. The House of Commons Science and Technology emphasised this worrying finding in their October 2018 report on clinical trials transparency:
In our third full year of existence we produced even more exciting outputs and continued to grow. We welcomed Lydia Berry, back from maternity leave; Dave Evans, Consultant Programmer, who joined the OpenPrescribing technical team; and Brian MacKenna, an Honorary Research Fellow Pharmacist and member of the NHS England Medicines and Diagnostics Policy Unit. We also welcomed Darren Smyth, a UK and European Patent Attorney - our work so far includes our pregabalin papers (here and here), and he has also contributed to our EUCTR work.
Earlier this week, the Media, Freedom & Information Access Clinic at Yale Law School filed a lawsuit against the heads of the Department of Health and Human Services, the NIH, and the FDA over their interpretation and implementation of trial reporting provisions in the FDA Amendments Act of 2007 (FDAAA 2007). The lawsuit was filed on behalf of Charles Seife, a journalist and NYU professor, and Dr. Peter Lurie, President of the Center for Science in the Public Interest.
At OpenPrescribing we pride ourselves on developing our tools in response to the needs of our users. Last week NHS England announced a new “Do Not Prescribe” list for consultation. Within an hour we made graphs showing every GP practice’s prescribing of these items. You can drill down to CCG level, and then practice level.

We hope that this data will be useful for clinicians and CCG pharmacists to identify where there is most room for improvement, or change. Or, to drive discussion about agreement on the guidance.
At OpenPrescribing we pride ourselves on developing our tools in response to the needs of our users. OpenPrescribing is being increasingly used at national organisations and we have had many requests for an All England dashboard. This month we launched the first version of the All England dashboard and blogged about it here.

At the Bennett Institute we get many emails about how people are using OpenPrescribing and this month we blogged about how OpenPrescribing can be helpful when medicines are in short supply. Epanutin (phenytoin) 30mg/5ml oral suspension is currently subject to a Supply Disruption Alert and it is very important that patients are identified promptly and their prescriptions are modified if appropriate. To read more about the alert and how OpenPrescribing can be helpful please see the blog here. If you have any other interesting usecases please get in touch at feedback@openprescribing.net
The EU TrialsTracker, which provides the results reporting status of every trial on the EU Clinical Trials Registry, launched last week alongside a paper in the BMJ. We already shared some of the media coverage from the launch of the tracker and comments from Norman Lamb MP, Chair of the House of Commons Science and Technology Committee and Fergus Sweeney, Head of Inspections, Human Medicines Pharmacovigilance and Committees at EMA.
Today we wanted to share a statement, released on Friday, from the Institute for Quality and Efficiency in Health Care (IQWiG) in Germany. For those that are not familiar, IQWiG is the German institute responsible for assessing the quality and efficiency of medical interventions, similar to the role NICE plays in the UK. We were pleased to see a government institute calling explicitly for better results reporting from publicly-funded institutions. IQWiG offers frank experiences from their own work in which withheld data compromised their ability to properly assess the evidence for treatments under review. The director of IQWiG, Jürgen Windeler, specifically calls for a system that checks “whether an applicant has completely reported its previously funded project in the EU Register and, if applicable, refuse further funding.”
Earlier this year we launched our FDAAA TrialsTracker, providing a live look at whether individual sponsors and trialists are meeting their responsibility to report the results of clinical trials on ClinicalTrials.gov. Now we have launched the EU TrialsTracker. This new tracker shows the results status for every trial on the EU Clinical Trials Registry (EUCTR). We are launching the tracker alongside a paper in the BMJ providing a detailed overview of our methods, an analysis of the factors associated with non-reporting and a discussion of the data issues that make assessing compliance difficult on EUCTR.
With the launch of our FDAAA TrialsTracker, applicable trials that have failed to report their results on ClinicalTrials.gov are starting to appear. If you go here on our Tracker (the “All Trials” view, and toggle the “due” filter) you can see all the trials that have not yet reported.
Non-reporting of clinical trial results in an ongoing, global public health issue. We are going to start highlighting some of these unreported trials in blog posts to shine a light on what information is being withheld from the public as a result of non-reporting.