Established to evaluate Digital Health products and services, the Digital Assessment Questionnaire (DAQ) has evolved over time, first setting standards too low, then too high. But after years of development and significant NHS investment, the DAQ became a global digital exemplar - one of the world’s most rigorous digital health assessment models, yet capable of processing an evaluation within a matter of days. A standard that adapted requirements and thresholds based on need in its broad-based sector.
Any standard intended to replace the DAQ must offer a leap in benefits to the NHS, patients and developers. As such, the initiative by NHSX to design a new standard offers a huge opportunity. After working with national bodies worldwide, helping to develop the NICE standards, and recently, as an approved assessor, applying the DAQ for the past 18 months, we are excited by what can be achieved.
As the consultation on the new NHS Digital Health Technology Standard closes, our call to NHSX will be to ensure that the new standard:
- Uses lessons learned: In the development of the DAQ, there were many approaches tested and removed because they were unsuccessful. A close understanding of lessons will ensure that time and resources are not wasted. For example, it is tempting to think that the tougher the standard, the more protection it offers. But set the standard too high and no solutions designed to help patients can make it to the market. Regulation needs to measure the right things and set the bar at an appropriate level, giving consideration to the risks that it is designed to mitigate.
- Does not collate but adapts: Simply collating all appropriate standards into a single assessment process does not work. For example, the Open Web Application Security Project (OWASP) security standard is required in the proposed new NHS Technology Standard, yet the ability to demonstrate compliance with its principles is a hugely complex task, which has, to date, proved impossible. We would encourage the approach taken by the DAQ for the DCB0129 Clinical Risk Management information standard. Here, the assessment process was refined to accommodate the standard’s requirements. This involved developing a granular interpretation of which solutions the standard was or was not applicable to, as well as a refined and more detailed breakdown to evaluate compliance. This step should also seek to remove duplication in checks made by standards.
- Leverages international programmes: If the new NHS Technology Standard is to utilise international best practice, a good starting point would be the EU sponsored project team behind the crafting of a CEN/ISO Technical Standard for Digital Health and Wellbeing solutions. ORCHA has been supporting this work and can recommend this initiative. The CEN/ISO standard is a significant programme that consolidates an array of the most applicable standards in the Digital Health space into one overarching model. This is a great starting point for any national standard and represents a solid foundation for assessment. The new NHS standard does reference the new CEN/ISO Draft Technical Standard, but with a view to adding it to a list of requirements, rather than using it as an opportunity to consolidate the list of standards required for developers and the process to follow.
- Is designed with process in mind: The creation of a standard is just the starting point. A process is needed to implement the standard, which can take between 9 to 12 months to develop. Considering how a standard will be applied throughout development will ease this development stage. Development with process in mind may also help the programme to consider a closed-loop system. For instance, ORCHA works with national bodies across EMEA, providing a review process, national app library and local app libraries which correspond to the national platform. This means the whole process and offer is connected, ensuring a 360° view and management. For example, every time an app is updated in an app store, our system triggers a re-review and the app libraries notify users that the review does not include the latest update.
- Incorporates AI and other emerging technologies:It will be important for any future standard to build suitable assessment elements around technologies such as AI. With experts in the field, ORCHA has already developed an assessment module for AI and would be happy to support the NHS team in this area. The issue of data interoperability is another essential consideration in terms of future technologies.
We call for these areas to be considered, but most importantly, we ask for clarity around the goal for the programme. There are some fantastic principles and concepts in the draft standard, but it would be good to understand the specific step change in outcomes wanted. Only once we know what the problem and target is, will ORCHA and the digital health ecosystem be able to help the NHS to shape the future of Digital Assessment and the New NHS Technology Standard.
ORCHA is the world’s leading health app advisor. It has evaluated more than 6,000 apps to date, runs national accreditation schemes for national bodies, and provides consultancy for schemes including the NICE Evidence Standards Framework. It has assessed apps for the NHS for 5 years, today managing bespoke health app library programmes for NHS organisations in 50% of regions. It has supported NHS Digital in conducting evaluations for the NHS Apps Library based on the DAQ standard for the past 18 months.