Testing and iterative improvement of the quality assessment of health applications using CEN ISO/TS 82304-2: pilot study of inter-rater reliability

Recently the journal JMIR published Testing and Iterative Improvement of the CEN ISO/TS 82304-2 Health App Quality Assessment: Pilot Interrater Reliability Study . A study that aims to develop and refine the assessment manual and developer guide of CEN ISO/TS 82304-2 through testing in five countries. The Digital Assets for Citizens department of the TIC Salut Social Foundation actively participated.

The study is motivated by the growing use of health apps and the resulting concerns about their safety, effectiveness and data privacy. This fact has led to the development of various quality assessment frameworks. However, the absence of an internationally recognised standard has led to multiple and repetitive assessments for developers who want to access different markets.

Standardisation study of the European standard

The evaluation of health applications based on the CEN ISO/TS 82304-2 standard aims to resolve the absence of a common international standard. A framework accepted globally and applied by organisations in different countries.

With this perspective, 24 applications were evaluated in three different phases or rounds. Each application was analysed by two different organisations and inter-rater reliability was measured. In addition, information was collected on the time required for the evaluation and the clarity of the guides.

The results showed an improvement in interrater reliability between the first and second rounds, remaining stable in the third, with 80% of the questions achieving moderate or higher agreement.

In the third round, developers needed an average of 8 hours and 59 minutes to prepare the information, while raters spent an average of 8 hours and 46 minutes reviewing the information provided. The clarity of the manual and guide was well rated, with scores close to “completely clear”.

This study is the first attempt to analyse the consistency of health app evaluation at an international level. The results suggest that the CEN ISO/TS 82304-2 standard has the potential to establish a standardised evaluation framework that reduces duplication of work, facilitates access to smaller markets, and increases trust in quality health apps.

• L'àrea d'Actius Digitals per la ciutadania de la Fundació TIC Salut Social hi ha participat de manera activa

  • 0 Kb

    
  •