Since February 2023, two UK clinics gradually rolled out three modules — hypertension (HTN), type 2 diabetes (DM2), hypercholesterolemia (HCL) — with a clinical usage that was systematically monitored (by Metadvice) for diagnosis and therapies so that the recommendations can now be linked with observable outcomes (stored by Electronic Medical Record systems, EMR independent from Metadvice). Can Metadvice AI measurably improve the quality of care?
By selecting all the therapeutic avenues from the EMRs that coincide with specific disease modules, two distinct sets of clinical decisions can be evaluated, made either in the presence or in absence of Metadvice AI to determine to which extent our product claims (improved guideline adherence, better clinical outcomes) can be substantiated. A percentage gain is measured to assess the extent of the improvement and a p-value corresponding to complement of the level of confidence for which it is possible to reject a null hypothesis supposing that both sets of decisions cannot be distinguished… and therefore endorse the alternative claiming measurable improvements.
Table 1 displays a 5% absolute gain in adherence (from 45 to 50%) for therapeutic decisions covering hypercholesterolemia, for a statistically significant p-value of 0.004 that is well below the 0.05 threshold. The 25% absolute gain (from 32% to 57%) is even larger for type 2 diabetes leading to a negligible p-value that again supports the claim of an improved adherence. Because the 4% absolute gain (from 29% to 33%) is smaller and because of insufficient usage, it is not yet possible with p=0.177 to reject a no-benefit hypothesis for hypertension adherence with sufficient confidence. In terms of achieving blood pressure outcome targets, however, we notice a 8% gain (from 53% to 61%) with a statistically significant p=0.024.
| HCL | HTN | DM2 | |
| Adherence | |||
| p-value | 0.004 | 0.177 | 0.000 |
| gain | 5% | 4% | 25% |
| Outcomes | |||
| p-value | 0.098 | 0.024 | 0.144 |
| gain | 4% | 8% | 13% |
Table 1. Hypothesis testing showing the minimum p-value and the corresponding gain for a fraction of the population crystallised by Metadvice AI for three clinical modules that have been deployed in two clinics over the period Feb 2023 – Aug 2025.
These results are truly encouraging, this even if the number of cases remain too small to reject the desired null hypothesis for all the condition covered by the product — including more recent modules for chronic kidney disease (CKD), heart failure (HF), atrial fibrillation (AF) and polycystic ovary syndrome (PCOS). With the clinical usage picking up, we however look forward to revisit this analysis that is part of our annual Post Market Clinical Follow-up documentation under the European Medical Device Regulation (EU-MDR).