Feasibility
The current management app functions as an intelligent, epilepsy-specific, medical record which produces a summary with a management plan sent to an experienced neurologist. For untreated patients, it suggests a management plan when an experienced doctor is not available. Our results showed that the system was feasible, sustainable – it continues to be used after 18 months, and in particular timely – 85% of referrals were completed within 24 h.
Quality
According to the US Institute of Medicine’s recommendation, the quality of the app can be assessed in six domains: safety, timeliness, efficiency, effectiveness, patient-centredness, and equity [13].
Safety
The most important issue here is the misdiagnosis of non-epileptic episodes as epileptic and vice versa. Our app misdiagnosed two patients with epilepsy, one had psychogenic non-epileptic seizures on FF assessment, and the other falling within the uncertain range. The current app had a misdiagnosis rate of 6%, and sensitivity and specificity both of 97%. But epilepsy diagnosis is always difficult and no system is perfect; even FF consultation has been reported to have a rate of misdiagnosis ranging between 5% and 30% [14].
The system mistook newly-diagnosed epilepsy for acute symptomatic seizures probably due to the wording of the questions. This can be modified by revising the wording.
The app was also over-sensitive in diagnosing non-convulsive seizures, picking up all those recognised at FF consultation, but also 12 who were judged not having them. This piece of history review is the most difficult part as it is important to distinguish events separate from the main episode from events associated with it. This is not always easy even during FF consultation in one’s native language. The presence of non-convulsive seizures may potentially lead to the recording of myoclonic seizures by the app but not by FF examination. The epilepsy type was diagnosed as generalised by the app and uncertain by FF. As treatment options for females with generalised and uncertain epilepsy types are the same, the identification of non-convulsive seizures in this case had no effect on the management suggested.
The variability in FF classification of epilepsy type has not been studied so well as episode type, but one study [15] has suggested a rate comparable to the 27% in the present study, which was mostly due to the overdiagnosis of acute symptomatic seizures already referred to.
The treatment advice given by the system was appropriate in all cases apart from the one with misdiagnosis. The app-generated advice in six untreated patients was also regarded as appropriate. Comorbidity can affect the choice of drug used, but this was not an issue in the patients we assessed.
Timeliness
There was a 98% response in 24 h and, even in those requiring a follow-up email, 85% of the referrals were complete within 24 h. This is unlikely to happen by FF examination in any country, however high-income it is.
Efficiency
It took 5 to 10 min for the neurologist to reply to a referral, considerably less than the time of FF consultation for a newly-referred patient. This time efficiency depends first on the neurologist’s trust in the referrer and second, on the app-generated record that is both succinct and relevant to epilepsy management.
Effectiveness
Effectiveness was assessed with a single question, to which 70% of the included patients responded to feel better after the treatment recommended by the app.
Patient-centredness
Patient-centredness was not studied formally in this study; however, it is clear that this app saved both the cost of travelling to obtain a specialist’s opinion and the cost of the opinion itself.
Equity
This referral system is more equitable than FF examination on three counts. First, the information, not the patient, travels directly to the specialist, no matter how remote the patient is from the urban centres which specialists tend to inhabit. Second, as this system is likely to be provided at a lower (if any) cost, it will be more available to poorer patients. Third, during the coronavirus disease 2019 pandemic, conventional FF care is dislocated and this method can readily serve as an alternative.
Study weaknesses
One of the weaknesses of this study is that only a single referrer and a single specialist were involved. But that is how epilepsy services to LMICs are likely to evolve in the absence of investment from governments or large non-governmental organisations. And there has been precious little evidence that such investment is increasing [1] since the declaration by the World Health Assembly in 2015 [6]. Another weakness was the attendance rate of 56% at FF examination which has been dealt with earlier.
Generalisability
This system of asynchronous telemedicine using an app to generate a disease-specific summary is easily replicable but requires both an open-minded referrer and an open-minded specialist; the app simply provides a medical record which links them. Financially for the NPHW, access to such a system may generate further paying referrals; for the specialist the activity can be either voluntary or paid. But both parties have to want to do something to address the well-publicised problems of epilepsy in LMICs.