Performance of the tariff method and physicians in determining stroke as the cause of deaths using verbal autopsy in areas with a limited number of physicians: cases in Indonesia

Background: Physician-certified verbal autopsy (PCVA) is the primary method used to determine the cause of death in Indonesia, although it is very costly and problematic to use in areas where physicians are not widely available with most deaths occur at home. The Tariff method has been piloted to obtain an alternative approach that does not require a physician to determine the cause of death. This validation study presents how the Tariff captures the correctness and distinctiveness of stroke symptoms to the PCVA. Methods: Medical records of 298 adult deaths that occured in four teaching hospitals in the Jakarta from January 1, 2015 to March 2017 were collected prospectively. Verbal Autopsy (VA) was applied using the 2014 WHO instrument diagnosed by a trained physician (PCVA) and by Tariff method. The validity of the VA was assessed by comparing the PCVA diagnoses with the Tariff diagnoses, referring to the best standard. Results: Sensitivity, specificity and positive predictive value (PPV) of VAs using physician’s diagnosis (PCVA) for stroke were 73.9%, 73.5% and 93.4% respectively. The corresponding sensitivity, specificity and positive predictive value (PPV) of VAs diagnosed by the Tariff method were 75%, 61% and 91%. The negative predictive values (NPV) of both techniques were low, 35.6% and 32.6% respectively. Conclusion: The performance of the Tariff method for stroke was almost similar with PCVA, and with a narrower variation, or more consistent than PCVA. Therefore, the Tariff method is a potential alternative to be used on a large scale, because the difficult geographical conditions where physician are not widely available for causes of deaths with distinct signs and symptoms. (Health Science Journal of Indonesia 2021;12(1):39-46)

As a developing country, Indonesia has not fulfilled the WHO's minimum threshold of 23 physicians, nurses and midwives per 10 000 population. 1 To makes it worse, Indonesia's Civil Registration and Vital Statistics (CRVS) system is still in the process of being developed making to provide timely vital statistics, complete and accurate. Located between two continents, Asia and Australia, Indonesia consists of eight large islands and nearly 17,000 small islands stretching more than 5,000 kilometers from Sabang in northern Sumatra to Merauke in Papua. This geographical difficulty has direct negative consequences on the deployment of physicians and other type of health workers in remote areas. Indonesia's governments need to obtain reliable cause of death statistics from various parts of the country including areas without physician, to inform public health policy, respond to emerging health needs, and document progress towards Sustainable Development Goals (SDGs). Therefore, data is collected using the electronic questionnaires on mobile devices and computer algorithms to responses analyzing and estimate the probability causes of death have increased the potential of Computer-Certified Verbal Autopsy method such as Tariff, to be an alternative method to determine cause of death.
Researchers at the Institute for Health Metrics and Evaluation (IHME) develop the Tariff method with an algorithm approach to symptoms or symptoms are recorded in the verbal autopsy instrument. The Tariff method is transparent, intuitive, flexible and at more cost less than PCVA. 2 This application can be downloaded for free through the IHME website. 2 The National Institute for Health Research and Development (NIHRD) has been using this application since 2015 for system activities recording deaths in several areas such as South Kalimantan, Bali and West Nusa Tenggara (NTB). 3 The result showed that the officers at the Health Centre and the District Health Office were able to execute verbal autopsy (VA) using the computerized Tariff method. However, a study for validation of the method to determine cause of death using the Tariff method has never been carried out in Indonesia.
This study presents the strength and specificity of the tariff method to capture stroke symptoms that compared between PCVA and best standard for the method. Stroke is selected cases for point of interest in the validation test, because stroke was the highest cause of death (21.1%) at Indonesia in 2014. 4,5 The result of validation for the two methods in cause of death determining, will be valuable in to appropriate method regarding geographical condition and the limited number of physicians in Indonesia.

METHODS
This study used secondary data from the 'Validation study of determining the cause of death in stroke cases with a physician-certified verbal autopsy and a computer-certified verbal autopsy-tariff method compared to the gold standard'. 6 This study using a cross-sectional design, where sample selection was done purposively in two stages. The first stage is the selection of hospitals as research sample based on the number of deaths due to stroke reported to the DKI Jakarta Provincial Death Surveillance System. Four hospitals were selected in which two hospitals were tertiary hospitals and the rest were secondary hospitals, which all of them were equipped with CT scan facility, and three out of four are teaching hospitals. The second stage is the selection from causes of death in the selected hospital using the death surveillance list and the date when the event happened between January 1st 2015 -March 2017. The measurement for the stroke sample candidate is the death caused by stroke (based on the death surveillance record). The non-stroke sample was chosen if there is no stroke cases was recorded on the death surveillance. All samples were attached by searching their medical records at the sample hospitals, before doing home visit for a verbal autopsy interviews then.  candidate is the death caused by stroke (based on the death surveillance record). The non-stroke sample was chosen if there is no stroke cases was recorded on the death surveillance. All samples were attached by searching their medical records at the sample hospitals, before doing home visit for a verbal autopsy interviews then. 270 cases did not full filled the inclusion of criterias, therefore were not included for further analysis. From 270 cases, 27% of cases were not verified because the un-retrieveable medical records (RM). There are exception for cases who also excluded because deceased's family had moved or not living in the same place (30.7%), and family refused (7.4%). Name of the deceased did not stay at the address as recorded in the medical record (28.9%) or the Tariff method could not determine the cause of death (undetermined cases) (5.9%). Interviews with the deceased family doing through with visited who made by trained interviewers using VA instruments, based on the death surveillance data, and every interviewers are graduates from public health and midwife academies. We used paper-based 2014 WHO VA instrument for adults 7 with additional stroke signs of vertigo, slanted lips and aphasia.
Interviews were held only to witnesses, friends, or family that were assumed to know the stroke symptoms or earlier signs prior to the death.
In the begining, interviews were conducted two times by two trainned enumerators. First enumerator interviewed with the WHO instrument and the second enumerator interviewed with PHMRC instruments. As a matter of fact, many of the deceased families were unable for two time interviewed. Most of the interviews were conducted only by using WHO instrument since PHMRC instrument's questions were also covered in WHO instrument.

The gold standard
The gold standard to determine the underlying cause of death follows the procedure established by the Population Health Metrics Research Consortium (PHMRC). 8 Each medical record is reviewed by 2 to 3 physicians to determine the cause of death and it will be come the gold standard. Death caused by stroke was determined by these following clinical diagnosis criteria:

Stroke (A)
Level 1 Cases were confirmed by: • A Computed Tomography (CT) scan or Magnetic Resonance Imaging (MRI); • Sudden onset of paralysis, coma Level 2A Within the 28 days prior to death, rapidly developing signs of a focal or global loss of cerebral function lasting more than 24 hours (or leading to death) with no apparent cause other than that of vascular origin Then the same medical records were reviewed by two trained and experienced physicians to determine certified cause of death, and if both physicians were concluded the same results, the cause of death was categorized as the cause of death at gold standard. If the result is disparate, the two physicians would have discussions to agree about the causes of death. If there is no agreement for the result, that case was reconciled to the third physician, until it would be used as the gold standard. The cause of death was coded according to the mortality tabulation, classified as a stroke or non-stroke case.

Physician-certified verbal autopsy
Every case of VA was reviewed by a general practitioner who has previous VA training on cause of death determination using ICD-10 codes, and in accordance to the WHO instrument method (without using algorithms), to determine the direct cause, antecedent causes and underlying causes. 6,9 Tariff method-computer-certified verbal autopsy Tariff method in this study were conducted using using Smart VA version-1.2.0 (published May 2017). Data were inputted using PHMRC shortened instrument developed by PHMRC, IHME, Washington University, consisted of 143 questions includes both closed-ended questions and an open-ended narrative.
Based on the response pattern in the VA instrument, the Tariffs subsequently summed and yielding an item-specific Tariff score for each death for each cause. The cause that claims the highest Tariff score for a particular death is assigned as the predicted cause of death for that individual. The tariffs, scores, and ranks are easily observable at each step, and users can easily inspect the basis for any cause decision. 2,10 Analysis and data management The physician is supposed to be able for determine from one to four causes of death, while the Tariff method could determine only one cause of death. We decide to choose for only physician's underlying cause of death to undergo a validation test, defined by 2 categories: the stroke (stroke reffer to ICD-10: I60-I69) and non-stroke. Validity test is held by measuring sensitivity, specificity, positive predictive value of the cause of death determined by physicians and Tariff computerized method.

RESULTS
The average time of interview for each case was approximately 30 minutes, where VA interview with the deceased family of these 298 cases was held after 1-27 months of death. There were 26,8% cases after 0-6 months, 24,2% after 7-12 months, 27,5% after 13-18 months and 21,5% more than 18 months after the events. However, the results of statistical tests between different interview period and the accuracy of physiscians or the Tariff method in determining the cause of death are not significant (P value=0,370 and P.value =0,317). About 49% respondents are children of the deceased, and 31.2% are the spouses. Age range for the deceased was 16-91years old (mean: 61 years old) old and 56% of them is male.
The cause of death was determined based on positive response on symptoms or signs that found on the deceased. The highest response on stroke cases (PCVA) was 'loss of consciousness more than 24 hours' (66%). Most of loss of consciousness occurred all of a sudden (73%) and continued up to the death time (91.6% ). Positive response on 'paralysis' was about 63% cases, out of these 73% experienced hemiparesis (paralyzed on one side of the body). Vertigo was mentioned by 15% cases, asymmetrical lips (19%) and aphasia (27%). The medical record of the deceased (stroke patients) unraveled that: 69% had ever suffered from stroke, 77.2% ever suffered from hypertension, 23.5% ever suffered from a heart disease and 21.3% ever suffered from diabetes mellitus.
The result of all three cause of death methods (gold standard, PCVA and Tariff method) confirms that the major cause of death was stroke (84%) (ICD-10: I60-I69), followed by diabetes mellitus and ischemic heart disease (see Figure 3).    , merely based on a written record about the loss or change of focal and/ or global cerebral functions in the medical record within 28 days before the death, the prediction ability of PCVA was lower than the prediction ability of Tariff computerized method.

DISCUSSION
The findings indicate that both PCVA and Tariff method have similar performance in determining stroke as the cause of death. This study yields a higher sensitivity in determining stroke as cause of death compared to similar research conducted by IHME. 11 Even when compared with the gold standard diagnoses that were made with CT scan result (level 1), the sensitivity result is still higher than the IHME study. The PCVA sensitivity result is closer to validation studies conducted in North India (75%) and China (81.5%). 12,13 It is assumed that higher level of gold standard (with CT scan) would be reflected in a better prediction of PCVA, but there is no effect on Tariff computerized method's performance. Because, even when family members were informed by the hospital about CT In a way, the findings pointed out that Tariff method which calculates a score, or "tariff" for each cause, for each sign/symptom, across a pool of validated verbal autopsy data presented a more reliable response pattern in a verbal autopsy compared to PCVA, which varied according to physician's knowledge, experience, CT Scan results, and supporting notes on additional symptoms within the last 28 days in the VA instrument. In addition, the use of the latest Tariff method version, results show a high sensitivity of 75.1%, which is higher than the previous IHME study results of 51.8%. 11 In conclusion, based on the results, verbal autopsies using Tariff method have similar performance level with PCVA in diagnosing stroke as the cause of death. Verbal autopsies for stroke cases using Tariff method were feasible for Indonesia which has many difficult geographical areas and lack of physicians in remote, border and islands areas. Lack of physicians is not a barrier anymore if Indonesia uses the tariff method in the implementation of VA. The Tariff method is proven to be reliable, transparent and flexible and can be readily implemented by users without training in statistics or computer science. This validation study provides scientific evidence that Tariff method has adequate validity level, as high as PCVA and the gold standard, and in areas where there is no physician.