Background
The pregnancy period is a time for mothers to empower themselves so that their pregnancy and childbirth are safe and comfortable. Pregnant women should seek information about their pregnancy, including dealing with pregnancy discomfort, nutrition, pregnancy danger signs, and preparation for childbirth. Pregnancy danger signs are warning signs that women encounter during pregnancy, birth, and postpartum. It is essential to know these warning signs for women and health care providers to rule out severe complications and start treatment immediately. Lack of knowledge about the danger signs of pregnancy is one of the leading causes of maternal mortality (Dessu et al., 2018; Ministry of Health of the Republic of Indonesia, 2013).
The World Health Organization states that in 2018, 10% of maternal deaths were due to complications during pregnancy, the highest was due to hypertension in pregnancy, namely 31% and bleeding 20% of the research conducted in Tanzania 2019, 49% of deaths occurred during pregnancy and 44% (World Health Organization, 2019). The low level of education causes a lack of knowledge of early detection to prevent complications in pregnancy, childbirth, postpartum, newborns, and toddlers. The Maternal Mortality Rate (MMR) in Indonesia continues to decline from 2017 to 228 per 100,000 live births (Katmini, 2020) to 307 per 100,000 live births in 2018. In 2019, it decreased to 243 per 100,000 live births (Ministry of Health of the Republic of Indonesia, 2019), and the target set in the SDGs, namely 70 per 100,000 live births by 2030 (West Nusa Tenggara Provincial Health Office, 2019). Maternal mortality could be prevented through early detection, including the period preceding pregnancy. The prevention of pregnancy complications is possible through early detection, conducted by regular and quality examinations, especially for high-risk pregnancies (Diana et al., 2020; Nyflot & Sitras, 2018).
Inline to reduce risk factors in pregnancy, the policy direction in the National Medium-Term Development Plan (RJPMN) in the health sector 2020-2024 (Ministry of Health of the Republic of Indonesia, 2020) is strengthening primary health care, the significant efforts to prevent and promote health by developing innovations in the use of technology, in improving health towards universal health coverage. The primary strategy of improving maternal and child health, especially health services during pregnancy, aims to fulfill every pregnant woman's right to obtain quality health services to have a healthy pregnancy, give birth safely, and give birth to a healthy and quality baby. During pregnancy, this health service is carried out through comprehensive and quality integrated antenatal services, one of which is the early detection of problems, diseases, and complications or complications of pregnancy under standards and is recorded in the Maternal and Child Health (MCH or known as KIA) book (Ministry of Health of the Republic of Indonesia, 2015).
In Indonesia, access to health facilities is 94.1% more than five kilometers from residential areas. It is the basis for developing several telehealthcare that can be accessed by the community directly, without having to meet with health workers. Telehealthcare can be done anywhere anytime and can change the existing paradigm in society to get health services and information about their health in real-time and get direct recommendations from experts according to their needs. One of the most widely used ways today is to develop applications that can be accessed quickly and get accurate information about their condition (Wiweko et al., 2016). In pregnancy conditions, the care needed by pregnant women is critical to be accessed in real-time because recommendations and detection can be implemented so as not to enter into dangerous things because care during pregnancy can be continued during childbirth and postpartum and newborns (Manuaba, 2012).
The Indonesian Ministry of Health has designed the MCH Handbook to contain notes from pregnant women to toddlers, including examination standards, danger signs, education, care, and how to provide first aid measures to toddlers. The reality in the field states that the MCH handbook is only 56%, and the compliance of midwives in filling out the complete MCH handbook is only 18% (Putri, 2018). The MCH handbook was not fully utilized, which should have been an effective medium for counseling and education. One of the factors that caused the MCH Handbook to be less effective, among others, was in the form of a book; it was often torn, easily lost, and impractical, causing disconnection of communication tools between health workers recorded in the MCH book. In 2017, an MCH book was developed in the form of a Web on Android-based that before and after introducing danger signs to pregnant women by providing hazard sign material in the form of a website, there is a change in accuracy speed in case finding. Still, there are no recommendations for pregnant women (Fajriya et al., 2017). Detection of dangerous signs of pregnancy with the MCH Handbook, among others, states that the MCH Handbook can be an easy-to-understand, involving husbands, providing more effective interventions, and providing knowledge about danger signs so that mothers and husbands make decisions more efficiently. Knowledge, attitudes of staff, and family support can improve the use of the MCH Handbook to recognize danger signs (Sari & Suhita, 2019). Research by Hidayah et al. (2019) regarding the development of the MCH book states that the MCH book expert validation test with a value of 90.15% and trials with 92.8% results so that e-books can be developed with the PDF method, which is used as an independent learning method.
In this era of digitalization, smartphones have become a daily necessity. Currently, the use of smartphones in Indonesia is the seventh of all countries in the world, namely 2.34 billion users, and pregnant women use smartphones to access information in their pregnancy (Wiweko et al., 2018), which is expected to improve public health status and change people's behavior according to health references (World Health Organization, 2019). The result shows that mobile and internet-based technology increases maternal knowledge in carrying out high-risk early detection so that case finding can be done as early as possible and decision making can be carried out immediately by pregnant women and their families.
The development of health technology today focuses on applying systems that can be accessed in real-time with a guaranteed level of trust and adapted to medical ethics. Human-In-The-Loop (HITL) is the development of health technology that can be used to detect the danger signs of pregnancy with the role of health workers to determine the instructions and recommendations of HITL. In developing HITL interactions, the algorithm will make predictions in determining the conditions of pregnant women's danger signs, and the expert will assess to optimize it in decision making (Holzinger et al., 2016), which can be done in real-time by accessing the application.
In research conducted during the COVID-19 pandemic, the government provided a policy to postpone all types of pregnancy examinations during the COVID-19 period to make the social distance program a success. However, pregnant women can be assessed if there are complaints or danger signs that can be read through the book (Perkumpulan Obstetri dan Ginekologi Indonesia, 2020). An innovation that can be developed for early detection of pregnancy danger signs during the COVID-19 period is the development of an electronic MCH book which is equipped with artificial intelligence and recommendations where decision making is based on algorithms combined with artificial intelligence to make decisions with the application of human-in-the-loop Indonesia (Nurjasmi, 2020). Therefore, this study aimed to develop the application of HITL for early detection of pregnancy danger signs to prevent morbidity and mortality of pregnant women.
Methods
We collected data from the cohort of 5324 pregnant women at the Community Health Centers (CHCs) in the West Lombok District from 2020 to February 2021. The data included age, parity, height, inter-pregnancy interval, hemoglobin levels, upper arm circumference, previous diseases, and bleeding history. Of the 5324 obtained data, 67 percent were healthy pregnant women. Pregnant women who exhibited signs of danger during pregnancy made up 33% of the total, with 27% of those at low risk, 5% of those at moderate risk, and 1% of those at high risk.
System Design
Figure 1 depicts the system design for a Human-In-The-Loop (HITL) system for identifying pregnancy danger signs. The data were processed with machine learning to predict the danger signs. The training was carried out using ten individual classifiers, including Nearest Neighbors, Decision Tree, Random Forest, Neural Net, AdaBoost, Gaussian Naïve Bayes, Bagging, Extra Tree, Gradient Boosting, and Stacking. The decision tree outperformed the other ten individual classifiers, with an accuracy of 92% ± 0.0351. Furthermore, the classification results and recommendations were sent to the server to build a REST API, which allowed the program to be called by other applications. The midwife then reviewed and clarified the suitability of the data for generating the final detection and made a recommendation (Table 1).
| Type of Risk | Recommendation |
|---|---|
| Pregnant women with no risk | Pregnancy check-up at village health post by a village midwife, pregnancy check-up with the doctor at the Community Health Center (CHC) once, complete blood and urine tests in Trimester 1 and Trimester 3. Delivery at the village health post. |
| Low risk | Pregnancy check-up at village health post by a village midwife, pregnancy check-up at the CHC for integrated antenatal care, complete blood, and complete urine tests. Delivery at the CHC |
| Moderate risk | A pregnancy check-up with the midwife at the Community Health Center was to get a laboratory analysis and prepare for delivery at the CHC. |
| High risk | Perform a pregnancy check-up immediately! Get further examination by a doctor at the CHC and a referral to a specialist doctor. Preparation for delivery at the hospital. |
Decision Tree
Decision Tree aims to create a model in which behavior is easy to explain and understand a particular classification by examining all the filters learned in the path between the node and the root of the problem (Frosst & Hinton, 2017). This model does not rely on hierarchies but utilizes the hierarchical features of traditional neural networks that allow us to learn new representations from inputted data (Figure 2).
Using trained neural networks to make decisions in a decision tree can be learned by conducting data training. The network will make certain classifications depend on the distributed hierarchical representation. These decision tree data can also be used to understand the importance of the results for decision-making by using many data sets (Sharaff & Gupta, 2019).
The feature used in collecting the data set used several risk factors whose effects have been tested to relate to the occurrence of risk in pregnancy. The system predicted danger signs regarding the data of age, parity, height, inter-pregnancy interval, hemoglobin levels, upper arm circumference, previous diseases, and bleeding history. These data were then inputted into the classification system, and the further diagnosis was in line with machine learning prediction.
Human-In-The-Loop
Human-In-The-Loop is an application that combines expert intelligence and artificial intelligence (Holzinger et al., 2016), which uses machine learning to predict algorithms from patient data to classify diagnoses. Human-in-the-loop is involved in data retrieval and pre-processing to determine the application’s features (Kieseberg et al., 2016). The component in midwife-in-the-loop is that a midwife was selected based on qualifications of having a minimum education of midwifery diploma three, a registration certificate (STR), and more than two years of working experience. The human-in-the-loop clarified the data that provided diagnostic feedback. The means of communication between human and patient is a knowledge base and data that store all workflows. Data analysis is implemented as a knowledge base procedure. In building a human-in-the-loop, it is essential always to build trust wherein the respondent must fill in the data correctly, and a human will process the data and provide the correct diagnosis. Thus, it is essential to maintain the confidentiality of patient data and lock the application.
Performance Evaluation
The performance of this system had been evaluated based on the confusion matrix calculated with 5324 data on pregnant women. The accuracy of the midwife-in-the-loop application data was calculated by using the following formula:
Accuracy is the ratio of the actual data’s correct prediction (positive and negative). The accuracy value compares the data classified correctly and the data obtained from the equation result.
Ethical Considerations
Mataram University approved ethical clearance of the study with number 44 / UN.18.F7 /Etik / 2021.
Results
In this study, there were 5313 data on pregnant women. 90% of the data were from pregnant women with particular risk, and the rest were from pregnant women with no risk. The actual data used to predict the risk of danger signs for pregnant women were used in the system rules and found that 80% of pregnant women had danger signs of pregnancy, and 20% of pregnant women did not have danger signs of pregnancy.
Evaluation of data accuracy with K-folding stratification ten times resulted in data with 92% ± 0.0351 accuracies (Table 2).
| Precision | recall | f1-score | Support | |
|---|---|---|---|---|
| 0 | 0.97 | 0.96 | 0.96 | 3491 |
| 1 | 0.87 | 0.87 | 0.87 | 1464 |
| 2 | 0.66 | 0.80 | 0.72 | 299 |
| 3 | 0.76 | 0.42 | 0.54 | 59 |
| Accuracy | 0.92 | 5313 |
Android-Based Application of Midwife-In-The-Loop
We developed a human (midwife)-in-the-loop mobile application to predict the risk of danger signs of pregnancy and then provide appropriate recommendations. Figure 3 illustrates an example of a HITL mobile application. The classification model is constructed using a decision tree. The classification result will be reviewed in conjunction with the woman's profile data and the diagnosis results obtained after the profile data is stored, and the system will then provide the initial recommendation. Figure 4 demonstrates that the midwife-in-the-loop application reviewed and clarified the classification result to produce the final classification result and final recommendation.
Discussion
Detection of pregnant women's risk is done through screening using the Maternal and Child Health handbook. Maternal data parameters to determine risk classification using the Poeji Rochjati score (Widarta et al., 2015), making it easier for health workers to provide planned referrals and even supervision by the community such as health cadres. Poeji Rochjati scores can also be used as a communication tool between cadres and midwives, where initial screening can be easily carried out by non-medical personnel such as trained health cadres.
Detection at the beginning of the examination also shows the effectiveness of the intervention, in addition to providing services with an antenatal care standard, as known as 10T (Agustian et al., 2017), supporting examination services and doctor examination services at the primary health care which are carried out at least one time during pregnancy to carry out early detection of specific clinical conditions (Simbolon et al., 2020). 10T However, during the COVID-19 pandemic, examinations of pregnant women were carried out when they had complaints and experienced danger signs. This is done to reduce the risk of transmission between health workers and patients in this pandemic era.
Research conducted by Lee et al. (2012) on danger signs in pregnant women, high-risk pregnant women will agree with competent health workers for planned care and referrals, involving professional doctors who will carry out pregnancy care until delivery. However, according to Wiweko et al. (2018), sometimes there is a debate about determining risk, so research is needed to find accurate data to determine the level of risk experienced by pregnant women.
From the results of the study, it was found that high-risk predictions with Poeji Rochjati scores; older respondents, more parity, lower height, the distance of children < 2 years, hemoglobin < 11 g/dl, UPC (upper arm circumference) < 23.5 cm, suffering from HBsA, having HIV, having a history of diabetes mellitus, hypertension, and having a history of other diseases are more likely to have a high maternal risk. However, the supervision and monitoring of pregnant women with some predicted risks using the Poeji Rochjati score can be carried out by health workers, not forgetting to involve families, health cadres, and related cross-sectors. Planning for childbirth by the mother and family and the recommendations given can be a mutually agreed notice so that pregnancy care and delivery plans can also be determined by the mother and family as early as possible.
Grobman et al. (2018) research stated that pregnant women with multiparas had a lower risk than primiparas. This is contrary to the results of this study which indicated that pregnant women with high parity have a higher risk of being in danger of childbirth. The gap between the two studies is that the research subjects have different mean parities.
In addition, the research of Sundermann et al. (2017) suggests that mothers with a low risk of pregnancy ranges from 27 to 30 years. In 60 studies, it was found that older mothers had a higher risk of conceiving. The results differ between the two studies due to the different age ranges and maternal age categorization. This study states that the distance of children who are too close significantly affects the high risk of pregnant women.
The distance of children that are too close and affect pregnancy is in line with the research of Chen et al. (2020), which stated that a woman with a recurring birth frequency would be more at-risk during childbirth. In addition, this result is in line with this study which indicated that pregnant women are at high risk, pregnant women who have hemoglobin < 11 g/dl, upper arm circumference < 23.5 cm, have HBsAg, have HIV, have a history of DM, and hypertension.
Research conducted by Zhu et al. (2019) indicated that telemedicine is an option in the modern era to find information and communicate in real-time with health workers, especially during the pandemic, where there are restrictions. Furthermore, telemedicine is one of the most effective ways to connect health workers with patients.
Applications for monitoring and delivery planning for pregnant women are considered more efficient because they can be filled in anywhere and get recommendations from health workers in real-time. This is also following the medium-distance development plan that health technology focused on maternal and child health focuses on early detection to provide quality services to reduce morbidity and mortality rates.
In this application (research output), the persons designated as human-in-the-loop are midwives in the working area of primary health care. They will provide recommendations for places of prenatal care and recommendations for places of delivery following self-report data provided by patients. In case midwives and pregnant women at risk can make an appointment to examine under examination standards in the era of the COVID-19 pandemic, where pregnant women are not recommended to carry out routine checks but carry out minimal inspections with predetermined standards carry out inspections if experiencing danger signs.
In this study, the researchers developed an android application that can predict danger signs in pregnancy. Predictions were built by ten individual classifications, namely Nearest Neighbors, Decision tree, Random Forest, Neural Net, Adaboost, Gaussian Naïve-Bayes, Bagging, and Extra Tree, with values the accuracy of each algorithm (10-folding) results obtained from the accuracy decision tree or an accuracy rate of 92%. However, there is data confusion in machine learning results, namely 25 data at high risk. For the high risk, it is not based only on application predictions but also uses supporting examinations that will be carried out after human-in-the-loop provides recommendations for places antenatal care and delivery referral services. Furthermore, this application with the application of human-in-the-loop can also be used repeatedly, both in the I, II, and III trimesters, so that if there are complaints that can give a higher score, it can change the risk of pregnant women with treatment and referrals according to the classification.
Limitation of the Study
The limitation of this research is that the application is not equipped with a Global Positioning System (GPS), so monitoring of the local area that becomes the mapping of the location of pregnant women cannot be carried out. Thus, it is recommended to use GPS to facilitate the mapping of pregnant women so that early detection can be carried out quickly by mapping the location and providing recommendations for places of examination at facilities closer to the pregnant women's homes.
Conclusion
This study developed a Human-In-The-Loop mobile app that can be used by healthcare professionals, especially midwives and nurses, to detect danger indications early in pregnancy, accurately diagnose the high risk of pregnancy, and provide treatment and care recommendations during pregnancy and childbirth. This study employed a decision tree algorithm and produced a high accuracy value of 92% ± 0.0351.