by Researchers have recently emphasized the critical role of model interpretation in the management of type 1 diabetes (T1D). In a study published in Scientific Reports, the researchers discussed the importance of tools that can interpret the results generated by predictive models in T1D management.
To demonstrate this, the researchers utilized a case study focused on a female T1D patient from the OhioT1DM dataset. They retrospectively analyzed the patient’s data to identify a suitable prediction algorithm that could be integrated into a decision support system (DSS). This DSS would then provide suggestions for corrective insulin boluses (CIB) to improve the patient’s management of T1D.
The researchers conducted experiments using two Long Short Term Memory neural networks (LSTM) models: non-physiological (np)-LSTM and physiological (p)-LSTM. Both models had similar prediction accuracy but differed in the clinical decisions they provided.
LSTMs are ideal for time series prediction because they are capable of learning and maintaining long and short-term dependencies in data. While both np-LSTM and p-LSTM used the same input features and structure, p-LSTM had an additional non-learnable, pre-processing layer that approximated the physiological decay curves of insulin and carbohydrate (CHO) in the patient’s body.
Type 1 diabetes is characterized by impaired glucose homeostasis, and patients must self-administer insulin and adhere to restricted diets and exercise routines. Maintaining blood glucose (BG) levels within the required range reduces the risk of mortality and complications related to hyperglycemia.
Continuous glucose monitoring (CGM) sensors have made it easier for patients to monitor their BG levels. These sensors provide BG measurements every five minutes and generate alerts when BG exceeds certain thresholds. This enables patients to take corrective actions, such as administering CIB, to manage their glycemic levels effectively.
Advanced decision support systems and artificial pancreas systems (APS) are essential for real-time BG level predictions and automated insulin delivery. Machine learning models with decision support systems have become increasingly popular in T1D management as they can forecast BG levels and provide therapeutic recommendations, such as CIB.
Although these machine learning models guarantee accuracy, there are concerns about the interpretability of their outputs. The lack of transparency in their logic and the hidden biases in T1D datasets can sometimes lead to misinterpretations of the effect of inputs on BG levels. This could be dangerous when models are used to suggest therapeutic actions in clinical practice. Therefore, tools that can interpret model outcomes, such as SHapley Additive exPlanation (SHAP), are necessary.
SHAP comprehends the predictions of an algorithm on an individual basis and determines the contribution of each input to the model’s output.
For this study, the researchers selected a female T1D patient who diligently recorded her meals and CIB data over a period of 10 weeks. Her CGM data had only a 3% measurement error, allowing for a fair assessment of predictive algorithms and DSS performance. The patient had elevated time-above-range (TAR) and time-in-range (TIR) values throughout the test dataset.
The researchers used the last 10 days of the patient’s data to determine the prediction accuracy of the models and used the remaining six weeks of data to train the two LSTMs. They also used a subset of the test set to evaluate the insulin corrective actions suggested by the DSS.
The researchers utilized a novel in-silico methodology called ReplayBG to retrospectively evaluate the effectiveness of the corrective actions suggested by the DSS of the LSTM models.
The SHAP summary plot revealed the importance of each feature in the study dataset. The CGM and insulin were identified as the top two significant features. However, the impact of insulin on the model’s output appeared to be weak, as indicated by the small magnitude of the SHAP values associated with this feature.
Interestingly, some values of CHO had a positive impact on BG predictions, while others had a negative impact. This finding was surprising since CHO intake typically increases BG levels in T1D patients. It suggested that the model mainly relied on past CGM readings to predict future BG levels.
The observed SHAP values indicated that the collinearity between insulin and CHO in the test dataset made it challenging for the learning algorithm to distinguish the individual effects on the model’s output.
In np-LSTM, insulin positively contributed to the model’s output for specific time periods, implying that np-LSTM would predict a glucose spike after an insulin bolus, even if the patient did not consume CHO.
In conclusion, SHAP provided insights into the outputs of black-box models and demonstrated that only p-LSTM learned the physiological relationship between inputs and glucose prediction. Only p-LSTM improved the patient’s glycemic control when integrated into the DSS. Therefore, p-LSTM is the most suitable model for decision-making applications in T1D management.
*Note:
1. Source: Coherent Market Insights, Public sources, Desk research
2. We have leveraged AI tools to mine information and compile it
