Sudden stillbirth still poses as one of the key challenges in maternal and fetus care, especially in developing nations where sophisticated labor ward monitoring systems cannot be afforded. It becomes very challenging to detect a pregnancy at risk early due to the combination of several risk factors related to both mother and the fetus. This paper presents the design of a Human-in-the-Loop Explainable Artificial Intelligence (XA)I-based predictive labor ward model to help detect composite risks related to sudden stillbirth. For this, the research considers clinical records on 90 pregnant mothers and then utilizes machine learning (ML) models such as Logistic Regression, Random Forest, and XGBoost for predictions. XAI algorithms are utilized to enhance transparency, interpretability, and clinician understanding of predictive results. It is found that the highest prediction accuracy can be achieved by usinsg the XGBoost-XAI method, which is superior to traditional approaches. Hypertension in mother, fetal distress, placental inefficiency, gestational diabetes, and prolonged labor are some of the most significant predictors of sudden stillbirth. The Human-in-the-Loop concept makes it more reliable.