Window-Aware Scheduling for Health Messaging under Platform Delivery Constraints: Evidence from an AI-Powered WhatsApp Maternal and Infant Health Chatbot in Nigeria
Organizations: HelpMum Africa
Abstract
WhatsApp lets a business send free-form messages only within 24 hours of a user's last message; outside this customer service window, only pre-approved templates are delivered. Health chatbots inherit this rule, but its cost is rarely measured. We analyse 98,607 messages sent to 3,696 mothers by a WhatsApp maternal health chatbot in Nigeria, with a largely separate export as replication. No free-form message failed with the window open (0 of 35,500), and failure jumped by 93.0 percentage points (95% CI 87.0 to 99.1) at exactly 24 hours. Two flows produced 66.1% of failures, including an offer to speak with a medical professional that, as built, reached 0.3 in 100 mothers whose window had closed. We formalise window-aware scheduling as a decision problem with an action of known zero value. Model-based offline evaluation, with bounds for unconfirmed delivery, estimates that sending the offer as an approved template would reach 81.5 to 86.8 in 100 such mothers, against 16.6 for waiting and 11.6 for a reply prompt. In a calibrated simulation, tabular Q-learning converges to the same template rule and does not beat it in any of 13 stress tests, and opting out does not rise with message volume. The main gain comes from respecting the platform rule, and it needs no learning. All outcomes are deliveries, not health outcomes.
Figures & tables
| Main | Replication | |
| Bot messages | 98,607 | 23,138 |
| free-form | 50,574 | 12,266 |
| templates | 48,033 | 10,872 |
| Messages from mothers | 29,297 | 7,586 |
| Mothers | 3,696 | 927 |
| Period | 2023 to 2026 | 2023 to 2026 |
| Format | Window | Failed | Unconf. | |
|---|---|---|---|---|
| Free-form | Open | 35,500 | 0.0% | 0.1% |
| Closed | 6,267 | 98.6% | 0.0% | |
| None obs. | 8,807 | 90.9% | 0.0% | |
| Template | Open | 1,690 | 0.1% | 0.0% |
| Closed | 21,736 | 3.3% | 4.3% | |
| None obs. | 24,607 | 26.6% | 7.8% |
| Content | Logged | Template, confirmed | Template, not failed | Wait 7 days | |
|---|---|---|---|---|---|
| Medical offer | 3,933 | 0.3 (0.2 to 0.5) | 81.5 (79.8 to 83.0) | 86.8 (85.7 to 87.9) | 16.6 (14.8 to 18.1) |
| Error follow-up | 8,209 | 1.5 (1.2 to 2.0) | 80.9 (79.3 to 82.5) | 86.4 (85.2 to 87.5) | 17.2 (15.5 to 18.6) |
| Other content | 609 | 99.5 (98.8 to 100) | 67.5 (65.2 to 69.6) | 75.0 (73.1 to 77.2) | 88.8 (84.1 to 92.8) |
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
| Contents | |
|---|---|
| A | Proofs of Propositions 1 to 4; the hindsight bound; the scheduling logic (fig:logic) |
| B | Variables, observed and true window state, truncation check, error codes, discontinuity and placebos (tab:rd, tab:placebo, fig:handoff, fig:rd, fig:flows) |
| C | Decision models and feature coding, sample reconciliation, transport check, calibration, restricted samples, bandit replay (tab:calib, tab:restricted, tab:replay, fig:km, fig:map, fig:bandit) |
| D | Simulator inputs and dynamics, Q-learning details, strategies and costs, suppressing the offer, validation, stress tests (tab:strategies, tab:suppress, fig:val, fig:cost, fig:stress) |
| E | Burden definitions, within-mother and pooled models (tab:burden) |
| F | Send time within batched broadcasts, clustering (fig:time) |
| Bandwidth (h) | Jump | 95% CI |
|---|---|---|
| 6 | 89.8 | 75.9 to 103.7 |
| 12 | 89.1 | 80.2 to 98.0 |
| 18 | 91.5 | 84.2 to 98.9 |
| 24 | 93.0 | 87.0 to 99.1 |
| 48 | 95.1 | 91.3 to 99.0 |
| 72 | 93.9 | 90.8 to 96.9 |
| Free-form | Templates | |||
|---|---|---|---|---|
| Cut-off (h) | Jump | 95% CI | Jump | 95% CI |
| 12 | 0.0 | no failures | 0.0 | 0.0 to 0.1 |
| 36 | 4.3 | 11.6 to 3.0 | 1.5 | 3.1 to 6.2 |
| 48 | 14.1 | 28.1 to 0.0 | 0.0 | 3.2 to 3.1 |
| 72 | 4.4 | 16.2 to 7.4 | 2.5 | 1.9 to 7.0 |
| 120 | 4.1 | 7.3 to 15.5 | 2.7 | 1.9 to 7.2 |
| Decile | Predicted | Observed | Confirmed |
|---|---|---|---|
| 1 | 59 | 62 | 52 |
| 2 | 68 | 60 | 52 |
| 3 | 75 | 76 | 69 |
| 4 | 79 | 83 | 76 |
| 5 | 81 | 78 | 70 |
| 6 | 89 | 93 | 86 |
| Sample | Logged | Template lower | Template upper | Wait | Reply | |
|---|---|---|---|---|---|---|
| All closed or none obs. (paper) | 12,751 | 5.8 | 80.4 | 86.0 | 20.4 | 15.2 |
| 95% CI | 5.2 to 6.5 | 79.8 to 81.1 | 85.5 to 86.5 | 18.9 to 22.0 | 13.9 to 16.5 | |
| Earlier message observed | 4,225 | 0.1 | 97.3 | 99.0 | 28.8 | 20.6 |
| 95% CI | 0.0 to 0.1 | 97.2 to 97.4 | 98.9 to 99.0 | 26.4 to 31.4 | 18.4 to 22.7 | |
| Offer and error follow-up | 12,142 | 1.1 | 81.1 | 86.5 | 17.0 | 11.6 |
| 95% CI | 0.8 to 1.4 | 80.4 to 81.8 | 86.0 to 87.1 | 15.5 to 18.5 | 10.4 to 12.9 |
| Sample | Masked | Unmasked | Blind | Logged | |
|---|---|---|---|---|---|
| All decisions (main text) | 19,827 | 1.4 | 1.5 | 2.3 | 49.7 |
| Without closed other content | 19,218 | 1.0 | 1.0 | 1.8 | 51.4 |
| Earlier message observed | 11,301 | 0.8 | 0.9 | 2.3 | 35.3 |
| Strategy | Offers | Value | Reward | Cost |
|---|---|---|---|---|
| Logged | 1.3 | 1.3 | 22.5 | 0.24 |
| Wait rule | 10.2 | 3.8 | 23.1 | 0.26 |
| Reply button ( ) | 18.5 | 13.8 | 25.6 | 0.29 |
| Hybrid | 91.7 | 51.2 | 34.4 | 0.48 |
| Template rule | 90.3 | 90.3 | 44.6 | 0.47 |
| Threshold | 90.3 | 90.3 | 44.6 | 0.47 |
| Template | Suppress | Q-learning | Offers | ||
|---|---|---|---|---|---|
| 1.7% | 30.8 | 18.7 | 20.6 | 30.8 | 89.0 |
| 3.4% | 22.4 | 15.7 | 16.7 | 22.3 | 87.2 |
| 6.8% | 13.4 | 11.3 | 11.5 | 13.5 | 83.6 |
| 13.5% | 7.0 | 6.8 | 6.6 | 7.1 | 71.9 |
| 27.1% | 3.5 | 3.5 | 3.4 | 3.5 | 46.5 |
| Specification | Estimate (95% CI) |
|---|---|
| Opt-out within 24 hours, within mother | |
| All messages, 7 days | ( to ) |
| All messages, 24 hours | ( to ) |
| Before first opt-out only | ( to ) |
| Templates, with recency | ( to ) |
| Read within 24 hours, within mother | |
| Main | Replication | |
|---|---|---|
| Free-form failed, open | 0 of 35,500 | 0 of 9,546 |
| Free-form failed, closed | 98.6% | 97.6% |
| closed messages | 6,267 | 544 |
| 18 to 30 h, before 24 h | 0 of 114 | 0 of 19 |
| 18 to 30 h, after 24 h | 72 of 79 | 18 of 18 |
| Jump at 24 h (points) | 93.0 | 100.0 |
Conversational Agents
Conversational agents are AI systems that interact with people through multi-turn dialogue to answer questions, provide support, or complete tasks.
See all 306 papers