Organizations: CueZen, Inc., Seattle, 98101, WA, USA. · Department of Critical Care Medicine, Swedish Medical Center, Seattle, 98122, WA, USA. · Computer Science & Systems, University of Washington, Tacoma, 98402, Washington, USA.
The ability to shape health behaviors of large populations automatically, across wearable types and disease conditions at scale has tremendous potential to improve global health outcomes. We designed and implemented an AI driven platform for digital algorithmic nudging, enabled by a Graph-Neural Network (GNN) based Recommendation System, and granular health behavior data from wearable fitness devices. Here we describe the efficacy results of this platform with its capabilities of personalized and contextual nudging to n=84,764 individuals over a 12-week period in Singapore. We statistically validated that participants in the target group who received such AI optimized daily nudges increased daily physical activity like step count by 6.17% (p=3.09×10−4) and weekly minutes of Moderate to Vigorous Physical Activity (MVPA) by 7.61% (p=1.16×10−2), compared to matched participants in control group who did not receive any nudges. Further, such nudges were very well received, with a 13.1% of nudges sent being opened (open rate), and 11.7% of the opened nudges rated useful compared to 1.9% rated as not useful thereby demonstrating significant improvement in population level engagement metrics.
Figures & tables
Group
Group 1: Participants in physical activity program only
Group 2: Participants in both physical activity and nutrition programs
Group n
14,901
154,766
Treatment
No Nudges
Nudges
No Nudges
Nudges
n
7,465
7,436
77,438
77,328
Mean age
47.2
47.6
46.0
46.0
% Female
48.0
47.7
62.9
63.1
% Male
52.0
52.3
37.1
36.9
Table 1: Participants in control and treatment groups, and their characteristics.
Figure 1: Effect of daily personalized nudges on steps and MVPA, nudge open rates and ratings, and dose-response relationship between nudges opened and physical activity.
Figure 2: A. Schema of the NudgeRank™ system to deliver personalized algorithmic nudges to participants. B. Example of the knowledge graph used to select nudges from the Nudge Library. See Methods section for more details.
Behavior Goal
Nudge Technique
Nudge
Steps
Framing
Boost your step count by skipping the lift and taking the stairs when you are out and about! You can move more by adopting this healthy habit.
Steps
Gamification
Move a little more and aim to hit at least 5,000 steps each day. Get up and take a short walk after every 30 minutes of desk work to gain extra steps.
Steps
Reminder
Don’t let your fitness data go to waste! Sync your fitness tracker to record your progress.
Steps
Social Influence
9 in 10 active users clock an average of 5,000 steps daily! Move more and be part of this active community.
MVPA
Framing
No time to exercise? A quick 15-minute workout only takes up 1% of your day! Track your progress and stay motivated!
MVPA
Gamification
Your MVPA minutes in the last seven days is lower than what you’ve achieved the week before. But don’t throw in the towel! Fire up your motivation, and put in some extra time to work out for the daily prize.
Table 2: Examples of personalized nudges that were sent to participants.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Week
No Nudges ( n=84,903 )
Nudges ( n=84,764 )
Difference
t statistic
p -value
Mean
S.D.
Mean
S.D.
Steps
%
0
4,064.8
4,546.6
4,070.8
5,344.7
6.1
+0.15%
0.252
4.005×10−1
1
3,626.0
4,114.6
3,689.6
4,752.5
63.6
+1.75%
2.948
1.598×10−3
2
3,781.9
4,305.8
3,856.1
5,027.5
74.2
+1.96%
3.265
5.469×10−4
3
3,878.7
4,303.0
3,957.7
5,097.8
78.9
+2.03%
3.445
2.855×10−4
4
3,701.3
4,261.4
3,781.9
5,016.4
80.6
+2.18%
3.567
1.809×10−4
Appendix
Table 3: Average daily steps over 12 weeks, for all participants who received nudges (”Nudges” cohort) vs. those who did not receive nudges (”No Nudges” cohort). Week 0 is the week prior to the start of nudging. p -values in bold indicate statistical significance ( p≤0.05 ) in a one-tailed independent samples t-test.
Week
No Nudges ( n=7,465 )
Nudges ( n=7,436 )
Difference
t statistic
p -value
Mean
S.D.
Mean
S.D.
Steps
%
0
3,138.5
4,063.0
3,153.3
4,064.2
14.9
+0.47%
0.223
4.117×10−1
1
2,856.9
3,727.2
3,004.0
3,831.6
147.1
+5.15%
2.375
8.769×10−3
2
2,952.1
3,809.4
3,130.0
3,897.1
177.9
+6.03%
2.818
2.417×10−3
3
3,009.0
3,777.7
3,230.5
3,914.4
221.5
+7.36%
3.515
2.203×10−4
4
2,921.1
3,792.0
3,092.6
4,013.9
171.5
+5.87%
2.680
3.683×10−3
Appendix
Table 4: Average daily steps over 12 weeks, for Group 1 participants (enrolled in the physical activity program only) who received nudges (”Nudges” cohort) vs. those who did not receive nudges (”No Nudges” cohort). Week 0 is the week prior to the start of nudging. p -values in bold indicate statistical significance ( p≤0.05 ) in a one-tailed independent samples t-test.
Week
No Nudges ( n=77,438 )
Nudges ( n=77,328 )
Difference
t statistic
p -value
Mean
S.D.
Mean
S.D.
Steps
%
0
4,154.1
4,580.7
4,159.1
5,443.9
5.0
+0.12%
0.196
4.224×10−1
1
3,700.1
4,142.5
3,755.5
4,826.7
55.4
+1.50%
2.424
7.679×10−3
2
3,861.9
4,342.4
3,925.9
5,117.7
64.0
+1.66%
2.654
3.976×10−3
3
3,962.6
4,341.1
4,027.6
5,192.1
65.0
+1.64%
2.671
3.780×10−3
4
3,776.5
4,296.5
3,848.2
5,097.6
71.7
+1.90%
2.991
1.392×10−3
Appendix
Table 5: Average daily steps over 12 weeks, for Group 2 participants (enrolled in both the physical activity and nutrition programs) who received nudges (”Nudges” cohort) vs. those who did not receive nudges (”No Nudges” cohort). Week 0 is the week prior to the start of nudging. p -values in bold indicate statistical significance ( p≤0.05 ) in a one-tailed independent samples t-test.
Week
No Nudges ( n=84,903 )
Nudges ( n=84,764 )
Difference
t statistic
p -value
Mean
S.D.
Mean
S.D.
MVPA Minutes
%
0
81.76
218.62
82.73
227.71
0.96
+1.18%
0.888
1.874×10−1
1
78.49
208.15
80.29
216.06
1.79
+2.29%
1.741
4.082×10−2
2
82.31
216.17
85.42
228.36
3.11
+3.78%
2.884
1.965×10−3
3
84.69
223.41
86.07
226.57
1.38
+1.63%
1.260
1.038×10−1
4
82.11
219.72
84.19
226.55
2.07
+2.53%
1.914
2.783×10−2
Appendix
Table 6: Weekly minutes of MVPA over 12 weeks, for all participants who received nudges (”Nudges” cohort) vs. those who did not receive nudges (”No Nudges” cohort). Week 0 is the week prior to the start of nudging. p -values in bold indicate statistical significance ( p≤0.05 ) in a one-tailed independent samples t-test.
Week
No Nudges ( n=7,465 )
Nudges ( n=7,436 )
Difference
t statistic
p -value
Mean
S.D.
Mean
S.D.
MVPA Minutes
%
0
39.53
79.98
39.95
78.37
0.42
+1.05%
0.321
3.740×10−1
1
42.65
108.42
44.86
101.77
2.21
+5.18%
1.281
1.000×10−1
2
44.25
109.80
49.00
123.38
4.75
+10.74%
2.484
6.499×10−3
3
47.41
127.02
50.58
123.62
3.17
+6.69%
1.544
6.128×10−2
4
47.90
125.28
50.64
123.65
2.74
+5.71%
1.341
8.992×10−2
Appendix
Table 7: Weekly minutes of MVPA over 12 weeks, for Group 1 participants (enrolled in the physical activity program only) who received nudges (”Nudges” cohort) vs. those who did not receive nudges (”No Nudges” cohort). Week 0 is the week prior to the start of nudging. p -values in bold indicate statistical significance ( p≤0.05 ) in a one-tailed independent samples t-test.
Week
No Nudges ( n=77,438 )
Nudges ( n=77,328 )
Difference
t statistic
p -value
Mean
S.D.
Mean
S.D.
MVPA Minutes
%
0
85.84
227.15
86.84
236.76
1.00
+1.17%
0.852
1.972×10−1
1
81.95
215.02
83.69
223.71
1.75
+2.13%
1.565
5.883×10−2
2
85.98
223.43
88.92
235.71
2.95
+3.43%
2.524
5.804×10−3
3
88.28
230.26
89.48
233.81
1.20
+1.35%
1.014
1.554×10−1
4
85.41
226.49
87.41
233.82
2.00
+2.34%
1.711
4.357×10−2
Appendix
Table 8: Weekly minutes of MVPA over 12 weeks, for Group 2 participants (enrolled in both the physical activity and nutrition programs) who received nudges (”Nudges” cohort) vs. those who did not receive nudges (”No Nudges” cohort). Week 0 is the week prior to the start of nudging. p -values in bold indicate statistical significance ( p≤0.05 ) in a one-tailed independent samples t-test.
Nudges Opened (Weeks 1-12)
n
Average Daily Steps (Weeks 1-12)
Total MVPA Minutes (Week 12)
Mean
S.D.
Mean
S.D.
0
54,562
2,491.48
4,296.21
56.91
200.96
1
10,867
3,212.84
3,807.07
49.22
167.17
2
5,385
3,587.72
4,150.51
48.23
178.31
≥ 3
13,950
4,444.72
4,129.64
57.00
136.61
Appendix
Table 9: Average daily steps and weekly minutes of MVPA in Week 12, versus total number of nudges opened in weeks 1 to 12, for all participants who received nudges.
Nudges Opened (Weeks 1-12)
n
Average Daily Steps (Weeks 1-12)
Total MVPA Minutes (Week 12)
Mean
S.D.
Mean
S.D.
0
4,649
2,639.05
3,897.80
45.95
138.43
1
972
3,156.61
3,951.75
49.89
163.06
2
483
3,305.84
3,995.83
54.48
158.70
≥ 3
1,332
4,229.52
3,955.43
68.63
142.93
Appendix
Table 10: Average daily steps and weekly minutes of MVPA in Week 12, versus total number of nudges opened in weeks 1 to 12, for Group 1 participants (enrolled in the physical activity program only) who received nudges.
Nudges Opened (Weeks 1-12)
n
Average Daily Steps (Weeks 1-12)
Total MVPA Minutes (Week 12)
Mean
S.D.
Mean
S.D.
0
49,913
2,477.73
4,331.23
57.93
205.79
1
9,895
3,218.36
3,792.72
49.15
167.58
2
4,902
3,615.49
4,164.80
47.61
180.13
≥ 3
12,618
4,467.44
4,147.11
55.77
135.87
Appendix
Table 11: Average daily steps and weekly minutes of MVPA in Week 12, versus total number of nudges opened in weeks 1 to 12, for Group 2 participants (enrolled in both the physical activity and nutrition programs) who received nudges.
Nudge Technique
Sent
Opened
Opened / Sent
Useful
Useful / Opened
Not Useful
Not Useful / Opened
Framing
333,614
46,413
13.9%
5,553
12.0%
759
1.6%
Gamification
218,428
28,816
13.2%
3,322
11.5%
564
2.0%
Reminder
339,078
38,459
11.3%
4,443
11.6%
804
2.1%
Social Influence
231,912
33,000
14.2%
3,793
11.5%
663
2.0%
Total
1,123,032
146,688
13.1%
17,111
11.7%
2,790
1.9%
Appendix
Table 12: Nudge engagement: nudges sent, opened, rated useful or not useful for all nudges and by nudge technique.
Nudge Technique
Sent
Opened
Opened / Sent
Useful
Useful / Opened
Not Useful
Not Useful / Opened
Framing
157,490
12,741
8.1%
1,655
13.0%
240
1.9%
Gamification
137,652
12,811
9.3%
1,506
11.8%
226
1.8%
Reminder
236,098
19,092
8.1%
2,332
12.2%
407
2.1%
Social Influence
128,198
12,697
9.9%
1,632
12.9%
245
1.9%
Total
659,438
57,341
8.7%
7,125
12.4%
1,118
1.9%
Appendix
Table 13: Nudge engagement: nudges sent, opened, rated useful or not useful for steps nudges and by nudge technique.
Nudge Technique
Sent
Opened
Opened / Sent
Useful
Useful / Opened
Not Useful
Not Useful / Opened
Framing
176,124
33,672
19.1%
3,898
11.6%
519
1.5%
Gamification
80,776
16,005
19.8%
1,816
11.3%
338
2.1%
Reminder
102,980
19,367
18.8%
2,111
10.9%
397
2.0%
Social Influence
103,714
20,303
19.6%
2,161
10.6%
418
2.1%
Total
463,594
89,347
19.3%
9,986
11.2%
1,672
1.9%
Appendix
Table 14: Nudge engagement: nudges sent, opened, rated useful or not useful for mvpa nudges and by nudge technique.
Department of Statistics, The George Washington University · Department of Mathematical Sciences, The University of Texas at Dallas · Department of Behavioral and Social Sciences, Brown University