Sharing is caring: data sharing in multi-agent supply chains
Authors: Wan Wang, Haiyan Wang, Adam Sobey
Organizations: School of Transportation and Logistics Engineering, Wuhan University of Technology, No.1178 Heping Avenue Wuchang District, Wuhan, 430063, Hubei, China · Maritime Engineering, University of Southampton, Burgess Rd, Southampton, SO17 1BJ, UK · Sustainability Mission, The Alan Turing Institute, The British Library, London, NW1 2DB, UK
Modern supply networks are complex interconnected systems. Multi-agent models are increasingly explored to optimise their performance. Most research assumes agents will have full observability of the system by having a single policy represent the agents, which seems unrealistic as this requires companies to share their data. The alternative is to develop a Hidden-Markov Process with separate policies, making the problem challenging to solve. In this paper, we propose a multi-agent system where the factory agent can share information downstream, increasing the observability of the environment. It can choose to share no information, lie, tell the truth or combine these in a mixed strategy. The results show that data sharing can boost the performance, especially when combined with a cooperative reward shaping. In the high demand scenario there is limited ability to change the strategy and therefore no data sharing approach benefits both agents. However, lying benefits the factory enough for an overall system improvement, although only by a relatively small amount compared to the overall reward. In the low demand scenario, the most successful data sharing is telling the truth which benefits all actors significantly.
Figures & tables
Figure 1: Flow of data sharing through the supply chain environment.
Hyperparameters sets
Values
max_seq_len
20
conv_activation
relu
fcnet_activation
tanh
fcnet_hiddens
[256,256]
Discount factor (gamma)
0.99
Learning rate (lr)
0.001
Table 1: Optimal hyperparameter settings for heterogeneous SAC.
Notation
Explanation
Retailer (i=1)
Factory (i=2)
Spi
Unit Sales Price
6
6
Qi
Order Quantity
[0,20]
[0,20]
ci
Unit Ordering Cost
Sp2
0.2
Hci
Unit Inventory Holding Cost
0.2
0.2
Ii
Initial Inventory Level
10
10
Ci
Inventory Capacity
19
59
Table 2: Environment parameters in the two-echelon supply chain.
Reward
NoComms
Communication
Truth
Lying
Mixed
Factory
1695.74
1,693.40
1710.62
1671.95
Retailer
-373.07
-376.01
-375.94
-361.12
Global
1,322.67
1,317.39
1,334.69
1310.82
Table 3: Agent baseline rewards in the high demand environment for the four forms of communication under simulated ten times. Higher scores indicate superior agent performance.
Reward
NoComms
Communication
Truth
Lying
Mixed
Factory
1653.04
1635.50
1718.62
1650.64
Retailer
-390.74
-440.39
-406.74
-362.95
Global
1262.3
1195.11
1311.88
1287.69
Table 4: The agent receives collaborative rewards for the four forms of communication in the reward shaping high demand environment under ten simulations. Higher scores indicate superior agent performance.
Reward
NoComms
Communication
Truth
Lying
Mixed
Factory
42.62
71.82
-5.85
46.27
Retailer
-94.67
-190.75
-139.17
-174.39
Global
-52.04
-118.92
-145.02
-127.98
Table 5: Agent Baseline rewards in the low demand environment for the four forms of communication under 10 times simulation. Higher scores indicate superior agent performance.
Reward
NoComms
Communication
Truth
Lying
mixed
Factory
4.67
109.86
-9.6
33.68
Retailer
3.33
-87.53
-135.17
-88.99
Global
8
22.32
-144.77
-55.32
Table 6: Ten simulated experiments were conducted within a reward-shaping environment to assess the collaborative reward mechanism among agents, specifically addressing low demand across four communication formats. Higher scores indicate superior agent performance.
Figure 2: Comparison of factory rewards for the various communication approaches to the baseline environment with No Communication in the high demand environment.
Figure 3: Comparison of factory rewards for the various communication approaches to the baseline environment with No Communication in the low demand environment.
Figure 4: Comparison of retailer rewards for the various communication approaches to the baseline environment with no communication in the high demand environment.
Figure 5: Comparison of retailer rewards to the baseline environment with no communication in the low demand environment.
Figure 6: A comparison of inventory levels under the baseline and collaborative scenarios for the high demand.
Figure 7: A comparison of inventory levels under the baseline and collaborative scenarios for the low demand.
Trust
NoComms
Lying
Total Reward
High Demand
Baseline
19%
39.7%
41.3%
1310.82
Collab
25.5%
38.62%
36%
1287.69
Low Demand
Baseline
29.2%
31%
40.16%
-127.98
Collab
30.8%
44.04%
25.16%
-55.32
Table 7: Mixture of communication styles for the mixed environments.