cs.AIJun 24, 2025
SaveHiMA-Ecom: A Long-Horizon Memory Benchmark for Hierarchical Multi-Agent E-commerce Assistants
Organizations: JD Retail, JD.com, Inc., Beijing, 101111, China · Faculty of Engineering and Information Technology, The University of Melbourne, Parkville, Victoria, 3082, Australia · School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing, 100049, China
Abstract
Long-horizon agentic systems become long-context systems by accumulation rather than through a single long document, since every step reassembles a prompt from role instructions, tool and sub-agent descriptions, recalled memories, dialogue history, and previous tool outputs, separately for every agent. This paper studies how such accumulated context can be trained, evaluated, and managed in hierarchical multi-agent assistants. We present HiMA-Ecom, a hierarchical multi-agent benchmark for e-commerce assistants with 22.8K instances, of which 17.7K carry memory entries in context, pairing agent-specific supervised fine-tuning data with system-level supervision for end-to-end multi-agent reinforcement learning. Building upon it, HiMA-R1 keeps the context budget of a trajectory under control through Variance-Reduction Group Relative Policy Optimization (VR-GRPO), which samples along an initial trajectory so that sampling cost grows additively with trajectory length and updates the agents with the largest reward variance, together with an efficiency reward on trajectory length and a memory evolution mechanism that repurposes GRPO rewards as cost-free signals for what stays in context. Agents built upon 3B and 7B open-source models match DeepSeek-R1 and surpass DeepSeek-V3 by 6.0 points on average, while reward-driven memory reduces the update steps needed for convergence by roughly 55%.
Figures & tables
Figure 1: Comparison of training paradigms for hierarchical LLM-based multi-agent systems. (a) Master-Only Training, (b) Parameter-Sharing Training, (c) Separate Training with fixed planning, and (d) Joint Multi-Agent RL Training with dynamic planning. The first three adopt static invocation structures, while this work focuses on the joint training paradigm with learned invocation in (d),
Figure 2: An illustrative example of the hierarchical multi-agent system under e-commerce scenarios, where each agent re-assembles its own context from role instructions, tool and sub-agent descriptions, recalled memories, dialogue history, and tool responses. Agent-level reasoning is visualized for clarity, while the dataset provides agent-specific SFT entries and joint input and output pairs for RL.
Figure 3: Examples of different agents making decisions based on dynamically recalled memories.
| Agent | Domain | Training stage | Instances | Instances with memory |
| Master | General + E-commerce | SFT | 13K | 9K |
| Question answering | General + E-commerce | SFT | 1K | 1K |
| E-commerce function call | E-commerce | SFT | 2K | 2K |
| General function call | General | SFT | 3.3K | 3.3K |
| Mathematics | General | SFT | 2.4K | 2.4K |
| Collaboration | General + E-commerce | RL | 1.1K | - |
Table 1: Overview of the HiMA-Ecom benchmark.
Figure 4: VR-GRPO for Multi-Agent Systems (MAS). (1) Initial trajectory-based Monte Carlo sampling sequentially samples agent actions along an initial trajectory, mitigating action space explosion. (2) Action reward calculation incorporates accuracy, format, and efficiency rewards based on the trajectory output. (3) Marginal benefit-driven updating prioritizes the top- agents with the highest reward variance to accelerate policy evolution.
| Model | Math | QA | E-commerce FC | General FC | Collaboration | Average |
| Claude3.5-sonnet | 98.0 | 38.0 | 1.0 | 62.5 | 2.0 | 40.3 |
| GPT - 4o mini | 47.0 | 31.0 | 0.0 | 65.5 | 0.0 | 28.7 |
| GPT - 4o | 85.0 | 35.0 | 56.0 | 83.0 | 6.0 | 53.0 |
| DeepSeek-R1 | 98.0 | 37.0 | 24.0 | 72.0 | 7.0 | 47.6 |
| DeepSeek-V3 | 96.0 | 32.0 | 10.0 | 68.0 | 2.0 | 41.6 |
| Qwen2.5-32B | 72.0 | 38.0 | 3.0 | 72.0 | 2.0 | 37.4 |
Table 2: Comparisons on the HiMA-Ecom benchmark (%). Under the open-source setting, bold is the optimal and ‘_’ is the suboptimal value. Closed-source methods ( ) are only used for reference.
| Method | RL | Think | Top- | Efficiency | Memory | Update Steps | Accuracy | Rounds |
| ✗ | ✗ | - | - | - | - | 17.2 | 5.4 | |
| ✗ | ✔ | - | - | - | - | 35.0 | 7.1 | |
| ✔ | ✔ | ✗ | ✔ | ✔ | 1380 | 40.0 | 7.4 | |
| ✔ | ✔ | ✔ | ✗ | ✔ | 1750 | 42.4 | 8.1 | |
| ✔ | ✔ | ✔ | ✔ | ✗ | 2464 | 40.0 | 7.7 | |
| ✔ | ✔ | ✔ | ✔ | ✔ | 1112 | 44.0 | 7.8 |
Table 5: Ablation study on HiMA-Ecom, covering whether to train with RL rather than SFT, to generate the think process, to update top- models rather than all, to use efficiency rewards, and to integrate memory. ‘Update Steps’ corresponds to the best-performing model.
Figure 5: HiMA-R1 with memory achieves higher reward peaks faster during training.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| Hyperparameter | Value |
| learning_rate | 5e-6 |
| max_length | 16384 |
| num_train_epochs | 5 |
| max_grad_norm | 1 |
| weight_decay | 0.01 |
| warmup_ratio | 0.03 |
Table 8: Training settings for supervised fine-tuning.
| Hyperparameter | Value |
| learning_rate | 1e-6 |
| max_length | 16384 |
| num_train_epochs | 5 |
| max_grad_norm | 1 |
| adam_epsilon | 1e-5 |
| num_groups | 5 |
Table 9: Training settings for reinforcement learning.
Figure 6: Different sampling strategies for multi-agents. Green circles indicate nodes sampling multiple actions via GRPO, while gray circles involve no additional sampling. Compared to naive sampling in (a), (b) could avoid the exponential explosion of the multi-agent joint action space.
| Method | Plan_ACC | Act_EM | Easy_F1 | Hard_F1 | F1 | No_Hallu | Average |
| In the domain | |||||||
| GPT-4o | 82.5 | 57.0 | 31.1 | 22.2 | 37.3 | 99.9 | 55.0 |
| DeepSeek-R1 | 72.3 | 45.2 | 18.7 | 18.2 | 28.5 | 100.0 | 47.1 |
| DeepSeek-V3 | 68.4 | 38.0 | 14.9 | 14.3 | 23.6 | 99.4 | 43.1 |
| Qwen2.5-32B | 72.7 | 43.7 | 15.1 | 21.0 | 28.8 | 99.3 | 46.8 |
| Qwen2.5-14B | 42.2 | 15.0 | 3.7 | 7.1 | 10.1 | 100.0 | 29.7 |
Table 10: Comparative results (%) on the ToolBench dataset. Bold represents the optimal score, and ‘_’ represents the suboptimal value. ‘HiMA-SFT-no’ means training without a thinking process.
| Case #1 of the master agent in a collaborative task | |
| User: I want to open a small personal store on [e-commerce platform A] to sell educational toys. How much deposit do I need to pay? If I open two stores, how much deposit will I need to pay in total? | |
| System | {Role} from Template #1 in Section A.8 Specialized Retrieval Tools/Memory: [{"name": "math_agent", "description": "When the user puts forward a clear request to give a solution or answer to a specific mathematical problem, the agent needs to be called. If it is only about a mathematically related topic, the agent cannot be called", "parameters": {}}, …] [{"ID": 1620, "Query": " Calculate how many times Danny folded his picnic blankets given the initial and final area.", "Plan": ["math_agent"], "Answer": "How many square feet are each blanket? ** The total area of each blanket is 88=<<88=64>>64. How many square feet are the blankets? ** Their total area is 192 square feet because 3 x 64 = <<3*64=192>>192. How many times did he fold them? ** He folded them 4 times because 192 / 48 = <<192/48=4>>4. #### 4", "Time": "2025-03-26 11:41:13", "Score": 6} …] {Problem Resolution Framework} from Template #1 in Section A.8 |
| Assistant | {"name": "rody_agent", "arguments": {"intention": ["Query the deposit required for opening a small personal store on [e-commerce platform A] to sell educational toys."]}} |
| Tool | Pass calling {"api_name": "check_shop_expenses", "parameter": {"keyword": ["Educational Toys"], "shop_body": ["Individual"], "shop_type": ["small shops"]}}, the API call was successful. Results are as follows: 1. Specific amount of margin per year: 6000 yuan. 2. Transaction service fee rate per transaction: 0.25%. 3. Technical service fee rate per transaction: 0.41%. |
| Assistant | {"name": "tool_retrievals", "arguments": {"intention": ["Calculate the total deposit required for opening two small personal stores on [e-commerce platform A] to sell educational toys, based on the queried deposit information of 6000 yuan per store."]}} |
| Tool | [{"name": "expert_agent", "description": "When users chat, greet, abuse, make jokes, discuss music/games/current politics/military/entertainment/food and other daily topics, or consult the rules of the e-commerce platform, the agent needs to be invoked ", "parameters": {}}, {"name": "math_agent", "description": "When the user puts forward a clear request to give a solution or answer to a specific mathematical problem, the agent needs to be called. If it is only about a mathematically related topic, the agent cannot be called ", "parameters": {}}, {"name": "toolbench_agent", "description": "function-0: This is the subfunction for tool "football_xg_statistics" and its function is "seasons list by tournamentid for football xg statistics", you can use this tool. The description of this function is: Football (soccer) xG statistics. We provide statistics for more than 80 leagues. There are more than 90,000 games and 2 million shots in our database. The method is used to get a list of seasons by the league ID; function-1: This is the subfunction for the tool "mailslurp_email_testing" and its function is "getbouncedrecipients for mailslurp email testing", you can use this tool. The description of this function is: Email sandbox testing API. Bounced recipients are email addresses that you have sent emails to that did not accept the email. Once a recipient is bounced, you cannot send emails to that address."}] |
Table 11: A case of the master agent in a collaborative task.
| Case #2 of the function-call agent in an e-commerce function-call task | |
| User: Hello, I found that some orders may have the label of door-to-door replacement, what information should be provided in this case? | |
| System | {Role} from Template #2 in Section A.8 Specialized Retrieval Tools/Memory: [{"name": "search_payment_method","description": "Inquire the settlement method of the user’s shop, such as B2B settlement, C2B settlement, bank card settlement, online banking wallet settlement.","parameters": {}}, {"name": "search_order_code", "description": "…", ,"parameters": {…}}, …] [{"ID": "1254", "Query": "Hello, I found that some orders may have the label of door-to-door replacement, what information should be provided in this case?", "Plan": [], "Answer": "Pass calling {’api_name’: ’search_order_code’, ’parameter’: {’order_service_mix_tag’: [’door-to-door replacement’], ’search_keys’: [’order label’]}}, the API call was successful. Results are as follows: We have retrieved the relevant order information for you.", "Time": "2025-04-29 02:20:30", "Score": 10}, …] {Problem Resolution Framework} from Template #2 in Section A.8 |
| Assistant | {"name":"search_order_code", "arguments":{"order_service_mix_tag": ["door-to-door replacement"], "search_keys": ["order label"]}} |
| Tool | Pass calling {"api_name": "search_order_code", "parameter":{"order_service_mix_tag": ["door-to-door replacement"], "search_keys": ["order label"]}}, the API call was successful. Results are as follows: We have retrieved the relevant order information for you. |
| Assistant Final | The Input issue is about retrieving information for orders with the label "door-to-door replacement." The Output is the result of calling the "search_order_code" tool with the parameters ‘{"order_service_mix_tag": ["door-to-door replacement"], "search_keys": ["order label"]}’. The Output indicates that the API call was successful and relevant order information has been retrieved. This matches the reference information in <memory>, where a similar query was resolved by calling the same tool with the same parameters. Therefore, the Output is directly derived from the reference information, and the reasoning process is consistent with the provided solution. Pass calling {"api_name": "search_order_code", "parameter": {"order_service_mix_tag": ["door-to-door replacement"], "search_keys": ["order label"]}}, the API call was successful. Results are as follows: We have retrieved the relevant order information for you. |
Table 12: A case of the function-call agent in an e-commerce task.
| Case #3 of the question-answering agent in a QA task | |
| User: Where to open the LCL service? | |
| System | {Role} from Template #4 in Section A.8 Specialized Retrieval Tools/Memory: [{"name": "tool_retrievals", "description": "Knowledge retrieval tool. Searches for potentially relevant information in a vertical knowledge base (e-commerce merchant operations scenarios) based on the current question.", "arguments": {}} {"name": "tool_summary", "description": "Intelligent Q&A tool. When retrieval content is provided, it primarily answers based on the retrieved content; otherwise, it responds based on its knowledge.", "arguments": {}}] [{"id": 4439, "model_name": "expert", "query": "Is my shop bounded with a card?", "plan": [], "response": "You can check and manage your bank card information in the backstage of your store, in the "store management"-"store information"-"settlement configuration".", "score": 1.0, "time": 7514}, …] {Problem Resolution Framework} from Template #4 in Section A.8 |
| Assistant | {"name": "tool_retrievals", "arguments": {}} |
| Tool | [General] Corporation_A Merchant Consolidation Warehouse Opening Configuration Page (see link: …) [Other] Merchant Question: Consolidation Opening Entry Merchant Assistant: 1) How to open: You can click on the left menu bar of the Merchant Center - Services - Merchant Service Center - Sales - Remote Area Logistics Service - Click to open to enter the product configuration page. Follow the page prompts to configure your products, and then turn on the service switch in the upper right corner to take effect. Direct link: … 2) Fees: For products bound to consolidation and free shipping, you only need to bear the cost of the first leg to Xi’an. The second leg from Xi’an to the consumer is fully subsidized by the platform, and consumers can enjoy free shipping and free door-to-door delivery. 3) Benefits: Free second-leg logistics for remote area shipments, free interception for second-leg logistics in remote areas, estimated doubling of remote area order volume, and exemption from store comprehensive experience scores. … |
| Assistant | {"name": "tool_summary", "arguments": {}} |
| Tool | The entry to open the consolidation service is as follows: 1. Click the left menu bar. 2. Select "Service". 3. Enter "Merchant Service Center". 4. Select "Remote Area Logistics Service" under "Selling Section". 5. Click "Open" to enter the product configuration page. 6. Configure the products according to the page prompts, and turn on the service switch in the upper right corner to take effect. Direct link: … |
Table 13: A case of the question-answering agent in a QA task.
Figure 7: An example of dynamic updating of the agent memory module.
Figure 8: The example of the QA agent in a question-answering task.
Figure 9: The example of a complete reasoning trajectory in an e-commerce task.
More in this topic
Long-Horizon Agent Evaluation
Long-Horizon Agent Evaluation measures whether AI agents can plan, act, remember, and stay reliable across extended tasks and changing environments.
See all 358 papers