← Back to Volume 13

Hierarchical Multi-Agent Deep Reinforcement Learning with Guided Search for U-Shaped Seru Production Scheduling

Li Zhang, Xiuli Wang
pp.167-180 DOI: 10.62051/ztx8rf65 August 13, 2026
Download PDF
Abstract: This paper investigates an integrated scheduling problem in U-shaped seru production systems under a static and deterministic order environment. The problem jointly determines order-to-seru assignment, intra-seru order sequencing, and multi-skilled worker allocation, with the objective of minimizing the makespan. Compared with conventional parallel-machine scheduling, U-shaped seru scheduling involves stronger decision coupling because workload distribution, learning effects, sequence-dependent setup times, and worker configuration jointly determine seru completion times. To characterize these features, a mixed-integer nonlinear programming model is formulated by incorporating the DeJong learning effect, sequence-dependent setup times, worker allocation constraints, and sequencing feasibility requirements. The scheduling problem is then reformulated as a single-step combinatorial Markov decision process. Based on this formulation, a hierarchical Multi-Agent Deep Deterministic Policy Gradient algorithm with Guided Search, termed MADDPG-GuidedSearch, is proposed. The framework consists of a global coordination agent for order assignment and multiple seru execution agents for intra-seru sequencing and worker allocation. Graph attention-based state encoding, centralized training with decentralized execution, Gumbel-Softmax relaxation, decoding repair, local search, and multi-candidate Guided Search are integrated to improve feasibility, search robustness, and solution quality. Computational experiments show that MADDPG-GuidedSearch-50 achieves a slightly lower average makespan than a strong genetic algorithm baseline while requiring less online solution time after offline training. Sensitivity and ablation analyses further indicate that Guided Search and Repair/Local Search are the main contributors to performance improvement.
Keywords: U-shaped seru; production scheduling; multi-agent deep reinforcement learning; MADDPG; guided search; makespan.
APA Citation: Li Zhang, Xiuli Wang (2026). Hierarchical Multi-Agent Deep Reinforcement Learning with Guided Search for U-Shaped Seru Production Scheduling. Transactions on Computer Science and Intelligent Systems Research, 13(1), 167-180. https://doi.org/10.62051/ztx8rf65

References

  1. Y. Fujita, K. Izui, S. Nishiwaki, Z. Zhang, Y. Yin, Production planning method for seru production systems under demand uncertainty, Computers & Industrial Engineering, 163 (2022) 107856.
  2. Y. Jiang, Z. Zhang, X. Gong, Y. Yin, An exact solution method for solving seru scheduling problems with past-sequence-dependent setup time and learning effect, Computers & Industrial Engineering, 158 (2021) 107354.
  3. Z. Zhang, X.L. Song, H.J. Huang, X. Zhou, Y. Yin, Logic-based Benders decomposition method for the seru scheduling problem with sequence-dependent setup time and DeJong's learning effect, European Journal of Operational Research, 297 (2022) 866–877.
  4. B. Li, Y. Wu, Integrated optimization of worker assignment, batch splitting and scheduling for a hybrid assembly line-seru production system, Computers & Industrial Engineering, 194 (2024) 110399.
  5. D. Li, H. Jin, Y. Zhang, Dynamic worker allocation in Seru production systems with actor–critic and pointer networks, European Journal of Operational Research, 324 (2025) 62–74.
  6. Z. Zhang, X. Gong, X. Song, Y. Yin, B. Lev, J. Chen, A column generation-based exact solution method for seru scheduling problems, Omega, 108 (2022) 102581.
  7. Y. Pu, F. Li, S. Rahimifard, Multi-Agent Reinforcement Learning for Job Shop Scheduling in Dynamic Environments, Sustainability, 16 (2024) 3234.
  8. H. Wang, W. Lin, T. Peng, Q. Xiao, R. Tang, Multi-agent deep reinforcement learning-based approach for dynamic flexible assembly job shop scheduling with uncertain processing and transport times, Expert Systems with Applications, 270 (2025) 126441, doi: 10.1016/j.eswa.2025.126441.
  9. J. Jang, D. Klabjan, H. Liu, N.S. Patel, X. Li, B. Ananthanarayanan, H. Dauod, T.-H. Juang, Scalable multi-agent reinforcement learning for factory-wide dynamic scheduling in semiconductor manufacturing, Engineering Applications of Artificial Intelligence, 161 (2025) 112168, doi: 10.1016/j.engappai.2025.112168.
  10. Y. Bengio, A. Lodi, A. Prouvost, Machine learning for combinatorial optimization: A methodological tour d'horizon, European Journal of Operational Research, 290 (2021) 405–421.