Modeling and optimization control of a demand-driven, conveyor-serviced production station

• We model the demand-driven CSPS with both production and sales centers as a SMDP.
• We examine the case of random part/customer arrival, limited buffer/bank capacity.
• A model-free learning optimization algorithm is effective to search the policy.
• We can design the optimal capacities of both buffer and bank by using the model.
• Given the bank state, the emptier the buffer is, the longer the look ahead rang is.

This study investigates the look-ahead control of a conveyor-serviced production station (CSPS), viewed as a production center, which is connected to a sales center. The production station is equipped with a buffer to temporarily store the parts that will flow into the product bank of the sales center after processing. The whole two-center system is characterized by random parts arrival, random customer demand, random processing time and limited buffer or bank capacities. Thus, the decision-making on the look-ahead range of such demand-driven CSPS is subject to the constraints of production and sales levels. In this paper, we will focus on modeling the stochastic control problem and providing solutions for finding the optimal look-ahead control policy under either average- or discounted-cost criteria. We first establish a detailed semi-Markov decision process for the look-ahead control of the demand-driven CSPS by combining the vacancies of both the buffer and the bank into one state, which can be solved by policy iteration or value iteration if the system parameters are known precisely. Then, to avoid the curse of dimensionality and modeling in the numerical optimization methods, we also propose a Q-learning algorithm combined with a simulated annealing technique to derive the approximate solutions. Simulation results are finally presented to show that by our established model and proposed optimization methods the system can achieve an optimal or suboptimal look-ahead control policy once the capacities of both the buffer and the bank are designed appropriately.