Optimal policies of M(t)/M/c/c queues with two different levels of servers

This paper deals with optimal control points of M(t)/M/c/c queues with periodic arrival rates and two levels of the number of servers. We use the results of this model to build a Markov decision process (MDP). The problem arose from a case study in the Kelowna General Hospital (KGH). The KGH uses surge beds when the emergency room is overcrowded which results in having two levels for the number of the beds. The objective is to minimize a cost function. The findings of this work are not limited to the healthcare; They may be used in any stochastic system with fluctuation in arrival rates and/or two levels of the number of servers, i.e., call centers, transportation, and internet services. We model the situation and define a cost function which needs to be minimized. In order to find the cost function we need transient solutions of the M(t)/M/c/c queue. We modify the fourth-order Runge-Kutta to calculate the transient solutions and we obtain better solutions than the existing Runge-Kutta method. We show that the periodic variation of arrival rates makes the control policies time-dependent and periodic. We also study how fast the policies converge to a periodic pattern and obtain a criterion for independence of policies in two sequential cycles.