An approximate dynamic programming approach for the empty container allocation problem

The objective of this study is to demonstrate the successful application of an approximate dynamic programming approach in deriving effective operational strategies for the relocation of empty containers in the containerized sea-cargo industry. A dynamic stochastic model for a simple two-ports two-voyages (TPTV) system is proposed first to demonstrate the effectiveness of the approximate optimal solution obtained through a simulation based approach known as the temporal difference (TD) learning for average cost minimization. An exact optimal solution can be obtained for this simple TPTV model. Approximate optimal results from the TPTV model utilizing a linear approximation architecture under the TD framework can then be compared to this exact solution. The results were found comparable and showed promising improvements over an existing commonly used heuristics. The modeling and solution approach can be extended to a realistic multiple-ports multiple-voyages (MPMV) system. Some results for the MPMV case are shown.