Optimal energy allocation for linear control with packet loss under energy harvesting constraints

This paper studies a closed loop linear control system over a lossy communication link. A sensor computes a state estimate of the observed discrete-time system and sends it (in the form of packetized transmission) to the controller in the receiver block over a randomly time-varying (fading) packet dropping link. The receiver sends an ACK/NACK packet to the transmitter over an acknowledgement channel which might also be prone to packet loss. It is assumed that the energy used in packet transmission depletes a battery of limited capacity at the sensor, but is also replenished by an energy harvester which has access to a source of everlasting but random harvested energy. Under an assumption of finite-state Markov chain models of the energy harvesting and the fading channel gain processes, the objective is to design an optimal energy allocation policy at the transmitter and an optimal control policy at the receiver so that an average infinite horizon linear quadratic Gaussian (LQG) control cost is minimized. It is shown that in the case of perfect channel feedback a separation principle holds, the optimal LQG controller is linear and the optimal energy allocation policy at the transmitter can be obtained via solving the Bellman dynamic programming equation. A Q-learning algorithm is used to approximate the optimal energy allocation policy in case the system parameters are unknown. Numerical simulation examples show that the dynamic programming based policies outperform various simple heuristic policies, especially at higher battery capacities.