Toward designing intelligent PDEs for computer vision: An optimal control approach

Many computer vision and image processing problems can be posed as solving partial differential equations (PDEs). However, designing a PDE system usually requires high mathematical skills and good insight into the problems. In this paper, we consider designing PDEs for various problems arising in computer vision and image processing in a lazy manner: learning PDEs from training data via an optimal control approach. We first propose a general intelligent PDE system which holds the basic translational and rotational invariance rule for most vision problems. By introducing a PDE-constrained optimal control framework, it is possible to use the training data resulting from multiple ways (ground truth, results from other methods, and manual results from humans) to learn PDEs for different computer vision tasks. The proposed optimal control based training framework aims at learning a PDE-based regressor to approximate the unknown (and usually nonlinear) mapping of different vision tasks. The experimental results show that the learnt PDEs can solve different vision problems reasonably well. In particular, we can obtain PDEs not only for problems that traditional PDEs work well but also for problems that PDE-based methods have never been tried before, due to the difficulty in describing those problems in a mathematical way.

► We introduce a PDE-based optimal control framework for PDEs design. ► Our intelligent PDE system provides a new way to learn PDEs for vision tasks. ► We propose a general optimal control framework for training the PDE system. ► We extend our framework for learning a system of PDEs for vector-valued images.