A 3D convolutional neural network method for land cover classification using LiDAR and multi-temporal Landsat imagery

Terrestrial landscape has complex three-dimensional (3D) features that are difficult to extract using traditional methods based on 2D representations. These methods often relegate such features to raster or metric-based (two-dimensional) representations based on Digital Surface Models (DSM) or Digital Elevation Models (DEM), and thus are not suitable for resolving morphological and intensity features for fine-scale land cover mapping. Small-footprint LiDAR provides an ideal way for capturing these 3D features. This research develops a novel method of integrating airborne LiDAR derived features and multi-temporal Landsat images to classify land cover types. We tested our approach in Williamson County, Illinois, which has diverse and mixed landscape features. Specifically, our method applied a 3D convolutional neural network (CNN) approach to extract features from LiDAR point clouds by (1) creating an occupancy grid, an intensity grid at 1-meter resolution, and then (2) normalizing and incorporating data into the 3D CNN. The extracted features (e.g., morphological and intensity features) from the 3D CNN were finally combined with multi-temporal spectral data to enhance the performance of land cover classification based on a Support Vector Machine classifier. Visual interpretation from both hyper-resolution photos and point clouds was used for training and preparation of testing data. The classification results show that our method outperforms a traditional method by 2.65% (from 81.52% to 84.17%) when solely using LiDAR and 2.19% (from 90.20% to 92.57%) when combining all available imageries. We demonstrate that our method can effectively extract LiDAR features and improve fine-scale land cover mapping through fusion of complementary types of remote sensing data.