Drone HDR Infrared Video Coding via Aerial Map Prediction
In recent years more and more video data are acquired by infrared sensors supporting high dynamic range (HDR), where each pixel is represented by an integer value with a bit depth higher than 8 bits. Such sensors could be mounted on unmanned airborne vehicles (UAV) and used in many applications, such as inspections of district heating or other energy systems, search and rescue operations, night video surveillance and so on. Due to channel capacity limitations such HDR videos should be compressed before transmission (or storage). It can be performed by well-known image coding standards, such as PNG, JPEG2000, JPEG-XT or other methods, supporting high bit depth formats. However, these standards have a limited coding efficiency, because they do not take temporal redundancy between neighbor frames into account. In contrast, the latest video coding standard H.265/HEVC supports HDR video formats and provides a high coding performance due to motion estimation and compensation.
However, the UAV camera motion has several features which are not reflected well by the motion estimation model utilizing in H.265/HEVC standard:
To address these features in [2,3] two similar approaches utilizing historical data have been proposed. First, before flying a huge video set covering wide area and long time (historical data) are prepared and stored at both the encoder and decoder. Then this data is used to improve the HEVC prediction performance. This approach allows to reduce the bit rate by approximately 30%. However, it requires a huge storage capacity and cannot give any benefits if the historical data for a given area is not collected. In this paper, we propose a novel HDR infrared video coding algorithm based on aerial map prediction which operates without historical data. In this approach we accumulate input frames in a buffer and use them to build an aerial map. The map is compressed by H.265/HEVC Intra and included into the overall bit stream. Then we apply global motion estimation to extract the most similar frame from the reconstructed aerial map and use it as an extra reference frame. The motion model used for the extraction reflects camera rotation which helps to improve the overall coding performance. Experimental results show that when the aerial map bit size is not taken into account (assuming that the UAV is flying many times above the same area) the proposed approach provides 20-60% bit rate savings comparing to the H.265/HEVC. If the map bit stream is included in the overall bit stream then for a test video with camera rotation the proposed algorithm provides 3-35% bit rate savings comparing to the H.265/HEVC.
For more detailed information please see .
2. Performance comparison
Experimental results were obtained for two test video sequences captured by Drone Systems ApS by a Flir Tau2 infrared camera with frame resolution 640x512, 16 bits per pixel. The first test video includes 101 frames and corresponds to a general case, when a drone is flying in one direction, rotating and returning back with a frame overlapping, approximately from 10-50%, i.e., the motion model assuming to be well reflected by the H.265/HEVC motion estimation. The second test video includes 70 frames and corresponds to the case when a drone is rotating from 0 to approximately 360 degrees, i.e., the motion model is too complex for the block-based motion estimation used in HEVC. The following figure shows corresponding tone mapped aerial maps for these videos. At the first stage, all frames were aligned using three parameters: rotation angle, vertical and horizontal displacements. Then the resulting pixel of an aerial map was computed as a weighting combination of the corresponding pixels in overlapped frames. The weights values for each frame were assigned relatively to the distance between a pixel and the center of the frame: smaller distance means a higher value.
The following figure shows rate-distortion performance of the HEVC codec with and without the proposed AMP.
HEVC with aerial map prediction, Version 1.0 [download]
If you plan to use the software, please also refer .
 E.Belyaev, S.Forchhammer, 'Drone HDR Infrared video coding via aerial map prediction', 2018 IEEE International Conference on Image Processing, 2018.