Learning features from data

Data-driven approaches are becoming indispensible tools for understanding and predicting physical problems. I use them heavily for interpretation of physical features in remote sensing data.

Any measurement that does not involve direct physical contact between the sensing device and the sensed object is a remote sensing measurement. I work primarily with remote sensing images from satellite data, ships etc. that sense dynamic phenomena, such as movement of sea ice, debris, and surface temperatures. These images primarily are of measurements  in the visible, long-wave and microwave parts of of the electromagnetic spectrum.

There are increasing volumes of these images. This, in tandem with computing resources and new methods in deep learning, allow us the extract rich information from these images. For example, we are currently (with Prof Zhao Pan) develping a sensor suite to obtain images from ships for indentification of hazards using deep learning. I am primarily interested in data-driven methods that incorporate physics to better understand and predict  the system.

fig ship

Source: Corwin MacMillian

Small object detection

Small objects, such as openings in the ice cover, or people in terrain, can be difficult to detect. However this is important for hazard detection as well as site monitoring. We have been working on ways to detecty polynyas (small openings in ice cover) using high resolution imagery and object detection networks.

fig2

Physical information in imagery using deep learning

Sea ice is a dynamic feature that is critical to global climate. It modulates energy and momentum transfer between the ocean and the atmosphere. We can visualize it very clearly using radar images, which are freely available these days. Extracting important dynamic features from these images is difficult because their signatures vary with environmental conditions and imaging geometry. We increasingly use deep learning to extract these features and ideally we can bring in physical models as well. These features are critical for heat and momentum transfer. 

multiscale2

(SAR) images: Copyright MacDONALD DETTWILER and ASSOCIATES LTD (2010)-All Rights Reserved

 

The challenge: Identification and modelling of discrete floes

multiscale floe

       deep learning model

floes

 

 

 

  • floe with background