Content-based remote sensing image retrieval techniques have been introduced for bridging the gap between low-level semantics of images and high-level semantics of user queries.

The proposed Intelligent Geo-Search System (IGSS) is a knowledge aware, spectral oriented  retrieval methodology. Knowledge about geographic objects and processes is formalized as an ontology. A reference spectral library is built consisting of spectral signatures. Tags are assigned to images using an endmember extraction algorithm and a labeling algorithm. Indexes such as (NDVI, NDMI, NBRI) and additional statistics for each index are stored along with the tags. In that way, queries can be formulated that enable both geographic entities detection (e.g. burned areas and forest type identification) and phenomena quantification (e.g. increased risk for forest fire), enabling more robust  domain oriented question answering. 

GEO Premium

Access our ENTIRE content with a yearly subscription. ONLY 8$/month (billed annually)

Get 1 Year Access Now

Introduction

Technological advances in remote sensing increased the availability of satellite images with different spatiotemporal and spectral characteristics. The problem of obtaining data has been transformed into the difficulty of retrieving the most appropriate data for each user's needs. The biggest challenge is to bridge the gap between the low-level semantics (detectable, quantifiable features) of the images and the semantic information in them (Sethi et al., 2001) with the view to designing intelligent geo-search systems. 

This bottleneck between low-level semantics of images and high-level semantics of user queries has already been acknowledged in the literature. Under the general term of content-based image retrieval a wealth of approaches has been introduced. We point the reader to Liu et al., (2007), Rani and Reddy, (2012) and Zhang et al., (2012) for a review and comparison of approaches via the prism of computer science. 

Remote sensing images  have special characteristics and therefore more specialized methodologies are needed (Xiran et al., 2012). Related work spans from the use of different data i.e. Veganzones et al., (2008) focus on techniques for hyperspectral remote sensing images while in the EU-funded project TELEIOS1, features are extracted from TerraSAR-X images and accompanied with image metadata and GIS data in order to unfold their semantics (Dumitru et al., 2011). The methodology in Maheshwary and Srivastava, (2009) is applied on multispectral images to different image processing and querying techniques. 

Segment oriented techniques for relating low-level features of images and ontological concepts can be seen in Ruan et al., (2006), Li and Bretschneider, (2007), Liu et al., (2012) and Wang et al., (2013), while  in Datcu et al., (2003), Li and Narayanan, (2004) and Aksoy et al., (2005) the labeling process is applied to  pixels. 

As far as the querying process is concerned, one research direction follows similarity metrics according to which a query image is compared to the image databased i.e. in Maheshwary and Srivastava, (2009) color and texture features are used for calculating the similarity between the query image and the images in the database. In contrast, in Ruan et al., (2006), Zhu et al., (2012) and Kaur and Jyoti, (2013) textual queries are posed to the system. 

In the current work we designed a methodology for a semantically aware remote sensing image retrieval systems, targeted at expert users and certain type of satellite data. Having the apriori knowledge of the data constraints, the expertise of the users (i.e. that they won't try to search for a car in a Landsat image [900 m2 pixel area]), and the domain knowledge to be formalized, the complexity reduction, the increase of performance and the accuracy of results can be achieved. The main objective was to build a data and domain specific search system. Search was based on tags and spectral indexes, which composed the metadata of each image within the system. 

The developed Intelligent Geo-Search System (IGSS) consists of 3 components: (a) knowledge formalization (ontology design and tag formulation), (b) Spectral Library (SL) building and (c) remote sensing image metadata formation. In the first phase, knowledge about the geographic entities, processes and indexes was formalized as an ontology. In the second phase, a spectral library was built consisting of the signatures of the objects contained within the ontology. The third step was the tagging of the remote sensing images using an endmember extraction algorithm (Nascimento and Dias, 2005) and a labeling algorithm (Sykas and Karathanassi, 2012). During this step indexes are calculated (NDVI, NDMI, NBRI). Statistics for each index (minimum and maximum values, mean, standard deviation and distribution type) are stored along with the tags.

GEO Premium

Access our ENTIRE content with a yearly subscription. ONLY 8$/month (billed annually)

Get 1 Year Access Now

Knowledge formulation

One important component of the system is its knowledge base. This, refers to the domain concepts about geographic objects that are present in remote sensing images. This knowledge is formalized in the IGSS ontology. We see ontology as a specification of a conceptualization (Gruber, 1993) able to explicitly constraint the meaning of concepts within one domain eliminating semantic interoperability problems (Bishr, 1998). The ontological commitments (Gruber, 1995) that were made are related to the spatial and spectral characteristics of the data (Landsat 7 ETM images). As our methodology is wholly spectral oriented, this influences also the concepts of the ontology which are chosen so as to bridge the gap between the reference SL and the high-level concepts of the users. Thus the IGSS ontology is not introduced as a universal remote sensing ontology but as a domain ontology oriented to the specific purpose.

Reference Spectral Library Building

The goal of this module is the collection, organization and creation of a reference SL for satellite images. The reference SL is built using a set of RSIs for which ground truth data is available. First, endmembers are extracted from the RSIs, using the appropriate pre-processing and processing algorithms such as a) radiometric corrections, b) georeference, c) Virtual Dimensionality (VD) determination (Chang and Du, 2004), d) endmember extraction (Nascimento and Dias, 2005). The extracted endmembers are manually labeled, based on the ontology and inserted in the reference SL. The number of entries in the reference SL depends on the level of information of the ground truth data and the spectral and spatial resolution of the sensor. The endmembers are extracted based on the prevailing material of the geographic object. The relation between concepts of the lowest level of the IGSS ontology and the spectral signatures is 1-1.

Metadata Assignment

The metadata assignment phase refers to the process in which meaningful information is attached to the low-level semantics of an RSI. Each RSI, that is contained in the IGSS, is enriched with information bridging the gap between 'what is in an image' (Camara et al., 2001) and high-level concepts in user queries. The information that is attached to each RSI includes the following: labels statistics per index for the whole image statistics per index for each label weights The “labels” correspond to concepts such as vegetation, water, soil, etc for each matched extracted endmember with the reference SL. The “statistics per index for the whole image” include the minimum, maximum, mean and standard deviation values from the NDVI, NDMI and NBRI for the whole image while the “statistics per index for each label” refer to the above mentioned values restricted to each label. The “weights” refer to the ratio between the number of pixels for each label divided by the total number of pixels of the image. For each label one “weight” is calculated.

Written by

I'm a Remote Sensing and a Surveying Engineer. I received my degree from NTUA in 2010, where I also received my Ph.D. in hyperspectral remote sensing in 2016. From graduation in 2010, my career started as a Researcher Associate and Teaching Associate in the Laboratory of Remote Sensing of NTUA. From that time I also worked at several private companies as a Remote Sensing Expert and Geospatial Analyst. From the beginning of 2015 I was positioned as Senior Earth Observation Expert. During these years, I have participated in more than 20 funded European Commission and European Space Agency projects, have over 16 peer reviewed scientific publications in the field of Remote Sensing, and have an international patent in hyperspectral data compression.My main research and professional interests are in the optical remote sensing area, where I specialize in data (images, point measurements) processing and algorithm design and development. Some of the software tools that I operate to accomplish my research and business dreams are SNAP, ENVI, IDL, QGIS, ERDAS Imagine, ArcGIS, and Python. I have been working with these tools since 2008.

Dimitris Sykas

Remote Sensing Expert

Dimitris Sykas