Skip to content

Commit

Permalink
* Add support for GlobCover 2009
Browse files Browse the repository at this point in the history
* Clean code to avoid nasty exception with incorrect geometries
  • Loading branch information
jjrom committed Sep 1, 2016
1 parent a015e96 commit fb9f923
Show file tree
Hide file tree
Showing 11 changed files with 793 additions and 263 deletions.
60 changes: 37 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ However there is no guaranty of success and unwanted result may occured !

## Installation

We suppose that :
We suppose that :

* $ITAG_HOME is the directory containing this file
* $ITAG_DATA is the directory containing the datasources file (see below)
Expand All @@ -46,33 +46,33 @@ First create the $ITAG_DATA directory
#### General data

Retrieve coastlines from [Natural Earth](http://www.naturalearthdata.com/downloads/10m-cultural-vectors/10m-admin-0-countries/)

cd $ITAG_DATA
wget http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/physical/ne_10m_coastline.zip
unzip ne_10m_coastline.zip

#### Political data

Retrieve countries from [Natural Earth](http://www.naturalearthdata.com/downloads/10m-cultural-vectors/10m-admin-0-countries/)

cd $ITAG_DATA
wget http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/cultural/ne_10m_admin_0_countries.zip
unzip ne_10m_admin_0_countries.zip

Retrieve World Administrative Level 1 data from [Natural Earth](http://www.naturalearthdata.com/downloads/10m-cultural-vectors/10m-admin-1-states-provinces/)

cd $ITAG_DATA
wget http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/cultural/ne_10m_admin_1_states_provinces.zip
unzip ne_10m_admin_1_states_provinces.zip

Retrieve toponyms from [geonames](http://geonames.org)

cd $ITAG_DATA
wget http://download.geonames.org/export/dump/allCountries.zip
wget http://download.geonames.org/export/dump/alternateNames.zip
unzip allCountries.zip
unzip alternateNames.zip

#### Geological data

Retrieve geophysical data from [Mapping Tectonic Hot Spots](http://www.colorado.edu/geography/foote/maps/assign/hotspots/hotspots.html)
Expand All @@ -82,70 +82,84 @@ Retrieve geophysical data from [Mapping Tectonic Hot Spots](http://www.colorado.
unzip hotspots.zip

Retrieve glaciers from [Natural Earth](http://www.naturalearthdata.com/downloads/10m-physical-vectors/10m-glaciated-areas/)

cd $ITAG_DATA
wget http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/physical/ne_10m_glaciated_areas.zip
unzip ne_10m_glaciated_areas.zip

#### Hydrological data

Retrieve rivers data from [Natural Earth](http://www.naturalearthdata.com/downloads/10m-physical-vectors/10m-rivers-lake-centerlines/)

cd $ITAG_DATA
wget http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/physical/ne_10m_rivers_lake_centerlines.zip
unzip ne_10m_rivers_lake_centerlines.zip

#### Other data (i.e. marine areas, mountains area, etc.)

# Marine areas
cd $ITAG_DATA
wget http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/physical/ne_10m_geography_marine_polys.zip
unzip ne_10m_geography_marine_polys.zip

### Install database

# Note : "password" must be the same as
# Note : "password" must be the same as
# the value of 'password' parameter in $ITAG_HOME/include/config.php

$ITAG_HOME/_install/installDB.sh -F -d <path_to_postgis_directory> -p password

### Populate database

**Note** : If you are using Fedora, Red Hat Enterprise Linux, CentOS, scientific Linux, or one of the other
distros that enable SELinux by default you should run the following commands as root :

setenforce 0

Run the following commands
Run the following commands

# General datasources
$ITAG_HOME/_install/installDatasources.sh -F -D $ITAG_DATA

# Gazetteer
$ITAG_HOME/_install/installGazetteerDB.sh -F -D $ITAG_DATA

# Wikipedia
# This step is optional and can only be performed if you have the geolocated wikipedia data (which probably you don't have :)
# In case of, these are the steps to follow in order to install this database within iTag
#
# Put the geolocated wikipedia data in $ITAG_DATA/wikipedia directory, then run the command
#
$ITAG_HOME/_install/installWikipediaDB.sh -D $ITAG_DATA/wikipedia

### Install landcover database

Install one of GlobCover2009 or GLC2000. GlobCover2009 is more recent and 300 meters resolution. GLC2000 is older and 1 kilometer resolution

**Note** Process GlobCover2009 would take more time and take more disk space

#### GLC2000
Download the world glc2000 GeoTIFF file from ["Global Land Cover 2000" - global product](http://forobs.jrc.ec.europa.eu/products/glc2000/products.php)

Then run the following :

**Attention** : this PHP script gets postgres superuser password as command line argument, change password after iTag installation !
**Warning** : this PHP script gets postgres superuser password as command line argument, change password after iTag installation !

$ITAG_HOME/_install/computeLandCover.php -p postgres_user_pass -I path_to_glc2000_tif_image

**Tip** : maybe you have to indicate the location of gdal tools (check -T and -P swich)

**Note** : depending on your server performance, the landcover computation can take a long time (more than two hours)

#### GlobCover2009
Download the world GlobCover 2009 GeoTIFF file from ["European Space Agency GlobCover Portal"](http://due.esrin.esa.int/files/GLOBCOVER_L4_200901_200912_V2.3.color.tif)

Then run the following :

**Warning** : this PHP script gets postgres superuser password as command line argument, change password after iTag installation !

$ITAG_HOME/_install/computeGlobCover2009.php -p postgres_user_pass -f GLOBCOVER_L4_200901_200912_V2.3.color.tif

### Install Gridded Population of the World database

Download most recent "Population Count Grid Future" (or "Population Count Grid") product of the whole World from [SEDAC](http://sedac.ciesin.columbia.edu/data/set/gpw-v3-population-count-future-estimates/data-download) in ASCII Grid format (*.ascii or *.asc). All four resolutions (1°, 1/2°, 1/4° and 2.5′) are needed!
Expand All @@ -170,7 +184,7 @@ Then run the following :
We suppose that $ITAG_TARGET is accessible to http://localhost/itag/ in Apache.

To tag footprint on Toulouse with geological information and all cities with a pretty GeoJSON output, open this url within you browser

http://localhost/itag/?taggers=Political&_pretty=true&footprint=POLYGON((1.350360%2043.532822,1.350360%2043.668522,1.515350%2043.668522,1.515350%2043.532822,1.350360%2043.532822))

Available parameters for Web service are :
Expand All @@ -182,10 +196,10 @@ You can check this [running instance] (http://mapshup.com/projects/itag/)
Examples :

Tag footprint on Toulouse with political and geological information and with a pretty GeoJSON output

http://mapshup.com/projects/itag/?taggers=Political,Geology&_pretty=true&footprint=POLYGON((1.350360%2043.532822,1.350360%2043.668522,1.515350%2043.668522,1.515350%2043.532822,1.350360%2043.532822))


Tag footprint intersecting France, Italy and Switzerland with political information. Hierarchical result as pretty GeoJSON output

http://mapshup.com/projects/itag/?taggers=Political&footprint=POLYGON((6.487426757812523%2045.76081241294796,6.487426757812523%2046.06798615804025,7.80578613281244%2046.06798615804025,7.80578613281244%2045.76081241294796,6.487426757812523%2045.76081241294796))
225 changes: 225 additions & 0 deletions _install/computeGlobCover2009.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,225 @@
#!/usr/bin/env php
<?php

/*
* Copyright 2013 Jérôme Gasperi
*
* Licensed under the Apache License, version 2.0 (the "License");
* You may not use this file except in compliance with the License.
* You may obtain a copy of the License at:
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/

/*
* iTag - prepare landcover database from GlobCover2009 TIF image
*/

$help = "## iTag GlobCover 2009 compute and installation\n\n Usage computeLandCover.php [options] -f <path to GlobCover TIF file>\n\n";
$help .= "OPTIONS:\n";
$help .= " -H : postgres server hostname (default localhost)\n";
$help .= " -d : iTag database name (default itag)\n";
$help .= " -s : postgresql superuser (default postgres)\n";
$help .= " -p : postgresql superuser password\n";
$help .= "\n";
$hostname = 'localhost';
$superuser = 'postgres';
$password = 'postgres';
$dbname = 'itag';
$options = getopt("f:hp:d:s:");
foreach ($options as $option => $value) {
if ($option === "h") {
echo $help;
exit;
}
if ($option === "f") {
$globcover = $value;
}
if ($option === "H") {
$hostname = $value;
}
if ($option === "p") {
$password = $value;
}
if ($option === "s") {
$superuser = $value;
}
if ($option === "d") {
$dbname = $value;
}
}

$gdaltranslate = exec('which gdal_translate');
$gdalpolygonize = exec('which gdal_polygonize.py');
if (!isset($globcover)) {
echo $help;
exit;
}

// Get db connection
try {
$dbh = pg_connect("host=" . $hostname . " dbname=" . $dbname . " user=" . $superuser . " password=" . $password . " port=5432");
} catch (Exception $e) {
echo ' ERROR : no connection to database';
exit;
}
pg_set_client_encoding($dbh, "UTF8");

$config = array(
'translate' => $gdaltranslate,
'polygonize' => $gdalpolygonize,
'image' => $globcover,
'dbname' => $dbname,
'host' => $hostname,
'user' => $superuser,
'password' => $password
);

// Produce the grid
$gridSize = 2;
// Longitude from -180 to 180
for ($lon = -180; $lon <= 180; $lon = $lon + $gridSize) {
// Latitude from -60 to 90
for ($lat = -60; $lat <= 90; $lat = $lat + $gridSize) {
$lat2 = $lat + $gridSize;
$lon2 = $lon + $gridSize;
$str = $lon . " " . $lat . "," . $lon . " " . $lat2 . "," . $lon2 . " " . $lat2 . "," . $lon2 . " " . $lat . "," . $lon . " " . $lat;
polygonize("POLYGON((" . $str . "))", $dbh, $config, $config);
}
}
echo ' --> done !';

/**
* Return crop origin from Bounding Box for GLC2000 raster
*
* @param <Array> $bbox : ['ulx', 'uly', 'lrx', 'lry']
* @return <Array> ['x','y','xsize','ysize']
*/
function cropOriginGLC2000($bbox) {

// GLC2000 full raster info
$dx = 0.0089285714;
$dy = -0.0089285714;
$lon0 = -180.0;
$lat0 = 89.99107138060005;

return cropOrigin($bbox, $lon0, $lat0, $dx, $dy);
}

/**
* Return crop origin from Bounding Box for GlobCover raster
*
* @param <Array> $bbox : ['ulx', 'uly', 'lrx', 'lry']
* @return <Array> ['x','y','xsize','ysize']
*/
function cropOriginGlobCover($bbox) {

// GlobCover full raster info
$dx = 0.002777777777778;
$dy = -0.002777777777778;
$lon0 = -180.001388888888897;
$lat0 = 90.001388888888883;

return cropOrigin($bbox, $lon0, $lat0, $dx, $dy);
}

/**
* Return crop origin from Bounding Box
*
* @param <Array> $bbox : ['ulx', 'uly', 'lrx', 'lry']
* @param <Float> $lon0
* @param <Float> $lat0
* @param <Float> $dx
* @param <Float> $dy
* @return <Array> ['x','y','xsize','ysize']
*/
function cropOrigin($bbox, $lon0, $lat0, $dx, $dy) {
$x = floor(($bbox['ulx'] - $lon0) / $dx);
$y = floor(($bbox['uly'] - $lat0) / $dy);
$xsize = ceil(($bbox['lrx'] - $bbox['ulx']) / $dx);
$ysize = ceil(($bbox['lry'] - $bbox['uly']) / $dy);
return array('x' => $x, 'y' => $y, 'xsize' => $xsize, 'ysize' => $ysize);
}

function polygonize($footprint, $dbh, $config) {

echo ' --> Polygonize ' . $footprint . "\n";

$tifName = '/tmp/' . md5(microtime()) . '.tif';
$tmpTable = 'tmptable';

// Crop GlobCover raster
$cropOrigin = cropOriginGlobCover(bbox($footprint));
$srcWin = $cropOrigin['x'] . ' ' . $cropOrigin['y'] . ' ' . $cropOrigin['xsize'] . ' ' . $cropOrigin['ysize'];

// Crop GlobCover raster to $srcWin
exec($config['translate'] . " -of GTiff -srcwin " . $srcWin . " -a_srs EPSG:4326 " . $config['image'] . ' ' . $tifName);

// In case of crashing crop - polygon is outside bounds for example
if (!file_exists($tifName)) {
echo " --> WARNING : globcover crop error\n";
return;
}

// Polygonize extracted raster within temporary table $tmpTable
exec($config['polygonize'] . ' ' . $tifName . ' -f "PostgreSQL" PG:"host=' . $config['host'] . ' user=' . $config['user'] . ' password=' . $config['password'] . ' dbname=' . $config['dbname'] . '" ' . $tmpTable . ' 2>&1');

pg_query($dbh, 'INSERT INTO datasources.landcover2009(wkb_geometry,dn) SELECT wkb_geometry,dn FROM ' . $tmpTable);
unlink($tifName);

// Drop tmpTable - bug in gdla_polygonize.py ?
pg_query($dbh, 'DROP TABLE ' . $tmpTable);

}

/**
*
* Returns bounding box [ulx, uly, lrx, lry] from a WKT
*
* ULx,ULy
* +------------------+
* | |
* | |
* | |
* | |
* +------------------+
* LRx,LRy
*
* Example of WKT POLYGON :
* POLYGON((-180.0044642857 89.9955356663,-180.0044642857 87.9955356727,-178.0044642921 87.9955356727,-178.0044642921 89.9955356663,-180.0044642857 89.9955356663))
*
* @param <string> $wkt : WKT
* @return string : random table name
*
*/
function bbox($wkt) {
$ulx = 180.0;
$uly = -90.0;
$lrx = -180.0;
$lry = 90.0;
$rep = array("(", ")", "multi", "polygon", "point", "linestring");
$pairs = preg_split('/,/', str_replace($rep, "", strtolower($wkt)));
for ($i = 0; $i < count($pairs); $i++) {
$coords = preg_split('/ /', trim($pairs[$i]));
$x = floatval($coords[0]);
$y = floatval($coords[1]);
if ($x < $ulx) {
$ulx = $x;
} else if ($x > $lrx) {
$lrx = $x;
}
if ($y > $uly) {
$uly = $y;
} else if ($y < $lry) {
$lry = $y;
}
}

return array('ulx' => $ulx, 'uly' => $uly, 'lrx' => $lrx, 'lry' => $lry);
}
Loading

0 comments on commit fb9f923

Please sign in to comment.