Tile Server (2011): Difference between revisions

From Planimate Knowledge Base
Jump to navigation Jump to search
No edit summary
No edit summary
Line 11: Line 11:
I gave the VM plenty of RAM for the osm2pgsl import stage (see below). Since then I've shrunk the VM's RAM configuration to 1.8GB which is more than plenty for a small scale test server.  
I gave the VM plenty of RAM for the osm2pgsl import stage (see below). Since then I've shrunk the VM's RAM configuration to 1.8GB which is more than plenty for a small scale test server.  


You'll need to know the IP address of your server. Running 'ifconfig' should tell you this and the "eth" device being used. If you want to give your server a static IP address, edit /etc/network/interfaces, substituting your network details.<br>
You'll need to know the IP address of your server. Running 'ifconfig' should tell you this and the "eth" device being used. If you want to give your server a static IP address, edit /etc/network/interfaces, substituting your network details.<br>  
<pre>sudo vim /etc/network/interfaces
<pre>sudo vim /etc/network/interfaces


Line 19: Line 19:
gateway 192.168.0.100
gateway 192.168.0.100
</pre>  
</pre>  
After this execute
After this execute  
<pre>sudo ifdown eth0; ifup eth0
<pre>sudo ifdown eth0; ifup eth0
</pre>
</pre>  
You can then use a terminal like PuTTY&nbsp;to log in.
You can then use a terminal like PuTTY&nbsp;to log in.  


=== Installing PostGIS, Mapnik  ===
=== Installing PostGIS, Mapnik  ===


Follow the guide linked above. Its useful to ssh in and copy/paste in a line at a time. The guide is pretty well spot on '''except these''':
Follow the guide linked above after reading the amendments below. Its useful to ssh in and copy/paste in a line at a time.  


*I found I had to install 'unzip', might as well do that first
*I had to install 'unzip', might as well do that first
<pre>sudo apt-get install unzip</pre>  
<pre>sudo apt-get install unzip</pre>  
*Later in the guide it says to do this:
*Later in the guide it says to do this:
Line 36: Line 36:
If you read the article's comments, you'll find this is how it used to be.  
If you read the article's comments, you'll find this is how it used to be.  


*I only downloaded Australian data, so my import was different. Also notice the -C parameter. This is the amount of RAM the import uses for caching, you might want to reduce it if space is tight.<br>
*I only downloaded Australian data, so my import was different. Also notice the -C parameter. This is the amount of RAM the import uses for caching, you might want to reduce it if your VM&nbsp;has less than 3GB&nbsp;of RAM.<br>
<pre>cd ~/bin/osm2pgsql
<pre>cd ~/bin/osm2pgsql
./osm2pgsql -S default.style --slim -d gis -C 2048 ~/planet/australia.osm.bz2
./osm2pgsql -S default.style --slim -d gis -C 2048 ~/planet/australia.osm.bz2
</pre>  
</pre>  
It took about 15 minutes. I continued with the mapnik installation while the database ground away. Take care copying those long URLs in the prepared data section.  
It took about 15 minutes. I actually put the bz2 file on a different drive to reduce thrashing. I continued with the mapnik installation while the database ground away. Take care copying those long URLs in the prepared data section.  


The guide got me to generating a map from the command line with little fuss.  
Final amendment - before the last step of running ./generate_image.py, perform these steps:<br>


=== Create an OSM file for your database  ===
*Customise generate_image.py to use the XML&nbsp;file we'll create next. Mapnick has evolved and now uses templates. I also edited some of the other py files, like generate_tiles.py, but this isn't necessary. For generate_image.py, the changes are:<br>
<pre>mapfile = "my_osm.xml"
bounds = (144.37, -38.0, 144.39, -38.2)
</pre>
*Create a customised OSM file.<br>


This file tells renderd about your database. I&nbsp;also edited some of the other py files like render_tiles.py to point to "my_osm.py".
This file tells mapnik (and renderd)&nbsp;about your data:<br>
<pre>cd ~/bin/mapnik
<pre>cd ~/bin/mapnik
./generate_xml.py osm.xml my_osm.xml --dbname gis --symbols ./symbols/ --world_boundaries ./world_boundaries/ --user rick --accept-none</pre>  
./generate_xml.py osm.xml my_osm.xml --dbname gis --symbols ./symbols/ --world_boundaries ./world_boundaries/ --user rick --accept-none</pre>  
Running ./generate_image.py now should create image.png showing a region in Victoria.<br>
=== Web Server  ===
=== Web Server  ===


The module uses a multithreaded version of apache.  
Now we set up the tile server. The mod_tile module uses a multithreaded version of apache.  
<pre>sudo apt-get install apache2 apache2-threaded-dev</pre>  
<pre>sudo apt-get install apache2 apache2-threaded-dev</pre>  
=== Build Apache Tile Module &amp;&nbsp;renderd  ===
=== Build Apache Tile Module &amp;&nbsp;renderd  ===
Line 66: Line 72:
sudo mkdir /var/run/renderd
sudo mkdir /var/run/renderd
sudo chown rick /var/run/renderd
sudo chown rick /var/run/renderd
</pre>  
</pre>  
(if directories already exist, make sure they have write access to the user).  
(if directories already exist, make sure they have write access to the user).  
Line 90: Line 95:
XML=/home/rick/bin/mapnik/my_osm.xml
XML=/home/rick/bin/mapnik/my_osm.xml
#HTCPHOST=proxy.openstreetmap.org</pre>  
#HTCPHOST=proxy.openstreetmap.org</pre>  
You can add alternate tilesets/layer configurations by adding another section eg:  
TIP: You can add alternate tilesets/layer configurations by adding another section eg:  
<pre>[tiles2]
<pre>[tiles2]
URI=/tiles2/
URI=/tiles2/
Line 127: Line 132:
For debugging, do this in a separate window - as regular user, not root.  
For debugging, do this in a separate window - as regular user, not root.  
<pre>~/src/mod_tile/renderd -f</pre>  
<pre>~/src/mod_tile/renderd -f</pre>  
You should see activity as tiles are requested. You'd start a single instance in the background using
We put this (without the -f)&nbsp;into rc.local later.
<pre>~/src/mod_tile/renderd</pre>
or multiple instances (for each thread) using
<pre> ~/src/mod_tile/renderd.py &amp;</pre>
=== Testing<br> ===


From a web browser, navigate to the address of your new server, <br>
=== Testing<br> ===


http://yourserver/mod_tile<br>
From a web browser, navigate to the address of your new server, <br>  


You should see something like:<br>
http://yourserver/mod_tile<br>
 
You should see something like:<br>  
<pre>NoResp200: 0
<pre>NoResp200: 0
NoResp304: 0
NoResp304: 0
Line 147: Line 150:
:
:
</pre>  
</pre>  
Now fetch a tile<br>
Now fetch a tile<br>  


http://yourserver/tiles/0/0/0.png<br>
http://yourserver/tiles/0/0/0.png<br>  


If things dont work, check:  
If things dont work, check:  
Line 158: Line 161:
*Any activity in the renderd console window?<br>
*Any activity in the renderd console window?<br>


=== Starting Automatically<br> ===
=== Starting Automatically<br> ===


I inserted this in /etc/rc.local<br>
I inserted this in /etc/rc.local<br>  
<pre>mkdir /var/run/renderd
<pre>mkdir /var/run/renderd
chown rick /var/run/renderd
chown rick /var/run/renderd
sudo -u rick /home/rick/src/mod_tile/renderd
sudo -u rick /home/rick/src/mod_tile/renderd
</pre>
</pre>  
=== Adding a map to view tiles in a browser<br> ===
=== Adding a map to view tiles in a browser<br> ===


Download and install OpenLayers:  
Download and install OpenLayers:  
Line 181: Line 184:
Change the layer = line to use our server instead of open street maps.  
Change the layer = line to use our server instead of open street maps.  
<pre>layer = new OpenLayers.Layer.OSM("RickTiles", "/tiles/${z}/${x}/${y}.png", {numZoomLevels: 19});</pre>  
<pre>layer = new OpenLayers.Layer.OSM("RickTiles", "/tiles/${z}/${x}/${y}.png", {numZoomLevels: 19});</pre>  
You can add: set layer.attribution="whatever"; if you like.<br>
You can add: set layer.attribution="whatever"; if you like.<br>  


Change the default co-ordinate to somewhere in Australia  
Change the default co-ordinate to somewhere in Australia  
<pre>...new OpenLayers.LonLat(144.38, -38.10).transform(...</pre>  
<pre>...new OpenLayers.LonLat(144.38, -38.10).transform(...</pre>  
Now navigate to http://yourserver/osm.html and you should have a map.
Now navigate to http://yourserver/osm.html and you should have a map.  


[[Category:Paint]]
[[Category:Paint]]
<br>

Revision as of 00:55, 3 August 2011

The Planimate Map object enables the use of tiled map data in models. These notes describe the setup of a map tile web server which can be used with Planimate or other open Street Maps projects.

The bulk of the install is described here and credit to the author for this guide, which gets you from a bare linux install to locally creating map tiles.

Setting up the VM

I created the VM by pointing VMWare to a Ubuntu 10.04 ISO (ubuntu-10.04-server-amd64.iso) in its new VM wizard. I initially set 3.5GB of RAM, 20GB HDD and bridged networking. Selected defaults for the rest.. In a few minutes I was at a login prompt.

This was a first test, only for the Australia region. In the end, about 6GB of space has been used (not including tiles). With the current growth of the world osm file, you'll need 600GB of disk space.

I gave the VM plenty of RAM for the osm2pgsl import stage (see below). Since then I've shrunk the VM's RAM configuration to 1.8GB which is more than plenty for a small scale test server.

You'll need to know the IP address of your server. Running 'ifconfig' should tell you this and the "eth" device being used. If you want to give your server a static IP address, edit /etc/network/interfaces, substituting your network details.

sudo vim /etc/network/interfaces

iface eth0 inet static
address 192.168.0.18
netmask 255.255.255.0
gateway 192.168.0.100

After this execute

sudo ifdown eth0; ifup eth0

You can then use a terminal like PuTTY to log in.

Installing PostGIS, Mapnik

Follow the guide linked above after reading the amendments below. Its useful to ssh in and copy/paste in a line at a time.

  • I had to install 'unzip', might as well do that first
sudo apt-get install unzip
  • Later in the guide it says to do this:
psql -f /usr/share/postgresql/8.4/contrib/postgis-1.5/postgis.sql -d gis

This directory didn't exist so instead I did:

psql -f /usr/share/postgresql/8.4/contrib/postgis.sql -d gis

If you read the article's comments, you'll find this is how it used to be.

  • I only downloaded Australian data, so my import was different. Also notice the -C parameter. This is the amount of RAM the import uses for caching, you might want to reduce it if your VM has less than 3GB of RAM.
cd ~/bin/osm2pgsql
./osm2pgsql -S default.style --slim -d gis -C 2048 ~/planet/australia.osm.bz2

It took about 15 minutes. I actually put the bz2 file on a different drive to reduce thrashing. I continued with the mapnik installation while the database ground away. Take care copying those long URLs in the prepared data section.

Final amendment - before the last step of running ./generate_image.py, perform these steps:

  • Customise generate_image.py to use the XML file we'll create next. Mapnick has evolved and now uses templates. I also edited some of the other py files, like generate_tiles.py, but this isn't necessary. For generate_image.py, the changes are:
mapfile = "my_osm.xml"
bounds = (144.37, -38.0, 144.39, -38.2)
  • Create a customised OSM file.

This file tells mapnik (and renderd) about your data:

cd ~/bin/mapnik
./generate_xml.py osm.xml my_osm.xml --dbname gis --symbols ./symbols/ --world_boundaries ./world_boundaries/ --user rick --accept-none

Running ./generate_image.py now should create image.png showing a region in Victoria.

Web Server

Now we set up the tile server. The mod_tile module uses a multithreaded version of apache.

sudo apt-get install apache2 apache2-threaded-dev

Build Apache Tile Module & renderd

Make sure you are the user that can access the DB, not root.

cd ~/src
svn co http://svn.openstreetmap.org/applications/utils/mod_tile/
cd mod_tile
make
sudo make install
sudo mkdir /var/lib/mod_tile
sudo chown rick /var/lib/mod_tile
touch /var/lib/mod_tile/planet-import-complete
sudo mkdir /var/run/renderd
sudo chown rick /var/run/renderd

(if directories already exist, make sure they have write access to the user).

Edit renderd configuration

The make install above copied /etc/renderd.conf from the src/mod_tile directory. In this setup, both Apache and the renderd program use this configuration file.

sudo vim /etc/renderd.conf

Edit as follows, adjust the home directory as appropriate:

[renderd]
socketname=/var/run/renderd/renderd.sock
num_threads=4
tile_dir=/var/lib/mod_tile ; DOES NOT WORK YET
stats_file=/var/run/renderd/renderd.stats

[mapnik]
plugins_dir=/usr/local/lib/mapnik/input
font_dir=/home/rick/src/mapnik/fonts
font_dir_recurse=1

[default]
URI=/tiles/
XML=/home/rick/bin/mapnik/my_osm.xml
#HTCPHOST=proxy.openstreetmap.org

TIP: You can add alternate tilesets/layer configurations by adding another section eg:

[tiles2]
URI=/tiles2/
XML=/home/rick/bin/mapnik/my_osm2.xml

In my test I copied my_osm.xml to my_osm2.xml and disabled some layers by adding attribute status="off" to the Layer elements in the second half of the file.

Set up Apache configuration

Copy the configuration file for mod_tile then edit it:

cd ~/src/mod_tile
sudo cp mod_tile.conf /etc/apache2/mods-available
sudo vim /etc/apache2/mods-available/mod_tile.conf

The changes are in the first few lines:

  • comment this out, we'll load it separately
#LoadModule tile_module modules/mod_tile.so
  • change the servername and comment out the alias line
ServerName maps
  • update the document root
DocumentRoot /var/www/
  • enable the use of the config file in /etc
LoadTileConfigFile /etc/renderd.conf

Thats it for the edits.

Now create a load file for the module and create the links in mods-enabled.

sudo su root
echo LoadModule tile_module /usr/lib/apache2/modules/mod_tile.so > /etc/apache2/mods-available/mod_tile.load
ln -s /etc/apache2/mods-available/mod_tile.load /etc/apache2/mods-enabled/mod_tile.load
ln -s /etc/apache2/mods-available/mod_tile.conf /etc/apache2/mods-enabled/mod_tile.conf
exit

You can now restart apache and check the module is loaded:

apache2ctl restart

Start renderd

For debugging, do this in a separate window - as regular user, not root.

~/src/mod_tile/renderd -f

We put this (without the -f) into rc.local later.

Testing

From a web browser, navigate to the address of your new server,

http://yourserver/mod_tile

You should see something like:

NoResp200: 0
NoResp304: 0
NoResp404: 0
NoResp503: 0
NoResp5XX: 0
NoRespOther: 0
NoFreshCache: 0
:

Now fetch a tile

http://yourserver/tiles/0/0/0.png

If things dont work, check:

  • Can renderd write to /var/lib/mod_tile and /var/run/renderd? Have files been created there?
  • Check apache's error log
  • Is font path correct in renderd.conf
  • Any activity in the renderd console window?

Starting Automatically

I inserted this in /etc/rc.local

mkdir /var/run/renderd
chown rick /var/run/renderd
sudo -u rick /home/rick/src/mod_tile/renderd

Adding a map to view tiles in a browser

Download and install OpenLayers:

cd ~
wget http://openlayers.org/download/OpenLayers-2.10.tar.gz
tar -xzvf OpenLayers-2.10.tar.gz
cd OpenLayers-2.10
sudo cp OpenLayers.js /var/www
sudo cp -r style /var/www
sudo cp -r theme /var/www
sudo cp examples/style.css /var/www
sudo cp examples/osm.html /var/www

Now customise osm.html to our site

sudo vim /var/www/osm.html

Change the layer = line to use our server instead of open street maps.

layer = new OpenLayers.Layer.OSM("RickTiles", "/tiles/${z}/${x}/${y}.png", {numZoomLevels: 19});

You can add: set layer.attribution="whatever"; if you like.

Change the default co-ordinate to somewhere in Australia

...new OpenLayers.LonLat(144.38, -38.10).transform(...

Now navigate to http://yourserver/osm.html and you should have a map.