Tile Server (2011): Difference between revisions

From Planimate Knowledge Base
Jump to navigation Jump to search
No edit summary
No edit summary
Line 1: Line 1:
The Planimate [[Map (Paint Object)|Map]] object enables the use of tiled map data in models. These notes describe the setup of a map tile web server which can be used with Planimate or other open Street Maps projects.<br>
The Planimate [[Map (Paint Object)|Map]] object enables the use of tiled map data in models. These notes describe the setup of a map tile web server which can be used with Planimate or other open Street Maps projects.<br>  


The bulk of the install is described [http://weait.com/content/build-your-own-openstreetmap-server here] and credit to the author for this guide, which gets you from a bare linux install to locally creating map tiles.<br>
The bulk of the install is described [http://weait.com/content/build-your-own-openstreetmap-server here] and credit to the author for this guide, which gets you from a bare linux install to locally creating map tiles.<br>  


=== Setting up the VM  ===
=== Setting up the VM  ===


I created the VM by pointing VMWare to a Ubuntu 10.04 ISO (ubuntu-10.04-server-amd64.iso) in its new VM&nbsp;wizard. I initially set 3.5GB of RAM, 20GB HDD and bridged networking. Selected defaults for the rest.. In a few minutes I was at a login prompt.
I created the VM by pointing VMWare to a Ubuntu 10.04 ISO (ubuntu-10.04-server-amd64.iso) in its new VM&nbsp;wizard. I initially set 3.5GB of RAM, 20GB HDD and bridged networking. Selected defaults for the rest.. In a few minutes I was at a login prompt.  


This was a first test, only for the Australia region. In the end, about 6GB of space has been used. With the current growth of the world osm file, you'll probably need 1TB of disk space.
This was a first test, only for the Australia region. In the end, about 6GB of space has been used (not including tiles). With the current growth of the world osm file, you'll need 600GB of disk space.  


I gave the VM plenty of RAM&nbsp;for the osm2pgsl import stage (see below). Since then I've shrunk the VM's RAM configuration to 1.8GB which is plenty for a small scale test server.
I gave the VM plenty of RAM&nbsp;for the osm2pgsl import stage (see below). Since then I've shrunk the VM's RAM configuration to 1.8GB which is more than plenty for a small scale test server.
 
You'll need to know the IP address of your server. Running 'ifconfig' should tell you this and the "eth" device being used. If you want to give your server a static IP address, edit /etc/network/interfaces, substituting your network details.<br>
<pre>sudo vim /etc/network/interfaces
 
iface eth0 inet static
address 192.168.0.18
netmask 255.255.255.0
gateway 192.168.0.100
</pre>
After this execute
<pre>sudo ifdown eth0; ifup eth0
</pre>
You can then use a terminal like PuTTY&nbsp;to log in.


=== Installing PostGIS, Mapnik  ===
=== Installing PostGIS, Mapnik  ===


Follow the guide linked above. Its useful to ssh in and copy/paste in a line at a time. I used PUTTY as the terminal. The guide is pretty well spot on except these:
Follow the guide linked above. Its useful to ssh in and copy/paste in a line at a time. The guide is pretty well spot on '''except these''':  


*I found I had to install 'unzip', might as well do that first
*I found I had to install 'unzip', might as well do that first
<pre>sudo apt-get install unzip</pre>
<pre>sudo apt-get install unzip</pre>  
*Later in the guide it says to do this:
*Later in the guide it says to do this:
<pre>psql -f /usr/share/postgresql/8.4/contrib/postgis-1.5/postgis.sql -d gis</pre>
<pre>psql -f /usr/share/postgresql/8.4/contrib/postgis-1.5/postgis.sql -d gis</pre>  
This directory didn't exist so instead I did:
This directory didn't exist so instead I did:  
<pre>psql -f /usr/share/postgresql/8.4/contrib/postgis.sql -d gis</pre>
<pre>psql -f /usr/share/postgresql/8.4/contrib/postgis.sql -d gis</pre>  
If you read the article's comments, you'll find this is how it used to be.
If you read the article's comments, you'll find this is how it used to be.  


*I only downloaded Australian data, so my import was different. Also notice the -C parameter. This is the amount of RAM the import uses for caching, you might want to reduce it if space is tight.<br>
*I only downloaded Australian data, so my import was different. Also notice the -C parameter. This is the amount of RAM the import uses for caching, you might want to reduce it if space is tight.<br>
<pre>cd ~/bin/osm2pgsql
<pre>cd ~/bin/osm2pgsql
./osm2pgsql -S default.style --slim -d gis -C 2048 ~/planet/australia.osm.bz2
./osm2pgsql -S default.style --slim -d gis -C 2048 ~/planet/australia.osm.bz2
</pre>
</pre>  
It took about 15 minutes. I continued with the mapnik installation while the database ground away. Take care copying those long URLs in the prepared data section.
It took about 15 minutes. I continued with the mapnik installation while the database ground away. Take care copying those long URLs in the prepared data section.  


The guide got me to generating a map from the command line with little fuss.
The guide got me to generating a map from the command line with little fuss.  


=== Create an OSM file for your database  ===
=== Create an OSM file for your database  ===


This file tells renderd about your database. I&nbsp;also edited some of the other py files like render_tiles.py to point to "my_osm.py".
This file tells renderd about your database. I&nbsp;also edited some of the other py files like render_tiles.py to point to "my_osm.py".  
<pre>cd ~/bin/mapnik
<pre>cd ~/bin/mapnik
./generate_xml.py osm.xml my_osm.xml --dbname gis --symbols ./symbols/ --world_boundaries ./world_boundaries/ --user rick --accept-none</pre>
./generate_xml.py osm.xml my_osm.xml --dbname gis --symbols ./symbols/ --world_boundaries ./world_boundaries/ --user rick --accept-none</pre>  
=== Web Server  ===
=== Web Server  ===


The module uses a multithreaded version of apache.
The module uses a multithreaded version of apache.  
<pre>sudo apt-get install apache2 apache2-threaded-dev</pre>
<pre>sudo apt-get install apache2 apache2-threaded-dev</pre>  
=== Build Apache Tile Module &amp;&nbsp;renderd  ===
=== Build Apache Tile Module &amp;&nbsp;renderd  ===


Make sure you are the user that can access the DB, not root.<br>
Make sure you are the user that can access the DB, not root.<br>  
<pre>cd ~/src
<pre>cd ~/src
svn co http://svn.openstreetmap.org/applications/utils/mod_tile/
svn co http://svn.openstreetmap.org/applications/utils/mod_tile/
Line 54: Line 67:
sudo chown rick /var/run/renderd
sudo chown rick /var/run/renderd


</pre>
</pre>  
(if directories already exist, make sure they have write access to the user).
(if directories already exist, make sure they have write access to the user).  


=== Edit renderd configuration  ===
=== Edit renderd configuration  ===


The make install above copied /etc/renderd.conf from the src directory. In this setup, both Apache and the renderd program use this configuration file.
The make install above copied /etc/renderd.conf from the src/mod_tile directory. In this setup, both Apache and the renderd program use this configuration file.  
<pre>sudo vim /etc/renderd.conf</pre>
<pre>sudo vim /etc/renderd.conf</pre>  
Edit as follows, adjust the home directory as appropriate:
Edit as follows, adjust the home directory as appropriate:  
<pre>[renderd]
<pre>[renderd]
socketname=/var/run/renderd/renderd.sock
socketname=/var/run/renderd/renderd.sock
Line 76: Line 89:
URI=/tiles/
URI=/tiles/
XML=/home/rick/bin/mapnik/my_osm.xml
XML=/home/rick/bin/mapnik/my_osm.xml
#HTCPHOST=proxy.openstreetmap.org</pre>
#HTCPHOST=proxy.openstreetmap.org</pre>  
You can add alternate tilesets/layer configurations by adding another section eg:
You can add alternate tilesets/layer configurations by adding another section eg:  
<pre>[tiles2]
<pre>[tiles2]
URI=/tiles2/
URI=/tiles2/
XML=/home/rick/bin/mapnik/my_osm2.xml</pre>
XML=/home/rick/bin/mapnik/my_osm2.xml</pre>  
 
In my test I copied my_osm.xml to my_osm2.xml and disabled some layers by adding attribute '''status="off"''' to the Layer elements in the second half of the file.  
In my test I copied my_osm.xml to my_osm2.xml and disabled some layers by adding attribute status="off" to the Layer element.


=== Set up Apache configuration  ===
=== Set up Apache configuration  ===


Copy the configuration file for mod_tile then edit it:
Copy the configuration file for mod_tile then edit it:  
<pre>cd ~/src/mod_tile
<pre>cd ~/src/mod_tile
sudo cp mod_tile.conf /etc/apache2/mods-available
sudo cp mod_tile.conf /etc/apache2/mods-available
sudo vim /etc/apache2/mods-available/mod_tile.conf</pre>
sudo vim /etc/apache2/mods-available/mod_tile.conf</pre>  
The changes are in the first few lines:
The changes are in the first few lines:  


*comment this out, we'll load it separately
*comment this out, we'll load it separately
<pre>#LoadModule tile_module modules/mod_tile.so</pre>
<pre>#LoadModule tile_module modules/mod_tile.so</pre>  
*change the servername and comment out the alias line
*change the servername and comment out the alias line
<pre>ServerName maps</pre>
<pre>ServerName maps</pre>  
*update the document root
*update the document root
<pre>DocumentRoot /var/www/</pre>
<pre>DocumentRoot /var/www/</pre>  
*enable the use of the config file in /etc
*enable the use of the config file in /etc
<pre>LoadTileConfigFile /etc/renderd.conf</pre>
<pre>LoadTileConfigFile /etc/renderd.conf</pre>  
Thats it for the edits.
Thats it for the edits.  


Now create a load file for the module and create the links in mods-enabled.
Now create a load file for the module and create the links in mods-enabled.  
<pre>sudo su root
<pre>sudo su root
echo LoadModule tile_module /usr/lib/apache2/modules/mod_tile.so &gt; /etc/apache2/mods-available/mod_tile.load
echo LoadModule tile_module /usr/lib/apache2/modules/mod_tile.so &gt; /etc/apache2/mods-available/mod_tile.load
Line 108: Line 120:
ln -s /etc/apache2/mods-available/mod_tile.conf /etc/apache2/mods-enabled/mod_tile.conf
ln -s /etc/apache2/mods-available/mod_tile.conf /etc/apache2/mods-enabled/mod_tile.conf
exit
exit
</pre>
</pre>  
You can now restart apache and check the module is loaded:
You can now restart apache and check the module is loaded:  
<pre>apache2ctl restart</pre>
<pre>apache2ctl restart</pre>  
=== Start renderd  ===
=== Start renderd  ===


For debugging, do this in a separate window - as regular user, not root.
For debugging, do this in a separate window - as regular user, not root.  
<pre>~/src/mod_tile/renderd -f</pre>
<pre>~/src/mod_tile/renderd -f</pre>  
You should see activity as tiles are requested. You'd start a single instance in the background using
You should see activity as tiles are requested. You'd start a single instance in the background using  
<pre>~/src/mod_tile/renderd</pre>
<pre>~/src/mod_tile/renderd</pre>  
or multiple instances (for each thread) using
or multiple instances (for each thread) using  
<pre> ~/src/mod_tile/renderd.py &amp;</pre>
<pre> ~/src/mod_tile/renderd.py &amp;</pre>  
=== Testing ===
=== Testing<br> ===
 
From a web browser, navigate to the address of your new server, <br>
 
http://yourserver/mod_tile<br>
 
You should see something like:<br>
<pre>NoResp200: 0
NoResp304: 0
NoResp404: 0
NoResp503: 0
NoResp5XX: 0
NoRespOther: 0
NoFreshCache: 0
:
</pre>
Now fetch a tile<br>


From a web browser, navigate to the IP of your VM
http://yourserver/tiles/0/0/0.png<br>
<pre>http://192.168.0.151/mod_tile</pre>
Now fetch a tile
<pre>http://192.168.0.151/tiles/0/0/0.png</pre>
If things dont work, check:


*Can renderd write to /var/lib/mod_tile and /bar/run/renderd
If things dont work, check:
*Apache's error log<br>
*Is font path correct in renderd.conf
=== Adding a map to view your tiles ===


Download and install OpenLayers
*Can renderd write to /var/lib/mod_tile and /var/run/renderd? Have files been created there?
*Check apache's error log<br>
*Is font path correct in renderd.conf
*Any activity in the renderd console window?<br>


=== Starting Automatically<br> ===
I inserted this in /etc/rc.local<br>
<pre>mkdir /var/run/renderd
chown rick /var/run/renderd
sudo -u rick /home/rick/src/mod_tile/renderd
</pre>
=== Adding a map to view tiles in a browser<br> ===
Download and install OpenLayers:
<pre>cd ~
<pre>cd ~
wget http://openlayers.org/download/OpenLayers-2.10.tar.gz
wget http://openlayers.org/download/OpenLayers-2.10.tar.gz
Line 142: Line 176:
sudo cp -r theme /var/www
sudo cp -r theme /var/www
sudo cp examples/style.css /var/www
sudo cp examples/style.css /var/www
sudo cp examples/osm.html /var/www</pre>
sudo cp examples/osm.html /var/www</pre>  
 
Now customise osm.html to our site  
Now customise osm.html to our site
<pre>sudo vim /var/www/osm.html</pre>  
<pre>sudo vim /var/www/osm.html</pre>
Change the layer = line to use our server instead of open street maps.  
Change layer = line to use our local server, not open street maps.
<pre>layer = new OpenLayers.Layer.OSM("RickTiles", "/tiles/${z}/${x}/${y}.png", {numZoomLevels: 19});</pre>  
<pre>layer = new OpenLayers.Layer.OSM("RickTiles", "/tiles/${z}/${x}/${y}.png", {numZoomLevels: 19});</pre>
You can add: set layer.attribution="whatever"; if you like.<br>
You can set layer.attribution="whatever";
Change the default co-ordinate to somewhere in Australia
<pre>...
new OpenLayers.LonLat(144.38, -38.10).transform(...</pre>


Change the default co-ordinate to somewhere in Australia
<pre>...new OpenLayers.LonLat(144.38, -38.10).transform(...</pre>
Now navigate to http://yourserver/osm.html and you should have a map.
Now navigate to http://yourserver/osm.html and you should have a map.


[[Category:Paint]]
[[Category:Paint]]

Revision as of 23:16, 2 August 2011

The Planimate Map object enables the use of tiled map data in models. These notes describe the setup of a map tile web server which can be used with Planimate or other open Street Maps projects.

The bulk of the install is described here and credit to the author for this guide, which gets you from a bare linux install to locally creating map tiles.

Setting up the VM

I created the VM by pointing VMWare to a Ubuntu 10.04 ISO (ubuntu-10.04-server-amd64.iso) in its new VM wizard. I initially set 3.5GB of RAM, 20GB HDD and bridged networking. Selected defaults for the rest.. In a few minutes I was at a login prompt.

This was a first test, only for the Australia region. In the end, about 6GB of space has been used (not including tiles). With the current growth of the world osm file, you'll need 600GB of disk space.

I gave the VM plenty of RAM for the osm2pgsl import stage (see below). Since then I've shrunk the VM's RAM configuration to 1.8GB which is more than plenty for a small scale test server.

You'll need to know the IP address of your server. Running 'ifconfig' should tell you this and the "eth" device being used. If you want to give your server a static IP address, edit /etc/network/interfaces, substituting your network details.

sudo vim /etc/network/interfaces

iface eth0 inet static
address 192.168.0.18
netmask 255.255.255.0
gateway 192.168.0.100

After this execute

sudo ifdown eth0; ifup eth0

You can then use a terminal like PuTTY to log in.

Installing PostGIS, Mapnik

Follow the guide linked above. Its useful to ssh in and copy/paste in a line at a time. The guide is pretty well spot on except these:

  • I found I had to install 'unzip', might as well do that first
sudo apt-get install unzip
  • Later in the guide it says to do this:
psql -f /usr/share/postgresql/8.4/contrib/postgis-1.5/postgis.sql -d gis

This directory didn't exist so instead I did:

psql -f /usr/share/postgresql/8.4/contrib/postgis.sql -d gis

If you read the article's comments, you'll find this is how it used to be.

  • I only downloaded Australian data, so my import was different. Also notice the -C parameter. This is the amount of RAM the import uses for caching, you might want to reduce it if space is tight.
cd ~/bin/osm2pgsql
./osm2pgsql -S default.style --slim -d gis -C 2048 ~/planet/australia.osm.bz2

It took about 15 minutes. I continued with the mapnik installation while the database ground away. Take care copying those long URLs in the prepared data section.

The guide got me to generating a map from the command line with little fuss.

Create an OSM file for your database

This file tells renderd about your database. I also edited some of the other py files like render_tiles.py to point to "my_osm.py".

cd ~/bin/mapnik
./generate_xml.py osm.xml my_osm.xml --dbname gis --symbols ./symbols/ --world_boundaries ./world_boundaries/ --user rick --accept-none

Web Server

The module uses a multithreaded version of apache.

sudo apt-get install apache2 apache2-threaded-dev

Build Apache Tile Module & renderd

Make sure you are the user that can access the DB, not root.

cd ~/src
svn co http://svn.openstreetmap.org/applications/utils/mod_tile/
cd mod_tile
make
sudo make install
sudo mkdir /var/lib/mod_tile
sudo chown rick /var/lib/mod_tile
touch /var/lib/mod_tile/planet-import-complete
sudo mkdir /var/run/renderd
sudo chown rick /var/run/renderd

(if directories already exist, make sure they have write access to the user).

Edit renderd configuration

The make install above copied /etc/renderd.conf from the src/mod_tile directory. In this setup, both Apache and the renderd program use this configuration file.

sudo vim /etc/renderd.conf

Edit as follows, adjust the home directory as appropriate:

[renderd]
socketname=/var/run/renderd/renderd.sock
num_threads=4
tile_dir=/var/lib/mod_tile ; DOES NOT WORK YET
stats_file=/var/run/renderd/renderd.stats

[mapnik]
plugins_dir=/usr/local/lib/mapnik/input
font_dir=/home/rick/src/mapnik/fonts
font_dir_recurse=1

[default]
URI=/tiles/
XML=/home/rick/bin/mapnik/my_osm.xml
#HTCPHOST=proxy.openstreetmap.org

You can add alternate tilesets/layer configurations by adding another section eg:

[tiles2]
URI=/tiles2/
XML=/home/rick/bin/mapnik/my_osm2.xml

In my test I copied my_osm.xml to my_osm2.xml and disabled some layers by adding attribute status="off" to the Layer elements in the second half of the file.

Set up Apache configuration

Copy the configuration file for mod_tile then edit it:

cd ~/src/mod_tile
sudo cp mod_tile.conf /etc/apache2/mods-available
sudo vim /etc/apache2/mods-available/mod_tile.conf

The changes are in the first few lines:

  • comment this out, we'll load it separately
#LoadModule tile_module modules/mod_tile.so
  • change the servername and comment out the alias line
ServerName maps
  • update the document root
DocumentRoot /var/www/
  • enable the use of the config file in /etc
LoadTileConfigFile /etc/renderd.conf

Thats it for the edits.

Now create a load file for the module and create the links in mods-enabled.

sudo su root
echo LoadModule tile_module /usr/lib/apache2/modules/mod_tile.so > /etc/apache2/mods-available/mod_tile.load
ln -s /etc/apache2/mods-available/mod_tile.load /etc/apache2/mods-enabled/mod_tile.load
ln -s /etc/apache2/mods-available/mod_tile.conf /etc/apache2/mods-enabled/mod_tile.conf
exit

You can now restart apache and check the module is loaded:

apache2ctl restart

Start renderd

For debugging, do this in a separate window - as regular user, not root.

~/src/mod_tile/renderd -f

You should see activity as tiles are requested. You'd start a single instance in the background using

~/src/mod_tile/renderd

or multiple instances (for each thread) using

 ~/src/mod_tile/renderd.py &

Testing

From a web browser, navigate to the address of your new server,

http://yourserver/mod_tile

You should see something like:

NoResp200: 0
NoResp304: 0
NoResp404: 0
NoResp503: 0
NoResp5XX: 0
NoRespOther: 0
NoFreshCache: 0
:

Now fetch a tile

http://yourserver/tiles/0/0/0.png

If things dont work, check:

  • Can renderd write to /var/lib/mod_tile and /var/run/renderd? Have files been created there?
  • Check apache's error log
  • Is font path correct in renderd.conf
  • Any activity in the renderd console window?

Starting Automatically

I inserted this in /etc/rc.local

mkdir /var/run/renderd
chown rick /var/run/renderd
sudo -u rick /home/rick/src/mod_tile/renderd

Adding a map to view tiles in a browser

Download and install OpenLayers:

cd ~
wget http://openlayers.org/download/OpenLayers-2.10.tar.gz
tar -xzvf OpenLayers-2.10.tar.gz
cd OpenLayers-2.10
sudo cp OpenLayers.js /var/www
sudo cp -r style /var/www
sudo cp -r theme /var/www
sudo cp examples/style.css /var/www
sudo cp examples/osm.html /var/www

Now customise osm.html to our site

sudo vim /var/www/osm.html

Change the layer = line to use our server instead of open street maps.

layer = new OpenLayers.Layer.OSM("RickTiles", "/tiles/${z}/${x}/${y}.png", {numZoomLevels: 19});

You can add: set layer.attribution="whatever"; if you like.

Change the default co-ordinate to somewhere in Australia

...new OpenLayers.LonLat(144.38, -38.10).transform(...

Now navigate to http://yourserver/osm.html and you should have a map.