Tile Server (2011): Difference between revisions
No edit summary |
No edit summary |
||
Line 1: | Line 1: | ||
The Planimate [[Map (Paint Object)|Map]] object enables the use of tiled map data in models. These notes describe the setup of a map tile web server which can be used with Planimate or other open Street Maps projects.<br> | The Planimate [[Map (Paint Object)|Map]] object enables the use of tiled map data in models. These notes describe the setup of a map tile web server which can be used with Planimate or other open Street Maps projects.<br> | ||
The bulk of the install is described [http://weait.com/content/build-your-own-openstreetmap-server here] and credit to the author for this guide, which gets you from a bare linux install to locally creating map tiles.<br> | The bulk of the install is described [http://weait.com/content/build-your-own-openstreetmap-server here] and credit to the author for this guide, which gets you from a bare linux install to locally creating map tiles.<br> | ||
=== Setting up the VM === | === Setting up the VM === | ||
I created the VM by pointing VMWare to a Ubuntu 10.04 ISO (ubuntu-10.04-server-amd64.iso) in its new VM wizard. I initially set 3.5GB of RAM, 20GB HDD and bridged networking. Selected defaults for the rest.. In a few minutes I was at a login prompt. | I created the VM by pointing VMWare to a Ubuntu 10.04 ISO (ubuntu-10.04-server-amd64.iso) in its new VM wizard. I initially set 3.5GB of RAM, 20GB HDD and bridged networking. Selected defaults for the rest.. In a few minutes I was at a login prompt. | ||
This was a first test, only for the Australia region. In the end, about 6GB of space has been used. With the current growth of the world osm file, you'll | This was a first test, only for the Australia region. In the end, about 6GB of space has been used (not including tiles). With the current growth of the world osm file, you'll need 600GB of disk space. | ||
I gave the VM plenty of RAM for the osm2pgsl import stage (see below). Since then I've shrunk the VM's RAM configuration to 1.8GB which is plenty for a small scale test server. | I gave the VM plenty of RAM for the osm2pgsl import stage (see below). Since then I've shrunk the VM's RAM configuration to 1.8GB which is more than plenty for a small scale test server. | ||
You'll need to know the IP address of your server. Running 'ifconfig' should tell you this and the "eth" device being used. If you want to give your server a static IP address, edit /etc/network/interfaces, substituting your network details.<br> | |||
<pre>sudo vim /etc/network/interfaces | |||
iface eth0 inet static | |||
address 192.168.0.18 | |||
netmask 255.255.255.0 | |||
gateway 192.168.0.100 | |||
</pre> | |||
After this execute | |||
<pre>sudo ifdown eth0; ifup eth0 | |||
</pre> | |||
You can then use a terminal like PuTTY to log in. | |||
=== Installing PostGIS, Mapnik === | === Installing PostGIS, Mapnik === | ||
Follow the guide linked above. Its useful to ssh in and copy/paste in a line at a time | Follow the guide linked above. Its useful to ssh in and copy/paste in a line at a time. The guide is pretty well spot on '''except these''': | ||
*I found I had to install 'unzip', might as well do that first | *I found I had to install 'unzip', might as well do that first | ||
<pre>sudo apt-get install unzip</pre> | <pre>sudo apt-get install unzip</pre> | ||
*Later in the guide it says to do this: | *Later in the guide it says to do this: | ||
<pre>psql -f /usr/share/postgresql/8.4/contrib/postgis-1.5/postgis.sql -d gis</pre> | <pre>psql -f /usr/share/postgresql/8.4/contrib/postgis-1.5/postgis.sql -d gis</pre> | ||
This directory didn't exist so instead I did: | This directory didn't exist so instead I did: | ||
<pre>psql -f /usr/share/postgresql/8.4/contrib/postgis.sql -d gis</pre> | <pre>psql -f /usr/share/postgresql/8.4/contrib/postgis.sql -d gis</pre> | ||
If you read the article's comments, you'll find this is how it used to be. | If you read the article's comments, you'll find this is how it used to be. | ||
*I only downloaded Australian data, so my import was different. Also notice the -C parameter. This is the amount of RAM the import uses for caching, you might want to reduce it if space is tight.<br> | *I only downloaded Australian data, so my import was different. Also notice the -C parameter. This is the amount of RAM the import uses for caching, you might want to reduce it if space is tight.<br> | ||
<pre>cd ~/bin/osm2pgsql | <pre>cd ~/bin/osm2pgsql | ||
./osm2pgsql -S default.style --slim -d gis -C 2048 ~/planet/australia.osm.bz2 | ./osm2pgsql -S default.style --slim -d gis -C 2048 ~/planet/australia.osm.bz2 | ||
</pre> | </pre> | ||
It took about 15 minutes. I continued with the mapnik installation while the database ground away. Take care copying those long URLs in the prepared data section. | It took about 15 minutes. I continued with the mapnik installation while the database ground away. Take care copying those long URLs in the prepared data section. | ||
The guide got me to generating a map from the command line with little fuss. | The guide got me to generating a map from the command line with little fuss. | ||
=== Create an OSM file for your database === | === Create an OSM file for your database === | ||
This file tells renderd about your database. I also edited some of the other py files like render_tiles.py to point to "my_osm.py". | This file tells renderd about your database. I also edited some of the other py files like render_tiles.py to point to "my_osm.py". | ||
<pre>cd ~/bin/mapnik | <pre>cd ~/bin/mapnik | ||
./generate_xml.py osm.xml my_osm.xml --dbname gis --symbols ./symbols/ --world_boundaries ./world_boundaries/ --user rick --accept-none</pre> | ./generate_xml.py osm.xml my_osm.xml --dbname gis --symbols ./symbols/ --world_boundaries ./world_boundaries/ --user rick --accept-none</pre> | ||
=== Web Server === | === Web Server === | ||
The module uses a multithreaded version of apache. | The module uses a multithreaded version of apache. | ||
<pre>sudo apt-get install apache2 apache2-threaded-dev</pre> | <pre>sudo apt-get install apache2 apache2-threaded-dev</pre> | ||
=== Build Apache Tile Module & renderd === | === Build Apache Tile Module & renderd === | ||
Make sure you are the user that can access the DB, not root.<br> | Make sure you are the user that can access the DB, not root.<br> | ||
<pre>cd ~/src | <pre>cd ~/src | ||
svn co http://svn.openstreetmap.org/applications/utils/mod_tile/ | svn co http://svn.openstreetmap.org/applications/utils/mod_tile/ | ||
Line 54: | Line 67: | ||
sudo chown rick /var/run/renderd | sudo chown rick /var/run/renderd | ||
</pre> | </pre> | ||
(if directories already exist, make sure they have write access to the user). | (if directories already exist, make sure they have write access to the user). | ||
=== Edit renderd configuration === | === Edit renderd configuration === | ||
The make install above copied /etc/renderd.conf from the src directory. In this setup, both Apache and the renderd program use this configuration file. | The make install above copied /etc/renderd.conf from the src/mod_tile directory. In this setup, both Apache and the renderd program use this configuration file. | ||
<pre>sudo vim /etc/renderd.conf</pre> | <pre>sudo vim /etc/renderd.conf</pre> | ||
Edit as follows, adjust the home directory as appropriate: | Edit as follows, adjust the home directory as appropriate: | ||
<pre>[renderd] | <pre>[renderd] | ||
socketname=/var/run/renderd/renderd.sock | socketname=/var/run/renderd/renderd.sock | ||
Line 76: | Line 89: | ||
URI=/tiles/ | URI=/tiles/ | ||
XML=/home/rick/bin/mapnik/my_osm.xml | XML=/home/rick/bin/mapnik/my_osm.xml | ||
#HTCPHOST=proxy.openstreetmap.org</pre> | #HTCPHOST=proxy.openstreetmap.org</pre> | ||
You can add alternate tilesets/layer configurations by adding another section eg: | You can add alternate tilesets/layer configurations by adding another section eg: | ||
<pre>[tiles2] | <pre>[tiles2] | ||
URI=/tiles2/ | URI=/tiles2/ | ||
XML=/home/rick/bin/mapnik/my_osm2.xml</pre> | XML=/home/rick/bin/mapnik/my_osm2.xml</pre> | ||
In my test I copied my_osm.xml to my_osm2.xml and disabled some layers by adding attribute '''status="off"''' to the Layer elements in the second half of the file. | |||
In my test I copied my_osm.xml to my_osm2.xml and disabled some layers by adding attribute status="off" to the Layer | |||
=== Set up Apache configuration === | === Set up Apache configuration === | ||
Copy the configuration file for mod_tile then edit it: | Copy the configuration file for mod_tile then edit it: | ||
<pre>cd ~/src/mod_tile | <pre>cd ~/src/mod_tile | ||
sudo cp mod_tile.conf /etc/apache2/mods-available | sudo cp mod_tile.conf /etc/apache2/mods-available | ||
sudo vim /etc/apache2/mods-available/mod_tile.conf</pre> | sudo vim /etc/apache2/mods-available/mod_tile.conf</pre> | ||
The changes are in the first few lines: | The changes are in the first few lines: | ||
*comment this out, we'll load it separately | *comment this out, we'll load it separately | ||
<pre>#LoadModule tile_module modules/mod_tile.so</pre> | <pre>#LoadModule tile_module modules/mod_tile.so</pre> | ||
*change the servername and comment out the alias line | *change the servername and comment out the alias line | ||
<pre>ServerName maps</pre> | <pre>ServerName maps</pre> | ||
*update the document root | *update the document root | ||
<pre>DocumentRoot /var/www/</pre> | <pre>DocumentRoot /var/www/</pre> | ||
*enable the use of the config file in /etc | *enable the use of the config file in /etc | ||
<pre>LoadTileConfigFile /etc/renderd.conf</pre> | <pre>LoadTileConfigFile /etc/renderd.conf</pre> | ||
Thats it for the edits. | Thats it for the edits. | ||
Now create a load file for the module and create the links in mods-enabled. | Now create a load file for the module and create the links in mods-enabled. | ||
<pre>sudo su root | <pre>sudo su root | ||
echo LoadModule tile_module /usr/lib/apache2/modules/mod_tile.so > /etc/apache2/mods-available/mod_tile.load | echo LoadModule tile_module /usr/lib/apache2/modules/mod_tile.so > /etc/apache2/mods-available/mod_tile.load | ||
Line 108: | Line 120: | ||
ln -s /etc/apache2/mods-available/mod_tile.conf /etc/apache2/mods-enabled/mod_tile.conf | ln -s /etc/apache2/mods-available/mod_tile.conf /etc/apache2/mods-enabled/mod_tile.conf | ||
exit | exit | ||
</pre> | </pre> | ||
You can now restart apache and check the module is loaded: | You can now restart apache and check the module is loaded: | ||
<pre>apache2ctl restart</pre> | <pre>apache2ctl restart</pre> | ||
=== Start renderd === | === Start renderd === | ||
For debugging, do this in a separate window - as regular user, not root. | For debugging, do this in a separate window - as regular user, not root. | ||
<pre>~/src/mod_tile/renderd -f</pre> | <pre>~/src/mod_tile/renderd -f</pre> | ||
You should see activity as tiles are requested. You'd start a single instance in the background using | You should see activity as tiles are requested. You'd start a single instance in the background using | ||
<pre>~/src/mod_tile/renderd</pre> | <pre>~/src/mod_tile/renderd</pre> | ||
or multiple instances (for each thread) using | or multiple instances (for each thread) using | ||
<pre> ~/src/mod_tile/renderd.py &</pre> | <pre> ~/src/mod_tile/renderd.py &</pre> | ||
=== Testing | === Testing<br> === | ||
From a web browser, navigate to the address of your new server, <br> | |||
http://yourserver/mod_tile<br> | |||
You should see something like:<br> | |||
<pre>NoResp200: 0 | |||
NoResp304: 0 | |||
NoResp404: 0 | |||
NoResp503: 0 | |||
NoResp5XX: 0 | |||
NoRespOther: 0 | |||
NoFreshCache: 0 | |||
: | |||
</pre> | |||
Now fetch a tile<br> | |||
http://yourserver/tiles/0/0/0.png<br> | |||
If things dont work, check: | |||
*Can renderd write to /var/lib/mod_tile and /var/run/renderd? Have files been created there? | |||
*Check apache's error log<br> | |||
*Is font path correct in renderd.conf | |||
*Any activity in the renderd console window?<br> | |||
=== Starting Automatically<br> === | |||
I inserted this in /etc/rc.local<br> | |||
<pre>mkdir /var/run/renderd | |||
chown rick /var/run/renderd | |||
sudo -u rick /home/rick/src/mod_tile/renderd | |||
</pre> | |||
=== Adding a map to view tiles in a browser<br> === | |||
Download and install OpenLayers: | |||
<pre>cd ~ | <pre>cd ~ | ||
wget http://openlayers.org/download/OpenLayers-2.10.tar.gz | wget http://openlayers.org/download/OpenLayers-2.10.tar.gz | ||
Line 142: | Line 176: | ||
sudo cp -r theme /var/www | sudo cp -r theme /var/www | ||
sudo cp examples/style.css /var/www | sudo cp examples/style.css /var/www | ||
sudo cp examples/osm.html /var/www</pre> | sudo cp examples/osm.html /var/www</pre> | ||
Now customise osm.html to our site | |||
Now customise osm.html to our site | <pre>sudo vim /var/www/osm.html</pre> | ||
<pre>sudo vim /var/www/osm.html</pre> | Change the layer = line to use our server instead of open street maps. | ||
Change layer = line to use our | <pre>layer = new OpenLayers.Layer.OSM("RickTiles", "/tiles/${z}/${x}/${y}.png", {numZoomLevels: 19});</pre> | ||
<pre>layer = new OpenLayers.Layer.OSM("RickTiles", "/tiles/${z}/${x}/${y}.png", {numZoomLevels: 19});</pre> | You can add: set layer.attribution="whatever"; if you like.<br> | ||
You can set layer.attribution="whatever"; | |||
Change the default co-ordinate to somewhere in Australia | |||
<pre>...new OpenLayers.LonLat(144.38, -38.10).transform(...</pre> | |||
Now navigate to http://yourserver/osm.html and you should have a map. | Now navigate to http://yourserver/osm.html and you should have a map. | ||
[[Category:Paint]] | [[Category:Paint]] |
Revision as of 23:16, 2 August 2011
The Planimate Map object enables the use of tiled map data in models. These notes describe the setup of a map tile web server which can be used with Planimate or other open Street Maps projects.
The bulk of the install is described here and credit to the author for this guide, which gets you from a bare linux install to locally creating map tiles.
Setting up the VM
I created the VM by pointing VMWare to a Ubuntu 10.04 ISO (ubuntu-10.04-server-amd64.iso) in its new VM wizard. I initially set 3.5GB of RAM, 20GB HDD and bridged networking. Selected defaults for the rest.. In a few minutes I was at a login prompt.
This was a first test, only for the Australia region. In the end, about 6GB of space has been used (not including tiles). With the current growth of the world osm file, you'll need 600GB of disk space.
I gave the VM plenty of RAM for the osm2pgsl import stage (see below). Since then I've shrunk the VM's RAM configuration to 1.8GB which is more than plenty for a small scale test server.
You'll need to know the IP address of your server. Running 'ifconfig' should tell you this and the "eth" device being used. If you want to give your server a static IP address, edit /etc/network/interfaces, substituting your network details.
sudo vim /etc/network/interfaces iface eth0 inet static address 192.168.0.18 netmask 255.255.255.0 gateway 192.168.0.100
After this execute
sudo ifdown eth0; ifup eth0
You can then use a terminal like PuTTY to log in.
Installing PostGIS, Mapnik
Follow the guide linked above. Its useful to ssh in and copy/paste in a line at a time. The guide is pretty well spot on except these:
- I found I had to install 'unzip', might as well do that first
sudo apt-get install unzip
- Later in the guide it says to do this:
psql -f /usr/share/postgresql/8.4/contrib/postgis-1.5/postgis.sql -d gis
This directory didn't exist so instead I did:
psql -f /usr/share/postgresql/8.4/contrib/postgis.sql -d gis
If you read the article's comments, you'll find this is how it used to be.
- I only downloaded Australian data, so my import was different. Also notice the -C parameter. This is the amount of RAM the import uses for caching, you might want to reduce it if space is tight.
cd ~/bin/osm2pgsql ./osm2pgsql -S default.style --slim -d gis -C 2048 ~/planet/australia.osm.bz2
It took about 15 minutes. I continued with the mapnik installation while the database ground away. Take care copying those long URLs in the prepared data section.
The guide got me to generating a map from the command line with little fuss.
Create an OSM file for your database
This file tells renderd about your database. I also edited some of the other py files like render_tiles.py to point to "my_osm.py".
cd ~/bin/mapnik ./generate_xml.py osm.xml my_osm.xml --dbname gis --symbols ./symbols/ --world_boundaries ./world_boundaries/ --user rick --accept-none
Web Server
The module uses a multithreaded version of apache.
sudo apt-get install apache2 apache2-threaded-dev
Build Apache Tile Module & renderd
Make sure you are the user that can access the DB, not root.
cd ~/src svn co http://svn.openstreetmap.org/applications/utils/mod_tile/ cd mod_tile make sudo make install sudo mkdir /var/lib/mod_tile sudo chown rick /var/lib/mod_tile touch /var/lib/mod_tile/planet-import-complete sudo mkdir /var/run/renderd sudo chown rick /var/run/renderd
(if directories already exist, make sure they have write access to the user).
Edit renderd configuration
The make install above copied /etc/renderd.conf from the src/mod_tile directory. In this setup, both Apache and the renderd program use this configuration file.
sudo vim /etc/renderd.conf
Edit as follows, adjust the home directory as appropriate:
[renderd] socketname=/var/run/renderd/renderd.sock num_threads=4 tile_dir=/var/lib/mod_tile ; DOES NOT WORK YET stats_file=/var/run/renderd/renderd.stats [mapnik] plugins_dir=/usr/local/lib/mapnik/input font_dir=/home/rick/src/mapnik/fonts font_dir_recurse=1 [default] URI=/tiles/ XML=/home/rick/bin/mapnik/my_osm.xml #HTCPHOST=proxy.openstreetmap.org
You can add alternate tilesets/layer configurations by adding another section eg:
[tiles2] URI=/tiles2/ XML=/home/rick/bin/mapnik/my_osm2.xml
In my test I copied my_osm.xml to my_osm2.xml and disabled some layers by adding attribute status="off" to the Layer elements in the second half of the file.
Set up Apache configuration
Copy the configuration file for mod_tile then edit it:
cd ~/src/mod_tile sudo cp mod_tile.conf /etc/apache2/mods-available sudo vim /etc/apache2/mods-available/mod_tile.conf
The changes are in the first few lines:
- comment this out, we'll load it separately
#LoadModule tile_module modules/mod_tile.so
- change the servername and comment out the alias line
ServerName maps
- update the document root
DocumentRoot /var/www/
- enable the use of the config file in /etc
LoadTileConfigFile /etc/renderd.conf
Thats it for the edits.
Now create a load file for the module and create the links in mods-enabled.
sudo su root echo LoadModule tile_module /usr/lib/apache2/modules/mod_tile.so > /etc/apache2/mods-available/mod_tile.load ln -s /etc/apache2/mods-available/mod_tile.load /etc/apache2/mods-enabled/mod_tile.load ln -s /etc/apache2/mods-available/mod_tile.conf /etc/apache2/mods-enabled/mod_tile.conf exit
You can now restart apache and check the module is loaded:
apache2ctl restart
Start renderd
For debugging, do this in a separate window - as regular user, not root.
~/src/mod_tile/renderd -f
You should see activity as tiles are requested. You'd start a single instance in the background using
~/src/mod_tile/renderd
or multiple instances (for each thread) using
~/src/mod_tile/renderd.py &
Testing
From a web browser, navigate to the address of your new server,
You should see something like:
NoResp200: 0 NoResp304: 0 NoResp404: 0 NoResp503: 0 NoResp5XX: 0 NoRespOther: 0 NoFreshCache: 0 :
Now fetch a tile
http://yourserver/tiles/0/0/0.png
If things dont work, check:
- Can renderd write to /var/lib/mod_tile and /var/run/renderd? Have files been created there?
- Check apache's error log
- Is font path correct in renderd.conf
- Any activity in the renderd console window?
Starting Automatically
I inserted this in /etc/rc.local
mkdir /var/run/renderd chown rick /var/run/renderd sudo -u rick /home/rick/src/mod_tile/renderd
Adding a map to view tiles in a browser
Download and install OpenLayers:
cd ~ wget http://openlayers.org/download/OpenLayers-2.10.tar.gz tar -xzvf OpenLayers-2.10.tar.gz cd OpenLayers-2.10 sudo cp OpenLayers.js /var/www sudo cp -r style /var/www sudo cp -r theme /var/www sudo cp examples/style.css /var/www sudo cp examples/osm.html /var/www
Now customise osm.html to our site
sudo vim /var/www/osm.html
Change the layer = line to use our server instead of open street maps.
layer = new OpenLayers.Layer.OSM("RickTiles", "/tiles/${z}/${x}/${y}.png", {numZoomLevels: 19});
You can add: set layer.attribution="whatever"; if you like.
Change the default co-ordinate to somewhere in Australia
...new OpenLayers.LonLat(144.38, -38.10).transform(...
Now navigate to http://yourserver/osm.html and you should have a map.