Tile Server (2011): Difference between revisions

From Planimate Knowledge Base
Jump to navigation Jump to search
No edit summary
No edit summary
Line 1: Line 1:
The Planimate [[Map (Paint Object)|Map]] object enables the use of tiled map data in models. These notes describe the setup of a map tile web server which can be used with Planimate or other open Street Maps projects.<br>  
The Planimate [[Map (Paint Object)|Map]] object enables the use of tiled map data in models. These notes describe the setup of a map tile web server which can be used with Planimate or other open Street Maps projects.<br>


The bulk of the install is described [http://weait.com/content/build-your-own-openstreetmap-server here] and credit to the author for this guide, which gets you from a bare linux install to locally creating map tiles.<br>  
The bulk of the install is described [http://weait.com/content/build-your-own-openstreetmap-server here] and credit to the author for this guide, which gets you from a bare linux install to locally creating map tiles.<br>


=== Setting up the VM  ===
=== Setting up the VM  ===


I created the VM by pointing VMWare to a Ubuntu 10.04 ISO (ubuntu-10.04-server-amd64.iso) in its new VM&nbsp;wizard. I initially set 3.5GB of RAM, 20GB HDD and bridged networking. Selected defaults for the rest.. In a few minutes I was at a login prompt.  
I created the VM by pointing VMWare to a Ubuntu 10.04 ISO (ubuntu-10.04-server-amd64.iso) in its new VM&nbsp;wizard. I initially set 3.5GB of RAM, 20GB HDD and bridged networking. Selected defaults for the rest.. In a few minutes I was at a login prompt.


This was a first test, only for the Australia region. In the end, about 6GB of space has been used. With the current growth of the world osm file, you'll probably need 1TB of disk space.
This was a first test, only for the Australia region. In the end, about 6GB of space has been used. With the current growth of the world osm file, you'll probably need 1TB of disk space.


I gave the VM plenty of RAM&nbsp;for the osm2pgsl import stage (see below). Since then I've shrunk the VM's RAM configuration to 1.8GB which is plenty for a small scale test server.  
I gave the VM plenty of RAM&nbsp;for the osm2pgsl import stage (see below). Since then I've shrunk the VM's RAM configuration to 1.8GB which is plenty for a small scale test server.


=== Installing PostGIS, Mapnik  ===
=== Installing PostGIS, Mapnik  ===


Follow the guide linked above. Its useful to ssh in and copy/paste in a line at a time. I used PUTTY as the terminal. The guide is pretty well spot on except these:  
Follow the guide linked above. Its useful to ssh in and copy/paste in a line at a time. I used PUTTY as the terminal. The guide is pretty well spot on except these:


*I found I had to install 'unzip', might as well do that first
*I found I had to install 'unzip', might as well do that first
<pre>sudo apt-get install unzip</pre>  
<pre>sudo apt-get install unzip</pre>
*Later in the guide it says to do this:
*Later in the guide it says to do this:
<pre>psql -f /usr/share/postgresql/8.4/contrib/postgis-1.5/postgis.sql -d gis</pre>  
<pre>psql -f /usr/share/postgresql/8.4/contrib/postgis-1.5/postgis.sql -d gis</pre>
This directory didn't exist so instead I did:  
This directory didn't exist so instead I did:
<pre>psql -f /usr/share/postgresql/8.4/contrib/postgis.sql -d gis</pre>  
<pre>psql -f /usr/share/postgresql/8.4/contrib/postgis.sql -d gis</pre>
If you read the article's comments, you'll find this is how it used to be.  
If you read the article's comments, you'll find this is how it used to be.


*I only downloaded Australian data, so my import was different. Also notice the -C parameter. This is the amount of RAM the import uses for caching, you might want to reduce it if space is tight.<br>
*I only downloaded Australian data, so my import was different. Also notice the -C parameter. This is the amount of RAM the import uses for caching, you might want to reduce it if space is tight.<br>
<pre>cd ~/bin/osm2pgsql
<pre>cd ~/bin/osm2pgsql
./osm2pgsql -S default.style --slim -d gis -C 2048 ~/planet/australia.osm.bz2
./osm2pgsql -S default.style --slim -d gis -C 2048 ~/planet/australia.osm.bz2
</pre>  
</pre>
It took about 15 minutes. I continued with the mapnik installation while the database ground away. Take care copying those long URLs in the prepared data section.  
It took about 15 minutes. I continued with the mapnik installation while the database ground away. Take care copying those long URLs in the prepared data section.


The guide got me to generating a map from the command line with little fuss.  
The guide got me to generating a map from the command line with little fuss.


=== Create an OSM file for your database  ===
=== Create an OSM file for your database  ===


This file tells renderd about your database. I&nbsp;also edited some of the other py files like render_tiles.py to point to "my_osm.py".  
This file tells renderd about your database. I&nbsp;also edited some of the other py files like render_tiles.py to point to "my_osm.py".
<pre>cd ~/bin/mapnik
<pre>cd ~/bin/mapnik
./generate_xml.py osm.xml my_osm.xml --dbname gis --symbols ./symbols/ --world_boundaries ./world_boundaries/ --user rick --accept-none</pre>  
./generate_xml.py osm.xml my_osm.xml --dbname gis --symbols ./symbols/ --world_boundaries ./world_boundaries/ --user rick --accept-none</pre>
=== Web Server  ===
=== Web Server  ===


The module uses a multithreaded version of apache.  
The module uses a multithreaded version of apache.
<pre>sudo apt-get install apache2 apache2-threaded-dev</pre>  
<pre>sudo apt-get install apache2 apache2-threaded-dev</pre>
=== Build Apache Tile Module &amp;&nbsp;renderd  ===
=== Build Apache Tile Module &amp;&nbsp;renderd  ===


Make sure you are the user that can access the DB, not root.<br>  
Make sure you are the user that can access the DB, not root.<br>
<pre>cd ~/src
<pre>cd ~/src
svn co http://svn.openstreetmap.org/applications/utils/mod_tile/
svn co http://svn.openstreetmap.org/applications/utils/mod_tile/
Line 50: Line 50:
sudo mkdir /var/lib/mod_tile
sudo mkdir /var/lib/mod_tile
sudo chown rick /var/lib/mod_tile
sudo chown rick /var/lib/mod_tile
touch /var/lib/mod_tile/planet-import-complete
sudo mkdir /var/run/renderd
sudo mkdir /var/run/renderd
sudo chown rick /var/run/renderd
sudo chown rick /var/run/renderd


</pre>  
</pre>
(if directories already exist, make sure they have write access to the user).  
(if directories already exist, make sure they have write access to the user).


=== Edit renderd configuration  ===
=== Edit renderd configuration  ===


The make install above copied /etc/renderd.conf from the src directory. In this setup, both Apache and the renderd program use this configuration file.  
The make install above copied /etc/renderd.conf from the src directory. In this setup, both Apache and the renderd program use this configuration file.
<pre>sudo vim /etc/renderd.conf</pre>  
<pre>sudo vim /etc/renderd.conf</pre>
Edit as follows, adjust the home directory as appropriate:  
Edit as follows, adjust the home directory as appropriate:
<pre>[renderd]
<pre>[renderd]
socketname=/var/run/renderd/renderd.sock
socketname=/var/run/renderd/renderd.sock
Line 75: Line 76:
URI=/tiles/
URI=/tiles/
XML=/home/rick/bin/mapnik/my_osm.xml
XML=/home/rick/bin/mapnik/my_osm.xml
#HTCPHOST=proxy.openstreetmap.org</pre>  
#HTCPHOST=proxy.openstreetmap.org</pre>
You can add alternate tilesets/layer configurations by adding another section eg:
<pre>[tiles2]
URI=/tiles2/
XML=/home/rick/bin/mapnik/my_osm2.xml</pre>
 
In my test I copied my_osm.xml to my_osm2.xml and disabled some layers, like this:
 
=== Set up Apache configuration  ===
=== Set up Apache configuration  ===


Copy the configuration file for mod_tile then edit it:  
Copy the configuration file for mod_tile then edit it:
<pre>cd ~/src/mod_tile
<pre>cd ~/src/mod_tile
sudo cp mod_tile.conf /etc/apache2/mods-available
sudo cp mod_tile.conf /etc/apache2/mods-available
sudo vim /etc/apache2/mods-available/mod_tile.conf</pre>  
sudo vim /etc/apache2/mods-available/mod_tile.conf</pre>
The changes are in the first few lines:  
The changes are in the first few lines:


*comment this out, we'll load it separately
*comment this out, we'll load it separately
<pre>#LoadModule tile_module modules/mod_tile.so</pre>  
<pre>#LoadModule tile_module modules/mod_tile.so</pre>
*change the servername and comment out the alias line
*change the servername and comment out the alias line
<pre>ServerName maps</pre>  
<pre>ServerName maps</pre>
*update the document root
*update the document root
<pre>DocumentRoot /var/www/</pre>  
<pre>DocumentRoot /var/www/</pre>
*enable the use of the config file in /etc
*enable the use of the config file in /etc
<pre>LoadTileConfigFile /etc/renderd.conf</pre>  
<pre>LoadTileConfigFile /etc/renderd.conf</pre>
Thats it for the edits.  
Thats it for the edits.


Now create a load file for the module and create the links in mods-enabled.  
Now create a load file for the module and create the links in mods-enabled.
<pre>sudo su root
<pre>sudo su root
echo LoadModule tile_module /usr/lib/apache2/modules/mod_tile.so &gt; /etc/apache2/mods-available/mod_tile.load
echo LoadModule tile_module /usr/lib/apache2/modules/mod_tile.so &gt; /etc/apache2/mods-available/mod_tile.load
Line 100: Line 108:
ln -s /etc/apache2/mods-available/mod_tile.conf /etc/apache2/mods-enabled/mod_tile.conf
ln -s /etc/apache2/mods-available/mod_tile.conf /etc/apache2/mods-enabled/mod_tile.conf
exit
exit
</pre>  
</pre>
You can now restart apache and check the module is loaded:  
You can now restart apache and check the module is loaded:
<pre>apache2ctl restart</pre>  
<pre>apache2ctl restart</pre>
=== Start renderd  ===
=== Start renderd  ===


For debugging, do this in a separate window - as regular user, not root.  
For debugging, do this in a separate window - as regular user, not root.
<pre>~/src/mod_tile/renderd -f</pre>  
<pre>~/src/mod_tile/renderd -f</pre>
You should see activity as tiles are requested. You'd start a single instance in the background using  
You should see activity as tiles are requested. You'd start a single instance in the background using
<pre>~/src/mod_tile/renderd</pre>  
<pre>~/src/mod_tile/renderd</pre>
or multiple instances (for each thread) using  
or multiple instances (for each thread) using
<pre> ~/src/mod_tile/renderd.py &amp;</pre>  
<pre> ~/src/mod_tile/renderd.py &amp;</pre>
=== Testing  ===
=== Testing  ===


From a web browser, navigate to the IP of your VM  
From a web browser, navigate to the IP of your VM
<pre>http://192.168.0.151/mod_tile</pre>  
<pre>http://192.168.0.151/mod_tile</pre>
Now fetch a tile  
Now fetch a tile
<pre>http://192.168.0.151/tiles/0/0/0.png</pre>  
<pre>http://192.168.0.151/tiles/0/0/0.png</pre>
If things dont work, check:  
If things dont work, check:


*Can renderd write to /var/lib/mod_tile and /bar/run/renderd  
*Can renderd write to /var/lib/mod_tile and /bar/run/renderd
*Apache's error log<br>  
*Apache's error log<br>
*Is font path correct in renderd.conf
*Is font path correct in renderd.conf


[[Category:Paint]]
[[Category:Paint]]

Revision as of 04:25, 2 August 2011

The Planimate Map object enables the use of tiled map data in models. These notes describe the setup of a map tile web server which can be used with Planimate or other open Street Maps projects.

The bulk of the install is described here and credit to the author for this guide, which gets you from a bare linux install to locally creating map tiles.

Setting up the VM

I created the VM by pointing VMWare to a Ubuntu 10.04 ISO (ubuntu-10.04-server-amd64.iso) in its new VM wizard. I initially set 3.5GB of RAM, 20GB HDD and bridged networking. Selected defaults for the rest.. In a few minutes I was at a login prompt.

This was a first test, only for the Australia region. In the end, about 6GB of space has been used. With the current growth of the world osm file, you'll probably need 1TB of disk space.

I gave the VM plenty of RAM for the osm2pgsl import stage (see below). Since then I've shrunk the VM's RAM configuration to 1.8GB which is plenty for a small scale test server.

Installing PostGIS, Mapnik

Follow the guide linked above. Its useful to ssh in and copy/paste in a line at a time. I used PUTTY as the terminal. The guide is pretty well spot on except these:

  • I found I had to install 'unzip', might as well do that first
sudo apt-get install unzip
  • Later in the guide it says to do this:
psql -f /usr/share/postgresql/8.4/contrib/postgis-1.5/postgis.sql -d gis

This directory didn't exist so instead I did:

psql -f /usr/share/postgresql/8.4/contrib/postgis.sql -d gis

If you read the article's comments, you'll find this is how it used to be.

  • I only downloaded Australian data, so my import was different. Also notice the -C parameter. This is the amount of RAM the import uses for caching, you might want to reduce it if space is tight.
cd ~/bin/osm2pgsql
./osm2pgsql -S default.style --slim -d gis -C 2048 ~/planet/australia.osm.bz2

It took about 15 minutes. I continued with the mapnik installation while the database ground away. Take care copying those long URLs in the prepared data section.

The guide got me to generating a map from the command line with little fuss.

Create an OSM file for your database

This file tells renderd about your database. I also edited some of the other py files like render_tiles.py to point to "my_osm.py".

cd ~/bin/mapnik
./generate_xml.py osm.xml my_osm.xml --dbname gis --symbols ./symbols/ --world_boundaries ./world_boundaries/ --user rick --accept-none

Web Server

The module uses a multithreaded version of apache.

sudo apt-get install apache2 apache2-threaded-dev

Build Apache Tile Module & renderd

Make sure you are the user that can access the DB, not root.

cd ~/src
svn co http://svn.openstreetmap.org/applications/utils/mod_tile/
cd mod_tile
make
sudo make install
sudo mkdir /var/lib/mod_tile
sudo chown rick /var/lib/mod_tile
touch /var/lib/mod_tile/planet-import-complete
sudo mkdir /var/run/renderd
sudo chown rick /var/run/renderd

(if directories already exist, make sure they have write access to the user).

Edit renderd configuration

The make install above copied /etc/renderd.conf from the src directory. In this setup, both Apache and the renderd program use this configuration file.

sudo vim /etc/renderd.conf

Edit as follows, adjust the home directory as appropriate:

[renderd]
socketname=/var/run/renderd/renderd.sock
num_threads=4
tile_dir=/var/lib/mod_tile ; DOES NOT WORK YET
stats_file=/var/run/renderd/renderd.stats

[mapnik]
plugins_dir=/usr/local/lib/mapnik/input
font_dir=/home/rick/src/mapnik/fonts
font_dir_recurse=1

[default]
URI=/tiles/
XML=/home/rick/bin/mapnik/my_osm.xml
#HTCPHOST=proxy.openstreetmap.org

You can add alternate tilesets/layer configurations by adding another section eg:

[tiles2]
URI=/tiles2/
XML=/home/rick/bin/mapnik/my_osm2.xml

In my test I copied my_osm.xml to my_osm2.xml and disabled some layers, like this:

Set up Apache configuration

Copy the configuration file for mod_tile then edit it:

cd ~/src/mod_tile
sudo cp mod_tile.conf /etc/apache2/mods-available
sudo vim /etc/apache2/mods-available/mod_tile.conf

The changes are in the first few lines:

  • comment this out, we'll load it separately
#LoadModule tile_module modules/mod_tile.so
  • change the servername and comment out the alias line
ServerName maps
  • update the document root
DocumentRoot /var/www/
  • enable the use of the config file in /etc
LoadTileConfigFile /etc/renderd.conf

Thats it for the edits.

Now create a load file for the module and create the links in mods-enabled.

sudo su root
echo LoadModule tile_module /usr/lib/apache2/modules/mod_tile.so > /etc/apache2/mods-available/mod_tile.load
ln -s /etc/apache2/mods-available/mod_tile.load /etc/apache2/mods-enabled/mod_tile.load
ln -s /etc/apache2/mods-available/mod_tile.conf /etc/apache2/mods-enabled/mod_tile.conf
exit

You can now restart apache and check the module is loaded:

apache2ctl restart

Start renderd

For debugging, do this in a separate window - as regular user, not root.

~/src/mod_tile/renderd -f

You should see activity as tiles are requested. You'd start a single instance in the background using

~/src/mod_tile/renderd

or multiple instances (for each thread) using

 ~/src/mod_tile/renderd.py &

Testing

From a web browser, navigate to the IP of your VM

http://192.168.0.151/mod_tile

Now fetch a tile

http://192.168.0.151/tiles/0/0/0.png

If things dont work, check:

  • Can renderd write to /var/lib/mod_tile and /bar/run/renderd
  • Apache's error log
  • Is font path correct in renderd.conf