Tile Server (2011)
The Planimate Map object enables the use of tiled map data in models. These notes describe the setup of a map tile web server which can be used with Planimate or other open Street Maps projects.
The bulk of the install is described here and credit to the author for this guide, which gets you from a bare linux install to locally creating map tiles.
Setting up the VM
I created the VM by pointing VMWare to a Ubuntu 10.04 ISO (ubuntu-10.04-server-amd64.iso) in its new VM wizard. I initially set 3.5GB of RAM, 20GB HDD and bridged networking. Selected defaults for the rest.. In a few minutes I was at a login prompt.
This was a first test, only for the Australia region. In the end, about 6GB of space has been used (not including tiles). With the current growth of the world osm file, you'll need 600GB of disk space.
I gave the VM plenty of RAM for the osm2pgsl import stage (see below). Since then I've shrunk the VM's RAM configuration to 1.8GB which is more than plenty for a small scale test server.
You'll need to know the IP address of your server. Running 'ifconfig' should tell you this and the "eth" device being used. If you want to give your server a static IP address, edit /etc/network/interfaces, substituting your network details.
sudo vim /etc/network/interfaces iface eth0 inet static address 192.168.0.18 netmask 255.255.255.0 gateway 192.168.0.100
After this execute
sudo ifdown eth0; ifup eth0
You can then use a terminal like PuTTY to log in.
Update operating system
sudo apt-get update sudo apt-get upgrade
Get some system tools
We'll need subversion to get the latest updates from OpenStreetMap and other places. Munin makes pretty pictures of activity on the server. I like screen. htop is neat-o.
sudo apt-get install subversion autoconf screen munin-node munin htop unzip
Organize the file system a bit
cd ~ mkdir src bin planet
Get the latest "Planet" - OpenStreetMap data file
new "planet" file is published approximately each week. The mirror and archives of the "planet" files are here. In March 2010 the planet file was about 8.2GB in length. If you are not interested in the entire planet you can choose to download an extract file instead.
Facing an 8.2GB or larger download, this is an excellent time to consider using screen if you haven't used it before. Screen allows you to operate several terminal windows through one ssh connection. And more. Have a look this screen tutorial if you haven't used it before. Or wait for your download to complete, or use another ssh session to continue.
cd planet wget http://planet.openstreetmap.org/planet-latest.osm.bz2
Prepare the postGIS database
Use the PostGIS extensions to postgresql for all sorts of geographical goodness. Install the postGIS and prerequisites.
sudo apt-get install postgresql-8.4-postgis postgresql-contrib-8.4 sudo apt-get install postgresql-server-dev-8.4 sudo apt-get install build-essential libxml2-dev libtool sudo apt-get install libgeos-dev libpq-dev libbz2-dev proj
Install osm2pgsql from the repository
The latest version of osm2pgsql has the most goodies, so we'll use that rather than a package.
cd ~/bin svn co http://svn.openstreetmap.org/applications/utils/export/osm2pgsql/ cd osm2pgsql ./autogen.sh ./configure make
Configure the PostGIS database
edit /etc/postgresql/8.4/main/postgresql.conf in four places. These changes help with the large quantities of data that we are using.
shared_buffers = 128MB # 16384 for 8.1 and earlier checkpoint_segments = 20 maintenance_work_mem = 256MB # 256000 for 8.1 and earlier autovacuum = off
Edit kernel parameter shmmax to increase maximum size of shared memory.
sudo sysctl -w kernel.shmmax=268435456 sudo sysctl -p /etc/sysctl.conf
Restart postgres to enable the changes
sudo /etc/init.d/postgresql-8.4 restart
It should restart as above.
* Restarting PostgreSQL 8.4 database server
...done.
Create a database called "gis". Some of our future tools presume that you will use this database name. Substitute your username for "username" in two places below. This should be the username that will render maps with mapnik.
sudo -u postgres -i createuser username # answer yes for superuser createdb -E UTF8 -O username gis createlang plpgsql gis exit
Set up PostGIS on the postresql database.
psql -f /usr/share/postgresql/8.4/contrib/postgis.sql -d gis
This should respond with many lines ending with
... CREATE FUNCTION COMMIT ... DROP FUNCTION
Note: recent change requiring a new path to postgis.sql, is now reflected above.
Substitute your username for "username" in two places in the next line. This should be the username that will render maps with mapnik.
echo "ALTER TABLE geometry_columns OWNER TO username; ALTER TABLE spatial_ref_sys OWNER TO username;" | psql -d gis
- Should reply with
ALTER TABLE ALTER TABLE
Set the Spatial Reference Identifier (SRID) on the new database.
psql -f ~/bin/osm2pgsql/900913.sql -d gis
Should reply with
INSERT 0 1
Installing PostGIS, Mapnik
Follow the guide linked above after reading the amendments below. Its useful to ssh in and copy/paste in a line at a time.
- I only downloaded Australian data, so my import was different. Also notice the -C parameter. This is the amount of RAM the import uses for caching, you might want to reduce it if your VM has less than 3GB of RAM.
cd ~/bin/osm2pgsql ./osm2pgsql -S default.style --slim -d gis -C 2048 ~/planet/australia.osm.bz2
It took about 15 minutes. I actually put the bz2 file on a different drive to reduce thrashing. I continued with the mapnik installation while the database ground away. Take care copying those long URLs in the prepared data section.
Final amendment - before the last step of running ./generate_image.py, perform these steps:
- Customise generate_image.py to use the XML file we'll create next. Mapnick has evolved and now uses templates. I also edited some of the other py files, like generate_tiles.py, but this isn't necessary. For generate_image.py, the changes are:
mapfile = "my_osm.xml" bounds = (144.37, -38.0, 144.39, -38.2)
- Create a customised OSM file.
This file tells mapnik (and renderd) about your data:
cd ~/bin/mapnik ./generate_xml.py osm.xml my_osm.xml --dbname gis --symbols ./symbols/ --world_boundaries ./world_boundaries/ --user rick --accept-none
Running ./generate_image.py now should create image.png showing a region in Victoria.
Web Server
Now we set up the tile server. The mod_tile module uses a multithreaded version of apache.
sudo apt-get install apache2 apache2-threaded-dev
Build Apache Tile Module & renderd
Make sure you are the user that can access the DB, not root.
cd ~/src svn co http://svn.openstreetmap.org/applications/utils/mod_tile/ cd mod_tile make sudo make install sudo mkdir /var/lib/mod_tile sudo chown rick /var/lib/mod_tile touch /var/lib/mod_tile/planet-import-complete sudo mkdir /var/run/renderd sudo chown rick /var/run/renderd
(if directories already exist, make sure they have write access to the user).
Edit renderd configuration
The make install above copied /etc/renderd.conf from the src/mod_tile directory. In this setup, both Apache and the renderd program use this configuration file.
sudo vim /etc/renderd.conf
Edit as follows, adjust the home directory as appropriate:
[renderd] socketname=/var/run/renderd/renderd.sock num_threads=4 tile_dir=/var/lib/mod_tile ; DOES NOT WORK YET stats_file=/var/run/renderd/renderd.stats [mapnik] plugins_dir=/usr/local/lib/mapnik/input font_dir=/home/rick/src/mapnik/fonts font_dir_recurse=1 [default] URI=/tiles/ XML=/home/rick/bin/mapnik/my_osm.xml #HTCPHOST=proxy.openstreetmap.org
TIP: You can add alternate tilesets/layer configurations by adding another section eg:
[tiles2] URI=/tiles2/ XML=/home/rick/bin/mapnik/my_osm2.xml
In my test I copied my_osm.xml to my_osm2.xml and disabled some layers by adding attribute status="off" to the Layer elements in the second half of the file.
Set up Apache configuration
Copy the configuration file for mod_tile then edit it:
cd ~/src/mod_tile sudo cp mod_tile.conf /etc/apache2/mods-available sudo vim /etc/apache2/mods-available/mod_tile.conf
The changes are in the first few lines:
- comment this out, we'll load it separately
#LoadModule tile_module modules/mod_tile.so
- change the servername and comment out the alias line
ServerName maps
- update the document root
DocumentRoot /var/www/
- enable the use of the config file in /etc
LoadTileConfigFile /etc/renderd.conf
Thats it for the edits.
Now create a load file for the module and create the links in mods-enabled.
sudo su root echo LoadModule tile_module /usr/lib/apache2/modules/mod_tile.so > /etc/apache2/mods-available/mod_tile.load ln -s /etc/apache2/mods-available/mod_tile.load /etc/apache2/mods-enabled/mod_tile.load ln -s /etc/apache2/mods-available/mod_tile.conf /etc/apache2/mods-enabled/mod_tile.conf exit
You can now restart apache and check the module is loaded:
apache2ctl restart
Start renderd
For debugging, do this in a separate window - as regular user, not root.
~/src/mod_tile/renderd -f
We put this (without the -f) into rc.local later.
Testing
From a web browser, navigate to the address of your new server,
You should see something like:
NoResp200: 0 NoResp304: 0 NoResp404: 0 NoResp503: 0 NoResp5XX: 0 NoRespOther: 0 NoFreshCache: 0 :
Now fetch a tile
http://yourserver/tiles/0/0/0.png
If things dont work, check:
- Can renderd write to /var/lib/mod_tile and /var/run/renderd? Have files been created there?
- Check apache's error log
- Is font path correct in renderd.conf
- Any activity in the renderd console window?
Starting Automatically
I inserted this in /etc/rc.local
mkdir /var/run/renderd chown rick /var/run/renderd sudo -u rick /home/rick/src/mod_tile/renderd
Adding a map to view tiles in a browser
Download and install OpenLayers:
cd ~ wget http://openlayers.org/download/OpenLayers-2.10.tar.gz tar -xzvf OpenLayers-2.10.tar.gz cd OpenLayers-2.10 sudo cp OpenLayers.js /var/www sudo cp -r style /var/www sudo cp -r theme /var/www sudo cp examples/style.css /var/www sudo cp examples/osm.html /var/www
Now customise osm.html to our site
sudo vim /var/www/osm.html
Change the layer = line to use our server instead of open street maps.
layer = new OpenLayers.Layer.OSM("RickTiles", "/tiles/${z}/${x}/${y}.png", {numZoomLevels: 19});
You can add: set layer.attribution="whatever"; if you like.
Change the default co-ordinate to somewhere in Australia
...new OpenLayers.LonLat(144.38, -38.10).transform(...
Now navigate to http://yourserver/osm.html and you should have a map.