Fast routes of OSRM just got faster…

One of the many bottle necks of today’s web services are network latency and bandwidth. While I was working on a research paper, I recognized that G**gle encodes some information when you calculate a route on G**gle Maps. This process reduces, besides the gzip compression, the response from the server. This means that this is a speed improvement for the server client communication besides the zoom-level-generalization. You can read more about the “Encoded Polyline Algorithm Format” and how it works here.

We integrated this nice feature into the code of the Open Source Routing Machine (OSRM) project. The following table shows a few results for some sample routes comparing the old and the new file sizes (@ zoomlevel 18):

And if you are not a numbers person, let us put this in perspective: The fast routes just got faster! And that’s not all. As a second new feature the web frontend of the OSRM has shortlinks for the calculated route now. You can find the link of your route at the route summary. Try it out here: http://map.project-osrm.org/

thx @ maɪˈæmɪ Dennis 🙂

“My Way” to cross Dublin without a Pub

First of all, I really like the following blog post and the idea behind it: “Yes! It is possible to cross Dublin without passing a pub

It shows the power of crowd sourced geodata (OpenStreetMap) and the skills of some individuals. In the following steps I am going to show you a different way to get to the same result.

The website OpenRouteService.org offers the same functionality to calculate a route. You can avoid certain areas by defining them (just simply draw them) on the map. However, your first step would be to “Search for Points of Interest (POI)” (PUBS), for example in Dublin within a distance of 10 km. The following picture shows the result:

After that you can create several polygons around your pubs resulting in a map with several, “red” areas:

And finally you can calculate a route from almost any point in Dublin. The last image show such a route without crossing a red area or Pub:

However, the original blog post is brilliant !
Or do you want it the opposite way: Search all pubs *along* my route? 😉

thx @ maɪˈæmɪ Dennis 🙂

OSM meets Academy for Crisis Management Emergency Planning and Civil Protection

On Saturday, May 28th members of the OpenStreetMap (OSM) community and employees of the Federal Office of Civil Protection and Disaster Assistance (in German “Bundesamt für Bevölkerungsschutz und Katastrophenhilfe (BBK)”) met for a workshop at Bad Neuenahr-Ahrweiler.

Source: Haiti Crisis Map - OpenStreetMap NL (http://haiti.openstreetmap.nl/)

The main goal of this event, taking place at the Academy for Crisis Management Emergency Planning and Civil Protection (in German “Akademie für Krisenmanagement, Notfallplanung und Zivilschutz (AKNZ)”), was to inform active contributors of the OpenStreetMap community and employees of AKNZ and BBK about OpenStreetMap and crisis mapping.

During the devastating earthquake in Haiti (2010) OSM proved to be a valuable source of up-to-date geodata and partner for international relief organizations. Crisis-Mapping includes but is not limited to e.g. the determination of infrastructure, collapsed buildings, spontaneous camps and more in crisis areas, which can be an important contribution to the orientation of rescue workers on-site. This has also been documented in specific reports by the mass media.

The workshop had 20 participants and started with a reception and short introduction about the AKNZ by Dieter Franke. Subsequently Dr. Michael Judex (BBK) explained the specific tasks of the BBK and highlighted the importance of geoinformation for the protection of citizens or during crisis management.

Stephan Bialonski started the OSM-part during this event with his presentation. Next to some basic information (Main goal of the OSM project, data model, tagging, license) he also introduced the participants to the way the OSM community thinks and works, since these factors differ highly from the working processes of regular administrative and governmental agencies. He specifically mentioned to the participants from the BBK that voluntary active contributors can be best included in their projects as long as the goal of the project is also a goal of the OSM-community.

During the following presentation Werner Kathe introduced the participants to the Humanitarian OSM Team (H.O.T.). “It is a new initiative to apply the principals and activities of Open Source and free data on humanitarian purposes and economic growth”

Special attention had been given to the presentation by Pascal Neis who explained how the operation sequence and processes during two well known OSM-relief operations (Haiti 2010 and Japan 2011) took place. Certain advantages and disadvantages considering the usage of OSM-data during these specific catastrophes were mentioned during this presentation. The participation of the OSM-community, the data collection increase in different crisis areas and several applications based on OSM-data were demonstrated. At the end Pascal showed with an example application how OSM can also be used for preventive crisis planning and organization of rescue forces.

After a lunch break including some intense conversations the employees of the BBK and AKNZ were introduced to OSM-procedures on-site. A practical workshop organized by Bernd Weigelt, Edbert van Eimeren and John Bergenholtz lead through an area of debris on the AKNZ-compound that is being used by the rescue forces for training purposes. The participants collected data about destroyed buildings and ways which shortly after have been imported to the OSM database. This way the participants were able to learn how to add GPS data, fotos taken by digital cameras and areal images to OSM using crisis-mapping specific tags in the OSM-Editors.

Trümmerstraße - Quelle: http://strdbn.posterous.com/osm-community-bonn-trifft-bbk-im-aknz-bilder

The entire event lasted for about six hours and the participants gave a very positive feedback. The open and informal atmosphere during the event allowed for several questions by the participants and showed mutual interests. We would like to thank everyone involved for the successful event.

This is a translation from the German blog post @ blog.openstreetmap.de by Stephan Bialonski & Pascal Neis. Translation provided by maɪˈæmɪ Dennis 🙂

Web-GUI for OS Routing Machine

During my Easter holidays I created a web-fronted for Dennis Luxen’s Open Source Routing Machine (OSRM Project). The OSRM project (http://project-osrm.org/) is in my opinion probably the fastest Open Source software which is using data from the OpenStreetMap project. “In contrast to most routing servers OSRM does not use an A* variant to compute shortest path, but Contraction Hierarchies.” You can read a little bit more about Contraction Hierarchies in Wikipedia.

The website that I created contains in its first version an address-search (geocoding) and of course routeplanning. For the geocoding I integrated the Nominatim search from OpenStreetMap. You can find a How-To on the MapQuest-site. Unfortunately the OSRM routing service covers only most parts of Europe for now. The current version of my web-fronted can be found here: http://map.project-osrm.org

Anyway, perhaps some of you would say: “A website with OSM routing? This is nothing new and we actually did have something like that before …” I agree, BUT I think none of them is as fast and can handle as many requests, try it out! Through its really good route calculation performance the route can also be changed or adjusted by dragging the start- and destination point (I think you know this from other websites). The following video shows the great functionality:

WebGUI for Project-OSRM from Pascal Neis

Further does the new website automatically generalize the desired route on the server side for smaller map scales. This reduces the traffic between server and client and less data needs to be visualized by the client which saves system resources. Additionally I created the following feature: If a calculated route contains a segment that has no name, it is going to be displayed in a different color. I think this is a great feature to map missed road names! The next picture shows such a route. You can find the complete sources of the website in the OSRM Sourceforge project folder.

There is a lot of discussion on OpenStreetMap.org about a Routing-Integration. In my opinion there should be two routing options available on the OSM start page. One option that provides a very fast routing such as OSRM and another option that is easy to adjust and configure and is able to work with different OSM tags. A combination of both will probably not be available soon. Bigger companies such as “G****E” offer high performance routing applications to their users. Thus it should also be a desirable goal for the new OSM software to get to the same performance level that the users are familiar with from other websites. I don’t think that an integration of ”Non-Open-Source Routing Software” such as provided by CloudMade or Mapquest is a good idea. It still should be an Open Source App after all. But this limits the selection to OSRM, YOURS, Routino and Roadeeno. For now OpenRouteService is not completely Open Source.

So *if* we add routing to the OSM main website I would vote for a fast-route-planning software e.g. by the OSRM project. Hence we must only search for a preferably configurable solution, what would be your suggestion?

thx @ Dennis L.: Continue with your great work!
thx @ Frederik (Geofabrik) for supporting the OSRM Project with a VM for hosting
thx @ *Fab* for the video
and finally: thx @ maɪˈæmɪ Dennis 🙂

Routing View Europe 2011-05

First of all, sorry that I did not create a new stat regarding the Routing View past month. To all the new readers: Usually I create an analysis about the Routing View of the OpenStreetMap Inspector for each month for Europe. You can find more information about the OSM Inspector (OSMI) here. The Routing View within the OSMI “shows problems in the data related to routing and navigation”. You can read more about it here … A direkt link to the OSMI Routing View is here!

However, here are the new stats for May, 2011: we have a total of about 124000 “Unconnected Roads” and about 108000 “Duplicate Ways” (number of duplicate segments). Overall this means that we have about 17000 *new* „Unconnected Roads” errors and only ca. 1300 “Duplicate Ways” have been fixed in Europe. For the past three months we have an increment of about 2850000 new OSM way segments for routing. (May 7th: 34500000, February 20th: 31700000, January 20th: 30600000)

In the following images you can see the amount of errors divided by country and the amount of errors in detail per country for “Europe”:

For this month only a few countries were able to reduce their errors. France (-2200) and Poland (-4800) are ahead of everyone else, so Poland this is your month 🙂 Here you can find the February stat of the OSMI Routing View. Hopefully this is going to be better in the next month :S …

thx @ maɪˈæmɪ Dennis 🙂

How did you contribute the *last couple of months* to OpenStreetMap?

I think most of you know my website: “How did you contribute to OpenStreetMap?“. It is a website where you can find information about OpenStreetMap contributors. You can see the first/last node the contributor placed and the amount of created nodes, ways, relations and other things.

Today I added a new diagram (1). It shows your monthly contributions to OpenStreetMap for the past 12 months as a chart and it’s based on the latest Full OSM History Planet Dump. In the following picture you can see the updates.

By clicking at “Get more Information …” (below the chart) you can see your past contributions in more detail. Separated into different categories such as created, touched and your “last modifier” OSM objects (nodes, ways and relations).

As a second update you can now see your first OSM way and OSM relation at the website (2). Check it out here: http://hdyc.neis-one.org

thx @ maɪˈæmɪ Dennis 🙂

Edit Stats for OSM Japan

Kate created some editing stats for OpenStreetMap Japan last Thursday. You can find her blog post here: “Quick Japan Editing Stats for OpenStreetMap”

During the creation of the layers for the “Road Status in Japan”, I log some editing information of OpenStreetMap too. As I mentioned in my blog post, I use the Geofabrik extracts for Japan (Sendai region only). They have the following bounding box (thx Frederik):

polygon
1
1.412259E+02 3.663895E+01
1.427964E+02 4.038643E+01
1.411296E+02 4.038351E+01
1.394639E+02 3.665750E+01
1.412259E+02 3.663895E+01
END
END

And here are several diagrams of the editing in Japan (Sendai region only):

In numbers (March 20th, 2011 12:50):

  • Overall amount of OSM Nodes: 5138123
  • Overall amount of OSM Ways: 149978
  • Overall amount of Highways: 47156
  • Number of Barrier Nodes: 528
  • Impassable Ways: 463
  • Number of Users (Contributors): 308
  • Length of OSM Ways [km]: 29049,71
  • Length of impassable Ways [km]: 222,58

thx @ Dennis and best of luck for tomorrow!

[Update #1 – March 25th, 2011 21.00] – I have updated all diagrams above !

  • Overall amount of OSM Nodes: 5258135
  • Overall amount of OSM Ways: 169557
  • Overall amount of Highways: 59120
  • Number of Barrier Nodes: 549
  • Impassable Ways: 801
  • Number of Users (Contributors): 414
  • Tsunami:Damage Polygons: 608
  • Length of OSM Ways [km]: 32408,75
  • Length of impassable Ways [km]: 371,67

[Update #2 – April 08th, 2011] – I have updated all diagrams above !

  • Overall amount of OSM Nodes: 6304539
  • Overall amount of OSM Ways: 271768
  • Overall amount of Highways: 150556
  • Number of Barrier Nodes: 597
  • Impassable Ways: 879
  • Number of Users (Contributors): 435
  • Tsunami:Damage Polygons: 622
  • Length of OSM Ways [km]: 58700,66
  • Length of impassable Ways [km]: 387,60

Road Status after the Earthquake in Japan based on OSM

Yesterday, Bing Maps has announced a map for Japan, which shows the “Road-Status after the Earthquake”. The information seems to be from Honda (footer: “Status of Roads provided by Honda”).

My first thought: with OpenStreetMap it is possible too!
So here we go: The Geofabrik generates hourly new files for Japan (Sendai region only). It’s available here. I have written a small script that only gets the highways out of these files and after that I imported them in a PostGIS/PostgreSQL database. Finally they are available in a Geoserver WMS/WFS.

The processing steps with its time range are:

  1. Download OSM file from the Geofabrik Server: ca. 25sec.
  2. Generate an *.osm file from the *.pbf with Osmosis: ca. 40sec.
  3. Create and import WMS PostGIS table: ca. 25sec.
  4. Overall: ca. *1min 30sec* (A cronjob is running every 30min at 20 and 50 on the hour!)

You can use the WMS Layer in your own application. If you have an OpenLayers client, the following lines of code adds the overlay:
var wms = new OpenLayers.Layer.WMS(
"Road Status Japan (OSM) (impassabel=yes)",
"http://openls.geog.uni-heidelberg.de/geoserver/wms",
{'layers':'osm_japan:roads', transparent: "true", format: "image/png"} );
map.addLayer(wms);

The layer has only two colors: green and red. The green color symbolizes all the roads which can be used for traffic. The red color symbolizes the roads which have an “impassable=yes”-tag in the OpenStreetMap database and which can NOT be used for traffic!

I put a small example of the overlay with an OpenStreetMap map online here:
http://resultmaps.neis-one.org/japan.html

You can find further information about OpenStreetMap and the Sendai earthquake/tsunami in the OpenStreetMap Wiki. Additional information you can find in the German OpenStreetMap Blog.

I hope this overlay is useful for some people!
If you run into problems or you have questions, do not hesitate to contact me …

thx @ All OpenStreetMap contributors !!1!
thx @ University of Heidelberg for the Web Mapping Service
thx @ Frederik from The Geofabrik for the Japan OSM files
thx @ *Fab*

[Update 1. – March 18th, 2011]
Last nigth I’ve added a second layer with node barriers (barrier=*). You can used it with the following lines of code:
var osm_japan_barriers = new OpenLayers.Layer.WMS(
"Node Barriers Japan (OSM) (barrier=*)",
"http://openls.geog.uni-heidelberg.de/geoserver/wms",
{'layers':'osm_japan:barriers', transparent: "true", format: "image/png"} );
map.addLayer(osm_japan_barriers);

[Update 2. – March 21th, 2011]
I’ve added a third layer with tsunami:damage polygons (key=”tsunami:damage” ; value=”flooded”, “yes”, “scoured”, “flood”, “debris”, “debris_field” or “destroyed”). You can used it with the following lines of code:
var osm_japan_tsunami_damage = new OpenLayers.Layer.WMS(
"Tsunami Damage Japan (OSM) (tsunami:damage)",
"http://openls.geog.uni-heidelberg.de/geoserver/wms",
{'layers':'osm_japan:tsunami_damage', transparent: "true", format: "image/png"} );
map.addLayer(osm_japan_tsunami_damage);

[Update 3. – March 24th, 2011]
You can now find the Honda Sanriku layer on the “Status Map after the Earthquake in Japan” too (update daily!). Further I have added some Japanese translations and a Bing Aerial layer to the site. You can use the Honda Layer with the following lines of code:
var honda_sanriku = new OpenLayers.Layer.WMS(
"Honda Sanriku Layer",
"http://openls.geog.uni-heidelberg.de/geoserver/wms",
{'layers':'osm_japan:honda_sanriku', transparent: "true", format: "image/png"} );
map.addLayer(honda_sanriku);

Again, my example site can be found here: http://resultmaps.neis-one.org/japan.html

The Return of “Unmapped Places in OSM EU”

My last blog post about “Unmapped Places in Europe” was read by more then 800 people. So I think it’s time to redo the analyses a second time after three months. At the moment (March 11th, 2011) we have (according to the Geofabrik extract) 505091 places in OpenStreetmap Europe. They can be separated into the following place-types:

  • city=1055 (as of Dec. 5th, 2010 it was 1045 -> +1%)
  • town=16106 (as of Dec. 5th, 2010 it was 16032 -> +0.5%)
  • suburb=24913 (as of Dec. 5th, 2010 it was 23563 -> +6%)
  • village=278691 (as of Dec. 5th, 2010 it was 271147 -> +3%)
  • hamlet=184326 (as of Dec. 5th, 2010 it was 165804 -> +11%)

During my last and also during this analysis I *only* used those places that had a corresponding “village”-value. My tool works so far as followed:

  1. Get only places with a village-tag.
  2. Search nearby (ca. 600m distance) for a street with one of the following highway-types: residential, service, living_street, cycleway, footway, pedestrian, steps or platform.
  3. If no street can be found, mark the place as “unmapped”!

Why did I only use villages? Well for the case of Germany it can be assumed that places with “higher” place-type tags such as “town” or “city” have already been mapped.

What are the “false=positives” and why is your village marked as unmapped? A village should usually have at least one of the roads mentioned above otherwise the place should be mapped as a hamlet. Would you agree with this?

The results showed that of the total 278691 villages, about 152337 (55%) (in Dec. 2010 it was 58%) who are located within “Europe”, have not been detected or mapped yet. The following diagram shows the distribution of the numbers by country.

YAY, Germany!!!

The results can again be displayed as a GPX-overlay on a map which can be found here http://resultmaps.neis-one.org :

(Remarks for http://resultmaps.neis-one.org: Not each and every country is available as an overlay. Some countries such as France, Poland and Ukraine showed longer browser loading times to display the GPX-overlays!)

thx @ Dennis

Growing agreement & relicensing OSM -Update-

My last blog post about the growing agreement to the new CTs is now nearly three months ago. Time for a short update: During the aforementioned time frame, about 32 contributors accepted the new CTs every day. Overall this means that since October 21 2010 there are about 43 contributors each day who accepted the new CTs. I updated my diagram with the latest numbers:

In December I conducted an analysis about the “Change of OSM object numbers through relicensing”. This time I only declared the last modifier of an OSM object (node/way/relation) as the owner of the object! The last and the new results can be seen in the following diagrams:

In my OSM-user-database of March 9th, 2011 a total of 120456* members are the “owners” of the following OSM objects (* Notice: Not every member of the OSM project (>350000 members) has contributed!):

  • Number of nodes: 1007604532
  • Number of ways: 85365727
  • Number of relations: 899145

As of March 9th, 2011, 8124 Users have accepted the new license. 35678 new OSM members (uid >= 286582) have accept the new contributor terms automatically. I created the following numbers of OSM objects, which will be available for relicensing (if you assume that the last modifier is the owner of the object):

  • Number of nodes: 801700665 (79,56%) (as of Dec. 15th, 2010 it was 66,52%)
  • Number of ways: 66236798 (77,59%) (as of Dec. 15th, 2010 it was 61,68%)
  • Number of relations: 716130 (79,65%) (as of Dec. 15th, 2010 it was 62,14%)

Are you still interested in any analysis regarding this topic?

thx @ Dennis

[Update – March 24th, 2011]
The following diagramm shows the above numbers of March 9th, 2011 in percent: