Tag: Benchmark

Comparison of (OSM) routing-engines – Reloaded

Maybe some of you remember that I conducted a comparison analysis between three OpenStreetMap (OSM) routing engine APIs (CloudMade, MapQuest Open and OSRM) and G**gle Maps API last week. You can find the results in my blog post here. As I mentioned in the article, I wanted to try to do a second analysis with more routing engines.

Thus, I added Bing Maps and two OSM engines (YourNavigation/YOURS and Routino/Roadeeno) to the comparison. All services have a continental coverage with the exception of OSRM. The following table shows an overview of (1) the request-response time of the service, (2) the calculated distance for the test-route and (3) the file size of the service response:

As you can see in the following diagram does the OSM routing engine (OSRM) give the fastest results. A little bit strange is that the Routino/Roadeeno service returns no valid route responses for requests which are longer than 600 km.

The same diagram in a more detailed view:

The routing engines have different ping times (round-trip time). Almost all services have a round-trip time off about 25 ms. You can see the times for each engine in the following diagram:

If you take those ping times into account and use a logarithmic transformation, the result look as shown below:

The above diagram shows in a quite impressive way the results of this comparison that allow the following conclusions: OSRM (OSM) shows the fastest results followed by G**gle Maps (Tele Atlas). Bing (Navteq), MapQuest (OSM) and CloudMade (OSM) are nearly equal in most cases. YOURS (OSM) and Routino (OSM) seem not to be the right choices at least for long route calculations (>600km). Maybe a second comparison with several routes between 10 and 500 km could be an enhancement?

thx @ maɪˈæmɪ Dennis 🙂

A comparison of several routing-engines – Which one is the fastest?

In the past blog post I wrote about the newest changes and encoding techniques that have been implemented in the Open Source Routing Project (OSRM). So I think it is time do a little comparison analysis about the request/response time of several routing APIs. The main question I wanted to answer was: “Is an OpenStreetMap direction service faster than G**gle?” I tested the following direction APIs for cars (fastest): MapQuest, CloudMade, G**gle and finally OSRM. For the analyses I wrote a small Java tool, which measured the time to get a result of a routing-service. I did all tests at home with a “regular” 12kbit/s internet connection. I tested several distance levels and the results can be seen in the following table. It shows the average times of five requests for each route with a delay of 3 seconds between each request/response. Overall I did this analysis three times.

The results are quite impressive. OSRM calculates the fastest route for all five test routes! Unfortunately we do not have any information about the server infrastructure at G**gle, MapQuest or CloudMade but the OSRM engine is running on a virtual server with limited hardware resources. It seems that the CloudMade directions service does not like Paris very much, as can be seen in the following image 😉

I will try to do a second comparison with Bing Maps, YOURS and Routino. So stay tuned …

— Update —

>> The second blog post is here: Comparison of (OSM) routing-engines – Reloaded <<

thx @ maɪˈæmɪ Dennis 🙂