Category: Allgemein

Processing compressed OpenStreetMap Data with Java

This blog post contains a summary on how you can write your own Java classes to process OpenStreetMap (OSM) pbf files. PBF is a compression format, which is nowadays more or less the standard utilized for reading and writing OSM data quickly. In the OSM world, many tools and programs implemented this file format (you can find additional information here). However, I think the following samples for reading and writing such compressed OSM data can be very helpful. In particular, if someone has to create some sort of test data or has to read some specific mapped objects of interest for her/his own project. The well-known Java Osmosis tool (command line application for processing OSM data), provides several libraries that are the basis for this brief tutorial.

Step 1: Maven is the key – If your Java project is already managed by Maven, you can just add the following lines to your pom.xml. It downloads and adds the required jar-files that are needed to process compressed OSM data to your project.

How to detect suspicious OpenStreetMap Changesets with incorrect edits?

Since its rise in popularity, the well-known online encyclopedia Wikipedia has been struggling with manipulation or, in the worst-case, vandalism attempts. Similarly, the OpenStreetMap (OSM) project suffered several times over the past few years of cases where incorrect map data edits were made. These erroneous edits can stem at times from (new) contributors or illegal data imports (or automated edits) which have not been discussed in advance with the community or the Data Working Group (DWG) and corrupted existing project data. The current OSM wiki page gives a great overview about general guidelines and e.g. types of vandalism. Another page in the wiki also mentions a prototype of a rule based system for the automatic detection of vandalism in OSM, which I developed in 2012. However, the system has never actually been implemented. Today, the contributors of OSM can use a variety of different tools to inspect an area or particular map changes. A few of them are listed below (complete list can be found here):

OpenStreetMap Crowd Report – Season 2015

Almost one year has passed again. This means it’s time for the fourth OpenStreetMap (OSM) member activity analysis. The previous editions are online here: 2014, 2013 and 2012. Simon Poole already posted some interesting stats about the past few years. You can find all his results on the OSM wiki page. However, similar to last year, I try to dig a little deeper in some aspects.

Overall the OSM project has officially more than 2.2 million registered members (Aug, 9th 2015). For several of my OSM related webpages I create a personal OSM contributor database, based on the official OSM API v0.6. Anyway, when using this API, the final table will show a list with more than 3 million individual OSM accounts (Aug, 9th 2015). I’m not sure what the cause for this gap of almost 1 million members between the official number and the member number extracted with the API could be. Maybe some of you have a possible explanation? However, I think many accounts are created by spammers or bots.

Visualizing the #MissingMaps OpenStreetMap Contributions

The Missing Maps project is a collaboration between the Humanitarian OpenStreetMap Team (HOT) and various partner agencies, such as the American or the British Red Cross. One of their main objectives is “to map the most vulnerable places in the developing world, in order that international and local NGOs and individuals can use the maps and data to better respond to crises affecting the areas.” You can find additional information about the Missing Maps Project on the OpenStreetMap (OSM) wiki and their project page.

A year ago, I created a webpage where you can filter OSM changesets by a specific comment. Sadly the webpage provides only a search for the latest seven days. However, the Missing Maps project asked me, if it’s possible to “look over a longer time scale”? Here we go, based on a similar concept that I used for a webpage that I created for the HOT Ebola Response, I made a Map that displays all OSM changesets which have the hashtag #MissingMaps in the comment attribute and have been created since August 1st, 2014. It’s online here and being updated on an hourly basis: http://resultmaps.neis-one.org/osm-missingmaps

Your Explored OSM World

Gregory Marler had the great idea to implement an “explored” map, based on a concept that some of you might know as “fog of war” from strategy video games. So here you go: I extended my OSM Heat Map with the “Explored Map Style”. It essentially reveals the contribution areas of an OpenStreetMap member in a “fog of war” style. The following figure shows Gregory’s amazing “explored” OSM map.

ExploredMapStyleToner

The Heat & Explored Maps are available for almost all OSM members who contributed at least several changesets here: http://yosmhm.neis-one.org (The new “Explored Map Style” can be selected in the layer panel (upper right corner). Additionally, I added the awesome looking and well known Watercolor and Toner map styles from Stamen design)

Thanks to maɪˈæmɪ Dennis

The OSM Contributor Activity Report – Edition 2014

The OpenStreetMap (OSM) project celebrated its 10th anniversary in August 2014. For almost 10 years it has increased its number of registered members. Even though some contributors stopped their contributions to the project, each day new mappers start collecting features for the free wiki world map (aka database).

In my last contributor report in 2013, the OSM project had a total of 1.3 Mio registered members. For July 2014 this number has increased to almost 1.6 Mio registered members. Similarly to last year, I checked how many contributors created one or more than ten changesets or performed more than 10 map edits. This information can be retrieved from the changeset dump.

NewContributorsPerMonth.201408

The figure above reveals a similar trend to the ones we saw in the past few years: Less than 1/3 of the 1.6 Mio registered members actively contribute to the project (450,000 members). Furthermore, only a small group of 16% (270,000) or respectively 6% (100,000) of the contributors performed more than 10 edits or 10 changesets.

Introducing OpenStreetMap Contributor Activity Areas

One month ago I wrote a blog post about a new website which allows you to see other OpenStreetMap contributors in your area. Overall the feedback was very positive, thank you very much for that! However, now it is time for a new extension to the “How did you contribute to OpenStreetMap?” (HDYC) webpage. As I mentioned in my last blog post, I used an algorithm (which is described in a paper that I wrote here) to compute and determine the activity area of a contributor based on her/his changeset centers. The following figure shows the new function that was added to the HDYC website visualizing the activity area of a contributor! Sorry Harry, as always you have to be our guinea pig, but you have a really awesome activity area 🙂

I Like OpenStreetMap (OpenLayers Plugin)

A few months ago, Frederik Ramm posted an idea on the German OpenStreetMap mailing list about a new (stochastic) approach to OSM data quality assurance. You can find his original German post here. His idea was to create a way to allow users to “like” or “dislike” a specific region on the OSM map, a function that other popular websites such as YouTube or Facebook implemented to allow users to provide feedback to videos or status updates. For OSM this particular function could give some indicators or trends about the OSM map data.

I really liked his idea and in collaboration with Frederik I created an Open Source OpenLayers plugin. For all new readers: OpenLayers is an Open Source library which can implement a dynamic (OSM) map into more or less any webpage. One of our goals was to make the integration of the ILikeOSM plugin as easy as adding a tile server to your OpenLayers map.

What type of Mapper are You?

Last weekend Frederik and Richard asked me if I could give some additional information on the „How did you contribute to OpenStreetMap?“ webpage. So here we go. Below the prior, familiar chart which shows the contributions per month, you will find two new charts. The first one shows the number of changesets per weekday and the second one the number of changesets per hour.

Additionally I added an output that roughly estimates what type of mapper the contributor is, based on his/her number of contributions (changesets). However, I will give no warranty regarding the group or type of mapper that each individual contributor falls into and I think you will figure out the different groups of mappers by yourself anyway 😉

The following picture highlights the new things on my webpage:

Most of you already know it, you will find “How Did You Contribute to OpenStreetMap?” here: http://hdyc.neis-one.org

New OpenStreetMap Germany Webpage

The German OpenStreetMap project webpage has been online for three years now. During these years several small updates have been made to the page. But it is about time to create a complete new webpage with a “state of the art” layout and style. Jonas created the first drafts of a possible new design of the site a while ago. Unfortunately he hasn’t had enough time to finish his work, but luckily Fabian and Pascal came up with a new webpage. Both of them considered Jonas first draft during the development and expanded it with their own ideas.

The new site is based on HTML, CSS and Twitter-bootstrap. Most of the content has been taken from the old webpage with minor improvements. In their opinion the biggest change lies within the way the users are welcomed to the page with information such as: “What is OpenStreetMap?“, “How can I help?” or “How can I use the data?“. The following image shows the important changes made to the welcome-page: