Category: Map

How to detect suspicious OpenStreetMap Changesets with incorrect edits?

Since its rise in popularity, the well-known online encyclopedia Wikipedia has been struggling with manipulation or, in the worst-case, vandalism attempts. Similarly, the OpenStreetMap (OSM) project suffered several times over the past few years of cases where incorrect map data edits were made. These erroneous edits can stem at times from (new) contributors or illegal data imports (or automated edits) which have not been discussed in advance with the community or the Data Working Group (DWG) and corrupted existing project data. The current OSM wiki page gives a great overview about general guidelines and e.g. types of vandalism. Another page in the wiki also mentions a prototype of a rule based system for the automatic detection of vandalism in OSM, which I developed in 2012. However, the system has never actually been implemented. Today, the contributors of OSM can use a variety of different tools to inspect an area or particular map changes. A few of them are listed below (complete list can be found here):

Based on the database which I use for multiple other services, I created an easy to use webpage to find suspicious OSM changesets with possibly incorrect map edits. The webpage offers some filter options such as the boundary of a country or the object change of interest. In contrast to the other aforementioned webpages you can also filter changesets based on the active “mapping days” of the contributor. A “mapping day” is a day on which the contributor created at least one changeset, independent from the registration date. I am also planning on adding additional user reputation information such as used editors or tagging behavior. And of course I am going to add some RSS feeds in the next version. The first version can be found here.

OSMSuspicious

What makes all of this different from other tools? Well, I think one of the major advantages is the simplicity of the webpage and that you can filter changesets based on the contributor activity and/or the changeset edits. In contrast to other tools, you can find changesets not only based on your area of interest, but also based on potential beginner mistakes and hopefully not vandalism attempts or fictional/ none existing map data.

Find Suspicious OSM Changesets here: http://resultmaps.neis-one.org/osm-suspicious

Thanks to maɪˈæmɪ Dennis.

The State of the Map. United States. Street Network. 2013

Last year we wrote a journal paper in which we analyzed the OpenStreetMap (OSM) dataset of the United States which was published on May 28th, 2013 in the Transactions in GIS Journal. You can download a free pre-print version here. This paper has been published just on time to add to the discussion at the upcoming State of the Map United States conference which will take place in San Francisco and includes some presentations about data imports to OSM. Unfortunately, Dennis and I cannot attend the conference this year, so we decided to write a blog post with some additional and up-to-date numbers.

In January there was an announcement on the OSM mailing list that in the past few months many connectivity errors in the United States OSM dataset had been fixed. Probably a lot of these fixes can be attributed to Martijn’s Maproulette website or to Geofabrik’s OSM Inspector (OSMI) Routing View. However, a short discussion started on the mailing list about the total number of errors that are left and how long it would take to fix all those errors. Thus, we downloaded four OSM planet files dated Jan 4th 2012, June 13th 2012, Jan 2nd 2013 and Jun 2nd 2013 to get some new results. After cutting the United States dataset from the planet files, we used the same algorithm as utilized in OSMI’s Routing View, to receive some stats about the street network of the US datasets.

First of all the, the following image shows the number of errors for each dataset that we included in the analysis. The errors that were detected are separated into unconnected and duplicate ways. You can find some additional information about both error types here.

As you can see, the number of unconnected OSM ways has been rapidly reduced in the past 17 months from around 141,000 to 19,000. The number of “duplicate way” errors has been reduced from 17,500 to 11,500. You can find the exact numbers in the following table and an updated error layer on the mentioned OSMI website. In certain cases the duplicate way error created several errors for one and the same way. For these particular cases the number of unique OSM way IDs were counted.

Date – Unconnected Ways – Duplicate Ways

  • Jan 4th, 2012 – 141,578 – unique 17,563 (overall errors: 535,923)
  • June 13th, 2012 – 145,468 – unique 17,977 (overall errors: 518,536)
  • Jan 2nd, 2013 – 15,911 – unique 12,287 (overall errors: 257,388)
  • Jun 2nd, 2013 – 19,073 – unique 11,582 (overall errors: 220,451)

Overall the length of the US street network did not really change a lot. At the beginning of 2012 it was around 11.07 million km while in 2013 it is 11.1 million km, which means an increase of around 30,000 km. The following image shows the distribution of the US street network divided by different OSM road classes.

The length of the residential roads is still decreasing (-496,000 km), similar to what we saw during the analysis for our paper, while the length of the other road types (+276,000 km) and secondary/tertiary roads (+205,000 km) is increasing. This is the result of a massive retagging process of the imported TIGER/Line dataset in OSM. Dennis mentioned this already in his SotM US 2012 presentation. Motorways also experienced an increase of around +44,000 km in 2012. You will find some additional, quite interesting statistics, charts and of course maps in the aforementioned journal publication. In particular a few more thoughts and facts about the effect and impact of data imports on OSM can be found in our research study about the United States OSM dataset.

The OpenStreetMap Contributors Map aka Who’s around me?

The wait is over! As I mentioned in December, I have been working on an interactive online map, which shows you all volunteers of the OpenStreetMap world on a map. The first three layers contain the activity center of a contributor, her or his first created and latest modified node. The algorithm to determine the activity area of an OSM volunteer has been described in my publication here. I should mention that I used all changeset centers instead of all created nodes of a contributor. This way the computations don’t take as long and the process can be repeated every week based on the weekly OSM changeset dump.

The above image shows the created map for the center of London. As you can see, there are several Mapper-Types to choose from on the left side. Each mapper type is distinguished by the number of changesets that were created (shown in parentheses). However, most of the time you will see many orange or red figures on the map , which means that the OSM project has a lot of members who only made a few changes. This “issue” or “phenomenon” has also been mentioned in the publication.

Below the map you can find additional information which shows the current number of mappers in your map’s bbox. Sometimes the number is a little bit larger than the real number of contributors in your area. This depends on the method that Openlayers uses to load the mappers from my server onto the map. For the next update, I maybe plan to add the dates of the latest contribution of a member. This way you can get current information about the number of (active) contributors for your area.

The data will be updated on a daily and weekly base. It is also important to mention that each layer has a limit of 1.000 contributors to show on the map. Thus, if you zoom out, you will always see contributors, but sometimes the real number is much higher than the number of mapper icons you see on the map.

Last but not least some stats: Today (Jan 6th, 2012) the OpenStreetMap project has  reached 1 000 000 registered members. Around 300 000 (30%) of all contributors created at least one changeset and about 200 000 (20%) volunteers created at least one Node.  Of the 300 000 contributors who created at least one changeset, around 240 000 created less than 10 changesets, 45 000 (4.5%) between 10 and 100 changesets and 17 000 more than 100 changesets. I think this amount of 17 000 mappers or 1.7% of all registered members is obviously the amount of active OSM members. This is also comparable with the official OSM “highly active users” stats from here.

The map is online here:
Overview of OpenStreetMap Contributors aka Who’s around me?

¡Muchas gracias maɪˈæmɪ Dennis!

Distribution of Active Users in OpenStreetMap – Oct-Nov 2012

Two years ago, we created some maps which showed you the number of users per country for a timeframe of one month. Maybe some of you remember that the highest concentration of active contributors in relation to the countries’ population could be found in Europe. We thought it was about time to make some new maps to see if things have changed. The following map shows you the number of active contributors per day per country.

Similar to our results two years ago, the above map only gives some general information about the total number of users per country and does not consider the population for each country. Therefore we created a second map which shows you the relation between active users and the population in each country per day.

Overall the maps show a slightly similar activity pattern of the OSM users as in our previous results two years ago. The major concentration of the project still lies in Europe expanding more and more to the east. However, you can see certain changes in other areas of the world too. First improvements have been made in Africa, South America and South-East Asia. Greenland of course is looking quite exceptional due to the low population value.

So, what do you think? Where are you located and how would you describe the mapping activity development in your country?

***Update: Nov. 20th, 2012***

OSM-GPS-Grid-Map

More than 6 weeks ago, I wrote a blog post about “Which country has the most OpenStreetMap GPS Points?“. You might remember that Russia and Germany provide the most OSM GPS points, as mentioned in the blog post.

Now, in a second step I created a worldwide grid, with an edge length of 15km, to show in more detail where the GPS points are located. The rendering of the tiles has been accomplished with the great TileMill software. Usually you would see some GPS noise in the grid cells above the equator. Thus, to eliminate this noise and to create a better overview, grid cells with less than 5 GPS points are made transparent. The following image shows an overview of Europe and the corresponding OSM GPS density. Of course you can also see some quite interesting and funny airplane, ferry or cruise routes.

However, I think the map overlay looks great and shows where the OSM GPS points are located. This information will also be published in the OSMatrix (thanks to Oliver Roick) within the next few days.

You can find my map here: http://resultmaps.neis-one.org/osmgps.html

PS: Below the map you will find a slider for the opacity of the GPS Grid Overlay, Enjoy!

thx @ maɪˈæmɪ Dennis for proofreading
thx @ *Fab* for creating the tiles/mapkey with MapBox’s TileMill