Visualizing the #MissingMaps OpenStreetMap Contributions

The Missing Maps project is a collaboration between the Humanitarian OpenStreetMap Team (HOT) and various partner agencies, such as the American or the British Red Cross. One of their main objectives is “to map the most vulnerable places in the developing world, in order that international and local NGOs and individuals can use the maps and data to better respond to crises affecting the areas.” You can find additional information about the Missing Maps Project on the OpenStreetMap (OSM) wiki and their project page.

A year ago, I created a webpage where you can filter OSM changesets by a specific comment. Sadly the webpage provides only a search for the latest seven days. However, the Missing Maps project asked me, if it’s possible to “look over a longer time scale”? Here we go, based on a similar concept that I used for a webpage that I created for the HOT Ebola Response, I made a Map that displays all OSM changesets which have the hashtag #MissingMaps in the comment attribute and have been created since August 1st, 2014. It’s online here and being updated on an hourly basis: http://resultmaps.neis-one.org/osm-missingmaps

http://resultmaps.neis-one.org/osm-missingmaps

The webpage also contains overall information about the number of OSM Contributors and map changes. So far more than 800 contributors created more than 1.5 Mio map changes in almost 22,000 changesets. The volunteers contributed in more than 30 countries such as Congo-Kinshasa, Sudan, Central African Republic, Indonesia, Ethiopia, Chad, Zimbabwe or Rwanda. Additionally the number of created changesets and contributors of the last seven days are displayed in two charts at the left-hand side. The time slider at the bottom of the map can be used to show the changesets between two specific dates. Finally an overview page lists the names of all OSM contributors who used the hashtag #MissingMaps in their changesets comment. Same as last time: Thank you & keep up the good work!

Thanks to maɪˈæmɪ Dennis.

Ebola Response Map and OSM contributor analysis

For almost eight months the OpenStreetMap (OSM) community has been collecting geo information for the West Africa Ebola outbreak response now. The collective work of the crowd is somewhat managed by the Humanitarian OpenStreetMap Team (HOT). For example, the Task Manager provided by HOT gives interested contributors information in which areas map features are needed. However, you can find additional information in an article by Pierre Beland, which he wrote during a conference where he presented the efforts of the OSM community. The OSM wiki contains some useful information about the West Africa Ebola Response too. Matt Irwin also wrote a summary about the OSM mapper contributions and created an interesting visualization of all the mapping work.

I created a response map for the OSM mapping activities in West Africa, a similar approach as I previously used for the “Typhoon Haiyan” deployment. It displays all OSM changesets created since March 1st, 2014. The analysis extent is displayed by a black rectangle. In total, more than 2,000 contributors made more than 10 million changes to the map. At the bottom you will find a time range slider which can control the displayed changesets. Sorry for the sometimes slow performance, but the community (maybe you too) collected a huge amount of data! The map is online here: http://resultmaps.neis-one.org/osm-ebola

osm_ebola_page

A complete list of all OSM contributor names with their number of map changes can be found here. It’s really fantastic, “Thank you & keep up the good work!”. This time I have also spent some time to analyze what type of contributors helped in this scenario. The following diagram shows the number of contributors per month in the above mentioned analysis extend. Additionally, it contains information on how long the individual contributors have been collecting map data. In contrast to the first months, many new members contributed data in the past three months.

ContributorActivityMembership

Based on the information from “How did you contribute to OSM“, which shows in what country a member performed the most map changes, I created the following chart. It displays the distribution of the contributor’s countries of origin. I assume that the high number of new registered members, who created many changesets during their Ebola response, moved their origin towards West Africa. However, overall more than 2,000 contributors from almost 110 countries helped so far, AWESOME! What do you think?

ContributorsOrigin

Lastly, I would like to point to the following blog post which shows you how to search for changesets with a specific comment for any region of the world: “Filtering OpenStreetMap Changesets by a Specific Comment“.

Thanks to maɪˈæmɪ Dennis.

Your Explored OSM World

Gregory Marler had the great idea to implement an “explored” map, based on a concept that some of you might know as “fog of war” from strategy video games. So here you go: I extended my OSM Heat Map with the “Explored Map Style”. It essentially reveals the contribution areas of an OpenStreetMap member in a “fog of war” style. The following figure shows Gregory’s amazing “explored” OSM map.

ExploredMapStyleToner

The Heat & Explored Maps are available for almost all OSM members who contributed at least several changesets here: http://yosmhm.neis-one.org (The new “Explored Map Style” can be selected in the layer panel (upper right corner). Additionally, I added the awesome looking and well known Watercolor and Toner map styles from Stamen design)

Thanks to maɪˈæmɪ Dennis

Welcome to an additional family member – OSMstats

Maybe some of you are already familiar with “OSMstats”, a website that provides numerous statistics about the OpenStreetMap (OSM) project. The site was created and is maintained by the two guys at altogetherlost.com. However, OSMstats has now been moved to the ResultMaps domain at osmstats.neis-one.org. I added several new features too. First of all, you can now select a specific date for your stats. Secondly, the main menu panel has been extended with a new entry for statistical information about OSM changesets.

osmstats

Additionally, the graphs for the country statistics, the active members and daily edits are also available in a “year”-overview. I hope you like the new extensions. A big thanks to both guys at altogetherlost.com who originally created OSMstats!

OSMstats is now available at: http://osmstats.neis-one.org
Feel free to check out my Resultmaps too which offer many helpful and funny OSM tools: http://resultmaps.neis-one.org

Notice: OSMstats was introduced in 2011, this means the webpage cannot provide statistics prior to that year. Also, the newly created Changeset-Tab has only data for July, 2014 and after.

Thanks to maɪˈæmɪ Dennis

The OSM Contributor Activity Report – Edition 2014

The OpenStreetMap (OSM) project celebrated its 10th anniversary in August 2014. For almost 10 years it has increased its number of registered members. Even though some contributors stopped their contributions to the project, each day new mappers start collecting features for the free wiki world map (aka database).

In my last contributor report in 2013, the OSM project had a total of 1.3 Mio registered members. For July 2014 this number has increased to almost 1.6 Mio registered members. Similarly to last year, I checked how many contributors created one or more than ten changesets or performed more than 10 map edits. This information can be retrieved from the changeset dump.

NewContributorsPerMonth.201408

The figure above reveals a similar trend to the ones we saw in the past few years: Less than 1/3 of the 1.6 Mio registered members actively contribute to the project (450,000 members). Furthermore, only a small group of 16% (270,000) or respectively 6% (100,000) of the contributors performed more than 10 edits or 10 changesets.

The long-term motivation of the contributors is quite important too. Therefore, similarly to the methods that we presented in our open access publication, I created a figure, which visualizes the increase in monthly volunteer numbers over the past few years and the consistencies in data contributions.

Activity.Stats.201408

As we already revealed in our study, only half of the monthly active members in OSM are also long-term contributors. Also, the previously discussed pattern which depicts a contributor loss of almost 70% over the years is again visible. However, it is good to see that at least some “senior” mappers still keep contributing to the OSM project.

Thanks to maɪˈæmɪ Dennis

The Average Age of OpenStreetMap Objects

Joseph Reeves asked me on twitter the other day if “anyone knows the average age of @openstreetmap objects?“. Here we go: Based on the complete OSM data history file from here (June 14th, 2014) and some additional lines of code, I conducted a simple analysis.

Overall 400,000 mappers of the more than 1.7 million registered members contributed to the OSM project. Almost 375,000 contributors created at least one Node, 325,000 one Way and 70,000 one Relation object. In total the contributors collected more than 2.7 billion Nodes, 263 million Ways and 3 million relations. The percentage of newly created OSM objects (Nodes, Ways & Relations) has been more or less at the same level for the past few years (2010 to 2014): with17% to 20%. The following diagram shows the percentage of each created OSM object type.

created_objects

Additionally, I evaluated the number of objects based on the date of their last modification. Utilizing the object timestamps of the last modification, we see a slightly different result for the last 4 years. 55% of the Nodes, 67% of the Ways and 74% of the Relations in the OSM database do not have a timestamp dated before 2012.

last_modifier

However, I guess it would be an interesting visualization, if we could put those numbers on a world map similar to the “OpenStreetMap availability” by Stefano De Sabbata. You can also find some up to date OSM statistics here.

Thanks to maɪˈæmɪ Dennis

A précis: Where are the US mappers at?

This blog post is a summary of Dennis’ and my State of the Map (SotM) United States presentation. Maybe some of you already know about our publication: “Comparison of Volunteered Geographic Information Data Contributions and Community Development for Selected World Regions”. From the abstract: “Our findings showed significantly different results in data collection efforts and local OSM community sizes. European cities provide quantitatively larger amounts of geodata and number of contributors in OSM …”. “Furthermore, the results showed significant data contributions by members whose main territory of interest lies more than one thousand kilometers from the tested areas.” Especially the last finding is quite interesting when considering “arm-chair-mapping” in OSM.

However, for our SotM US session we repeated some of the conducted analyses for 50 urban areas in the United States to see whether similar patterns could be determined. You can find the session abstract here; additionally the ppt slides and also a video are online. The following animation shows the number of contributor’s evolution in the US from 2007 to 2014.

us_mapper_animation_tiny

Similar to our prior research results for the selected 12 world regions, the US urban areas showed different individual patterns. Some cities such as Fargo (ND) experienced several data imports in the past which resulted in strong data density values (Nodes and Ways), whereas other areas solely rely on a small community of volunteers and contributors.

We also conducted a simple statistical analysis to evaluate whether certain socio-economic factors have an impact on the development of OSM communities in the different cities. Variables such as population density, per capita income and education showed a moderate to strong correlation with contributor numbers, highlighting that all of the aforementioned factors can have an impact on the success of OSM in the selected urban areas in the US.

us_socio_economic_factors

It was also quite useful to take a look at the local contributor numbers vs. external contributors. Certain cities such as Miami (FL) heavily rely on data contributions made by mappers whose home region is more than 1000 km away, whereas other cities such as Los Angeles(CA) show large values for both, local and external mappers. You can check out your own area here too. The corresponding blog post is online here: “The OpenStreetMap Contributors Map aka Who’s around me?”. A more detailed analysis that is currently being conducted will reveal if cities that prove to have large external mapper contributions show the same quality as areas with lots of local mappers or not.

 

Filtering OpenStreetMap Changesets by a Specific Comment

In my last blog post I showed that OpenStreetMap (OSM) changesets offer a great opportunity to visualize the latest changes to the OSM map or to compute some up to date descriptive statistics of a particular region (Typhoon Haiyan OSM Response Map). Oftentimes OSM contributors use tags, comments or hashtags in their changesets to provide additional information about the features they mapped. For example it is quite common to add a specific hashtag, such as #notlm (Night of the living maps), to the changeset comment to link to a mapping party or another event. To filter or collect changesets with these notations, I developed a new webpage: http://resultmaps.neis-one.org/osm-changesets

osm-changesets

The newly created website allows you to search for specific terms that have been used in a changeset’s comment section within the past 7 days. The changesets are based on an hourly updated OSM database. Every colored rectangle that is visualized on the map depicts one changeset. Additional descriptive statistics, such as the total number of contributors or map changes, are displayed on the left side of the website. Furthermore, charts illustrate the history of the number of changesets and contributors for the most recent 7 days. At the bottom of the left pane the TOP 5 countries with the most map changes that include the searched term are listed (for this computation the center of the changesets’ rectangle is used).

It is quite interesting to search for “bing” or “survey” (or “gps”). Give it a try and see which tags have been used in recent days. I hope that this tool is quite useful for any type of event or mapping party. It is available here: http://resultmaps.neis-one.org/osm-changesets

Thanks to maɪˈæmɪ Dennis

Typhoon Haiyan OSM Response Map

As you may know by now, the Philippines have been struck by one of the largest Super Typhoons ever recorded. Many casualties are expected (especially in Tacloban) and even more people are in need of help. While some of us have been busy working with the “Stand By Task Force” to analyze and geolocate tweets with useful information, the Humanitarian OpenStreetMap Team (HOT) has been making progress as well.
I have spent some time to develop a website that visualizes the latest changes to the OSM map (utilizing changesets) in the Philippines and hope that it helps to determine areas that might be already worked on, while other areas are lacking any new information. You can find the map here: http://resultmaps.neis-one.org/osm-typhoon-haiyan-2013

osm-activities

Additional information can also be found in the OSM Wiki and for all German fellows at the Wochennotiz blog, in case you are interested in actively contributing and helping the disaster response teams. Any mapping efforts are much appreciated!

Some statistics: Until midnight between Saturday and Sunday (November 9th/10th), almost 180 contributors made more than 245,000 map changes. On Sunday evening 6pm UTC (November 10th, 2013) the values increased to 245 volunteers who created nearly 440,000 map changes within the aforementioned extent. AWESOME, keep up that great work! Thank you.

Link to the website: http://resultmaps.neis-one.org/osm-typhoon-haiyan-2013

Some technical background information: The minutely changeset diff files are imported in a PostgreSQL database. The created website only visualizes the changesets which are within a predefined extent, i.e. the Philippines and some parts of the Vietnam coast. This extent is illustrated by a black line on the website. The OSM changeset data is updated on an hourly basis.

Thanks to maɪˈæmɪ Dennis

It’s about time – OpenStreetMap Contributor Activity Report 2013

One and a half years ago (end of 2011), one of my open access publications (“Analyzing the Contributor Activity of a Volunteered Geographic Information Project — The Case of OpenStreetMap“) was published. It contained several interesting findings about the contributions made by the community of the OSM project. The results showed that the community follows a particular pattern that many other online community based projects tend to struggle with too. Only a small number of the members really contribute in a meaningful way to the project. Additionally, the publication illustrated how many contributors are located in Europe and other areas of the world and how and where mappers contribute data over a certain period time.

I thought it was time to update this information with some new statistics. Between the end of 2011 and July 2013 the number of registered OSM members has increased more than two-an-a-half times to almost 1.34 Mio. Based on the freely available changeset dump of the project it is quite easy to check how many members created at least one changeset and thus hopefully made an edit to the database. The following figure shows the increase of registered members and the aforementioned results of the analysis of the changeset dump of July 31st, 2013.

2013_Members

By the end of 2011 almost 43% of the 505,000 registered members created at least one changeset. This number decreased by July 2013 to only 26% (355,000) of the 1.34 Mio registered members. As some of you already know, the real-number of actual contributors is also far below this. I decided to look into this in a little bit more detail and created some diagrams that show the number of changesets and active contributors per month. We can see a few events that had an impact on the numbers in the diagrams. First, the license-change in April 2012 followed by the run of the redaction bot in July 2012 (HDYC-profile) and at the end the release of the new OSM iD editor in May 2013. The number of changesets has not changed a lot when comparing current (July, 2013) numbers with prior months of last year.

2013_Changesets

The last diagram of this blog post shows the active contributors per month. The collected information tells us that the total number of “long-time” contributors is increasing whereas the number of “new” contributors is more or less on the same level in recent months.

2013_Contributors

It is also interesting to see an impact of “new” members in the month before the license change (March, 2012). Anyway, for the last one and a half years the number of active contributors per month is consistent with a total number between 19,000 and 23,000. What do you think?

You will find additional OSM editor usage statistics (by Oli-Wan) in the OSM Wiki. Also, it is interesting to see that currently the number of newly registered members is only growing between 700 and 900 per day. In the time before August, 2013 is was between 3,000 and 4,000 per day! Did anyone change something in the registration process at OSM.org, e.g. a new security/login mechanism during account creation?

Thanks to maɪˈæmɪ Dennis

PS: Happy 9th Birthday OpenStreetMap!