Additional insights about OSM changeset discussions: Who requests, receives and responds?
by Pascal Neis - Published: April 22nd, 2018
Last year I wrote two blog posts about the OpenStreetMap (OSM) feature that allows commenting on contributor map changes within a changeset. The first blog post showed some general descriptive statistics about the number of created changeset discussions, affected countries, the origin of the commenting contributors or their mapping reputation. The second post described a newly introduced feature, where contributors can flag their changeset so that their map edits can be reviewed. This blog post will follow up on this topic and conducts some similar but updated research.
The first chart shows the number of created comments (discussed changesets) and the contributors involved over the last 15 months. The number of created comments and discussed changesets fluctuates over time, whereas the number of contributors who take part in changeset discussions stays consistent at around 1,500 per month. Around 3,200 contributors received a comment on at least one changeset’s map edits a month.
After publishing the aforementioned blog post, people were asking for some numbers that show the commented changeset grouped by the editing application that was utilized. The results show that these numbers stayed more or less the same with 2/3 of all commented changesets (almost 160,000) being edited by the iD editor. This is not very surprising since this particular editor is used by many OSM beginners during first edits. It’s also interesting to see whether the changeset author responded (also grouped by the OSM editor that was used). Overall only around 32,000 contributors responded to their changeset comment. You can find some additional charts about the comments per discussed changeset in the previous blog post. Again, the majority (around 71%) of the changeset discussions contain one comment only.
Since last August, contributors can mark their changeset with a flag for “review_requested”. After a few months now I think it’s time for a first look at the numbers. The following charts display the number of requested reviews by contributors and their marked changesets. First of all, almost each month around 7,000 contributors asked for one review minimum. Overall almost 36,000 changesets have been marked for review each month. If we take a close look and filter changesets by hashtags, we can see that sometimes large numbers of the changesets are contributed by #HOTOSM or #MissingMaps members.
The following diagram shows probably the most disappointing results: The number of requested reviews that actually have been reviewed in the end. No matter if the changeset has the #HOTOSM or #MissingMaps tags or not, the relative value of reviewed changesets lies only between 6 and 18%. To be honest, I’m also a bit surprised that only a few of #HOTOSM or #MissingMaps changesets have been reviewed so far.
So, what do you think? Do you review contributions without commenting on the changesets? Do we need more attention here or is it just boring to look after changesets which are marked for review? I think it’s obvious, that we need more contributors who review map changes or least “documenting” their work. But can we handle this? Or do we need better tools?
Thanks to maɪˈæmɪ Dennis.
The “review request” features is great. However HOT or Missing Maps work has often been through it’s own Tasking Manager and that gives it’s own system of review/validation. When I’m teaching Missing Maps, I don’t see the point of telling people to tick the OSM’s “I’d like my edits changed” because that is covered when they tell HOT “I have finished this area”. I do often point it out, as something they can/should do when they’re not working on a HOT project.
The ideal would be if the HOT Tasking Manager linked into this changeset review feature. Although how that works would be a little complicated, and it’s a big ask to make so suddenly.
Hi Gregory, thanks for your comment. Yes, I know that HOT or Missing Maps has their own “review/validation” system. Furthermore I agree, they should link their reviews in some kind of way to the main OSM database. Or is this information available (API)? Another question, do HOT or Missing Maps use the information when a changeset received a comment? I guess not.
I don’t think the tools are lacking, it’s just a matter of finding the people who want to do the work. People like their local area to be empty of open notes, fixme’s, validation errors, etc, so I think the best tool would be a map where you can easily see the unreviewed changesets. Also, just like any other stat, it would be great to have a more local insight.
Next, older cases are less important to review. So a monthly rate of “reviewed within three months” would be useful.
[…] http://neis-one.org/2018/04/insights-osm-changeset-discussions/ […]
The statistics about “Commented changesets by used editor” would be interesting to see in relation with the total number of changesets uploaded through this editor.
Besides this, very interesting article, as usual!
Hi Martin, thank you very much for your comment. There you go:
OSM Editor – overall created CS (>2017-08-10) – discussed CS
====================================
JOSM – 2,582,339 -> 1%
iD – 4,183,562 -> 2%
MAPS.ME – 271,481 -> 4%
Potlatch – 168,845 -> 5%
If I want to help review the changesets flagged as such in my area, how would I go about it?
I think you can find some useful information in the following blog post:
“Review requests of OpenStreetMap contributors – How you can assist! –”
https://neis-one.org/2017/09/review-requests-osm/
Hi Pascal,
This is an interesting topic for me, since I recently got more active in quality control. I’ve mapped quite a lot last half year and I feel confident now that I can give insightful comments.
I’m struck however by how many of those comments never get a reply from the changeset author. Typically I use your overview (http://resultmaps.neis-one.org/osm-discussion-comments?uid=3485107&commented) to track which comments are awaiting response. What I see is that often the mappers have been active after I have given the comment, so they are still mapping. In those cases I would expect a reply.
However, it is not clear why no reply is given. Does the e-mail about the changeset comment end up in the SPAM folder? Do they just ignore help for some other reason?
Joost mentions that the tools are not lacking. I agree to that point, but I do see huge improvement potential for how integrated the tools are.
Basically if you are a novice OSM mapper, you have probably no clue of all the handy tools that are scattered around. To my opinion, many of the (great) tools you supply via this website should actually be integrated into openstreetmap.org in a way that you quickly have an overview of open comments, etc. In my opinion the openstreetmap.org website is currently very outdated.
Like the review option you discuss. Although the option to tickmark is integrated into openstreetmap.org (via iD), the tool to actually manage the reviewing is not integrated for the mappers who are interested in doing reviews. I would expect a tool that lets me filter the area and time frame for to be reviewed changesets and when I click gives me a workable overview of what actually happened (like achavi does).
I do think that the OSMF should consider putting effort in merging a lot of functionality now giving by side project into the openstreetmap.org website to aid people in their mapping work. The way all the great tools are scattered gives a terrible user experience, and I expect a lot of mappers have no clue of their existence.
Cheers,
dikkeknodel