Commons:OpenRefine/Training 2023-24/Final report

For 2023-24, as part of its support for Wikimedia Commons, the Wikimedia Foundation has funded OpenRefine for bug fixes to its Commons features, for a train-the-trainer program, documentation, and a WikiLearn course. This page contains the final report of this grant, June 2024.

Other relevant pointers:

Project page
Midpoint report (January 2024)
Project application: https://openrefine.org/assets/files/2023-Wikimedia-37b6be3a9d3b323cb5c20fef1600da0d.pdf

This report focuses on results and insights from the training and documentation parts of this grant. Software development will be finalized and reported later this year.

Project impact

Deliverables

For a detailed outline of each deliverable, see the original project application. This project’s midpoint report contains more details about various results.

Deliverable	Actual result	Remarks
Recruitment and establishment of a group of GLAM-Wiki-OpenRefine liaisons/trainers. At the end of this grant period, the Wikimedia movement and its content partners will be helped and trained by a clearly defined, and publicly visible and approachable, group of 4 to 7 individuals.	Out of 84 candidates via an open call, 16 participants for a Train-the-trainer course were selected. The course took place from November 2023 until end April 2024. Currently 11 of the participants have successfully finished the course and are now listed on a dedicated page on meta.wikimedia.org as certified OpenRefine-Wikimedia trainers.
Training materials and documentation. Creation of generic and re-usable training materials + cleanup of legacy documentation and training materials	There is updated OpenRefine-Commons documentation. The OpenRefine-Wikimedia Commons course on learn.wiki is published and open for enrolment. At the time of writing this report, there are finished translations in Spanish, French, and Italian. Translations in Portuguese, Basque and German are underway. An OpenRefine-Wikidata course on learn.wiki is in progress and will be finished by the end of June 2024.	Both learn.wiki courses are self-paced and can be followed without a trainer. This removes the need to engage and maintain trainer capacity and offers flexibility for learners.
Scheduled OpenRefine courses for GLAM staff and Wikimedians who engage in larger-scale Wikimedia content partnerships. At least 3 formal, scheduled, guided trainings by members of the liaison group established in point 1. (using the above online courses at https://learn.wiki, with graded homework and personal mentoring)	At least 11 out of the 16 trainees of the train-the-trainer program have delivered OpenRefine-Wikimedia trainings in their community. If each person trained an average of 4 people, then 44 people have been trained so far. As mentioned above, the learn.wiki courses have been designed to be self-paced and are followed without a trainer. At the time of writing this report, the following engagement has been seen: Enrolment / users who finished at least one module / users who finished the course (of whom beta testers): English: 226 / 44 / 14 (5) French: 39 / 10 / 0 Spanish: 33 / 3 / 1 Italian: 14 / 1 / 0
Software development. General maintenance, and bug fixes, to Wikimedia Commons functionalities in OpenRefine	Wikimedia Sverige is working on general maintenance and improvements to Wikimedia Commons functionalities in OpenRefine. This work will be concluded later in 2024. The initial focus is on the feature to enable uploads of larger files.	See in-depth report here.
Strategic partnership discussions on sustainability of OpenRefine’s Wikimedia features. Continued conversations to address reliable support for, and longer-term sustainability of OpenRefine’s Wikimedia / Wikibase features.	This project brought more Wikimedia organizations around the table to discuss future maintenance needs: the Wikimedia Foundation’s Culture and Heritage team, and Wikimedia Sverige. OpenRefine engaged Wikimedia Sverige to work on the Wikimedia Commons features. Separetly from this grant activities, OpenRefine applied to the Support Fund to seek additional funding to integrate with the Wikimedia platform. Although the grant application was not accepted, the process helped OpenRefine strengthen its relationship with the Foundation and explore alternative avenues to support the project.

Story

See a post on the Wikimedia Diff blog, by Giovanna Fontenelle.

Resources

Resources developed in the context of this project are listed on OpenRefine’s project page on Wikimedia Commons and the general OpenRefine info page on meta.wikimedia.org (the latter was reorganized and redesigned by Giovanna Fontenelle, using templates developed by Susanna Ånäs).

Learning and reflection

What worked well

Interest in learning OpenRefine

There is significant interest in learning OpenRefine within the Wikimedia movement. This is evidenced by the high number of candidates for the train-the-trainer course (84) and many participants in WikiLearn courses. This enthusiasm is also seen in spontaneous offers to translate course materials.

Increased engagement

The project certified 11 new OpenRefine trainees and engaged more Wikimedia organizations in supporting OpenRefine’s Wikimedia features.

This grant required more work than anticipated, making the support from Giovanna Fontenelle of the Wikimedia Foundation’s Culture and Heritage team invaluable. More experienced OpenRefine users are also increasingly providing support to new users.

Consultation and feedback

Proactive consultation and feedback during the development of the train-the-trainer and WikiLearn courses have been crucial for their success.

Challenges

Time and effort

This project’s training, support, and documentation work took more time than expected. Due to the complexity and size of Wikimedia support needed around OpenRefine, this work is too large and challenging to be executed by non-affiliated individuals. It should probably have been a group effort from the start, executed by staff from a formal Wikimedia organization. Similar future efforts will require higher resource allocation to ensure fair compensation.

The following list shows actual time spent during this grant, to inform future planning of similar efforts.

Developing and running a train-the-trainer course: circa 200 hours
Following a train-the-trainer course: circa 40-60 hours per trainee
Preparation, running and following up on one in-person OpenRefine-Wikimedia training: 8 hours per training
Major updates to existing documentation: circa 60 hours
Creation of one WikiLearn course, including production of videos: circa 120 hours
Translation cost into one language of one WikiLearn course: circa USD 3,000 (based on word count)
Ad hoc individual support in a “helpdesk” role: 2-4 hours per week. Ideally, each larger language and regional community develops this capacity with dedicated people who speak the language and understand the local context and use cases.

Learning curve

When selecting the cohort of 16 trainees for the OpenRefine-Wikimedia train-the-trainer course, existing knowledge and experience in OpenRefine was one of the deciding factors. Most of the candidates had already used OpenRefine for general data cleaning and/or Wikidata contributions. Some had already been teaching OpenRefine.

Nevertheless, during the train-the-trainer course itself, the group has requested a more in depth explanation of various general software features which they felt insufficiently confident about. Responding to these inquiries, tutorials were organized on:

Using APIs to start a project and retrieve data inside a project
Web scraping, either via other software or using OpenRefine
Reconciliation with non-Wikimedia reconciliation services and with a Wikibase
The basics of GREL
Clustering. Many trainees needed further explanation about this feature as they hadn’t used it yet, were uncertain how to properly use it, and/or wanted to hear more about the mechanism behind it.

In addition, the group regularly spent time looking at the peculiarities and challenges around contributing to Wikimedia Commons, and data modeling. This issue is common to any software for (batch) Wikimedia Commons contributions.

While (as mentioned above) many Wikimedians and partners are enthusiastic to learn OpenRefine, the actual numbers of skilled users stays relatively small (with mostly very data-literate users). While enrolment is very high, the number of completions of the WikiLearn course is (at this moment) still minimal: only 1% of enrolled trainees has completed a training. In a recent poll among Wikimedia Commons contributors, OpenRefine has not shown up as an upload tool that has preference. During more informal conversations with Wikimedia Commons contributors and GLAMs, and in other contexts, there is recurrent clear feedback that OpenRefine’s learning curve is very steep and the time needed to learn it is a barrier, especially for the many contributors who do this as non-specialists, in a volunteer capacity, or as time-challenged staff of partner institutions. Informally, quite a few people are asking for more simple, guided software that doesn’t require as much documentation or training.

This informal insight is confirmed in Andrew Lih’s recent GLAM CSI survey results (44 respondents) (not yet officially published).

“Complexity and Documentation: Tools like OpenRefine and Commons Upload Wizard are complex, with inadequate documentation and tutorials. This complexity creates a steep learning curve for new users and can discourage contributions.”
“Time and Skills: Many respondents mention a lack of time to explore and learn new tools. There is a desire for simpler tools that do not require advanced technical skills or programming knowledge.”

The GLAM CSI project should produce insights in the general (software-agnostic) needs and situation of Wikimedia partners when contributing to Wikimedia projects and will help the Wikimedia movement prioritize the focus of its own resources in this area.

Next steps and opportunities

Resourcing for further maintenance

During this project, basic documentation and training resources have been developed and a first international group of trainers has been established.

Especially if OpenRefine remains the only SDC-focused batch upload software for Wikimedia Commons, additional support structures will need to be in place. The following needs have been expressed:

Establishment of peer learning groups that regularly gather (e.g. on a monthly basis), share tips, and offer mutual help. In an ideal situation, to reduce cultural and language barriers and ensure that people feel comfortable joining and asking questions, each language and regional community would organize and maintain such a group.
Establishment of an international OpenRefine-Wikimedia stakeholder group, inspired by the Wikibase stakeholder group, with regular gatherings, tasked to work together on prioritization of needs, capacity building, and fundraising. Organizing and managing such a group takes approx. 2-4 hours per week.
OpenRefine on Wikimedia Commons is (currently) mainly used by experienced users and it may pose a barrier to uploaders with a lower skill level or less time. During this project, the idea arose to set up a “request an upload” or “request a batch edit” page, for instance on Wikimedia Commons. OpenRefine-savvy volunteers can then do the uploads for the requesters. This suggestion, however, has the risk of creating a backlog of unprocessed upload requests.
Regular trainer-led WikiLearn courses. The existing courses can be lightly adapted to become trainer-led; this will help motivate more people to actually complete these courses. Trainers will probably spend circa four hours per trainee per course run.
Regular updates and maintenance to existing documentation and training materials.

Gaps on Wikimedia Commons: coordination of efforts

Best practices for structured data on Wikimedia Commons are still emerging. There is still a need for the creation of standard data models and templates in many domains; the lack of these is an impediment for any uploader and for any piece of software that wants to encourage contribution to Wikimedia Commons. As a by-product of this project, a group of interested people has been pinged about this subject, but there has not been enough time and capacity to follow up on this. Proactive coordination is urgently needed in this area.

Uptake and impact analysis

The impact of OpenRefine on Wikimedia contributions should be analyzed over time. Baseline metrics have been collected from July 2023 to June 2024 (Google spreadsheet with data points), with reports available on Wikimedia Commons including a summarizing June 2024 report. This will help the Wikimedia community to assess the impact of efforts and investment. Compared to the work and resources dedicated to OpenRefine, how many people are empowered over time to contribute to Wikimedia projects?

The training and documentation efforts that were deployed during this grant will have an incubation period before fully taking effect; it is worth checking in, for instance, six months and a year.