Commons:Village pump/Proposals

From Wikimedia Commons, the free media repository
Jump to navigation Jump to search
Community portal
introduction
Help deskVillage pump
copyrightproposalstechnical
Administrators' noticeboard
vandalismuser problemsblocks and protections

Shortcut: COM:VP/P · COM:VPP

Welcome to the Village pump proposals section

This page is used for proposals relating to the operations, technical issues, and policies of Wikimedia Commons; it is distinguished from the main Village pump, which handles community-wide discussion of all kinds. The page may also be used to advertise significant discussions taking place elsewhere, such as on the talk page of a Commons policy. Recent sections with no replies for 30 days and sections tagged with {{section resolved|1=~~~~}} may be archived; for old discussions, see the archives.

COMMONS DISCUSSION PAGES (index)


Please note
  • One of Wikimedia Commons’ basic principles is: "Only free content is allowed." Please do not ask why unfree material is not allowed on Wikimedia Commons or suggest that allowing it would be a good thing.
  • Have you read the FAQ?

 

Create a Technical Village pump[edit]

COM:VPT is now open for business. --Majora (talk) 22:25, 28 November 2018 (UTC)

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

Currently there is no page called "Commons:Village pump/Technical even though there appears to be some demand for it, a lot of talk pages of tools seem to be unmonitored and often technical questions and/or requests seem to pop up here, at the help desk and at the regular village pump all the time, maybe creating a separate technical village pump could benefit users with more technical requests. This would also be easier for users who do have the technical expertise to help other people to only have to monitor a single place for technical questions. Often I see technical questions in the help desk go unanswered simply because that fortnight no technician bothered to check.

I think that the English-language Wikipedia also has one but I'm not certain. Is this viable? --Donald Trung 『徵國單』 (No Fake News 💬) (WikiProject Numismatics 💴) (Articles 📚) 18:45, 23 October 2018 (UTC)

  • Symbol support vote.svg Support.   — Jeff G. please ping or talk to me 02:51, 24 October 2018 (UTC)
  • Symbol support vote.svg Support I'm all for it. -- Sixflashphoto (talk) 02:56, 24 October 2018 (UTC)
  • Symbol support vote.svg Support I think that this is really a good idea. ℺ Gone Postal ( ) 13:17, 24 October 2018 (UTC)
  • Symbol support vote.svg Support. ReaperDawn 02:38, 27 October 2018 (UTC)
  • Symbol support vote.svg Support. This makes sense. Dan Koehl (talk) 05:21, 27 October 2018 (UTC)
  • Symbol support vote.svg Support very much needed. --✝iѵɛɳ२२४०†ลℓк †๏ мэ 11:35, 27 October 2018 (UTC)
  • Symbol support vote.svg Support in principle, but before it goes up I’d like to see a more detailed proposal, including revisions to the main noticeboard header (which exists in numerous languages), any other relevant navigation templates, and incoming links from Help pages &c.—Odysseus1479 (talk) 16:21, 27 October 2018 (UTC)
@Odysseus1479: Commons:Village pump/Copyright just has autotranslate templates on it. If this proposal has community support and someone could put those templates together and could set up the autoarchive system after its done, just designing the page Commons:Village pump/Technical wouldn't be that difficult. I don't think i'm missing a step other then implementation. Also I am going on the possibly wrong assumption that one could put together Template:Village pump/Technical/Header put it into a page and you don't have to do each language template individually. -- Sixflashphoto (talk) 16:30, 27 October 2018 (UTC)
  • Symbol oppose vote.svg Oppose I don't think a new board is solving the issue, yet a other board will just confuse new users. --Steinsplitter (talk) 15:20, 7 November 2018 (UTC)
  • Symbol support vote.svg Support Can it be created right away? I think someone more interested in technical aspects can do such requests there. George Ho (talk) 20:23, 16 November 2018 (UTC)
  • Symbol support vote.svg Support Sounds like a really good idea Abzeronow (talk) 16:57, 27 November 2018 (UTC)

✓ Done The base has been created at Commons:Village pump/Technical and the header template now exists at {{Village pump/Technical/Header}}. The header has been marked for translation. I have yet to add the page to the {{community tabs}} since I'm not really sure I should do so until it is at least translated a little bit. However, I can do so if that is what people want. --Majora (talk) 22:04, 27 November 2018 (UTC)

@Majora: Thank you. @George Ho made it a requirement in Template:Community tabs/core, resulting in "• [[{{{link-3.3}}}|{{{tab-3.3}}}]]" ugliness in many languages. I made it optional for not yet fully translated languages. Please at least add "| tab-3.3 = technical" and "| link-3.3 = Commons:Village pump/Technical" to {{community tabs/en}}.   — Jeff G. please ping or talk to me 11:48, 28 November 2018 (UTC)

The discussion above is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

Updating Template:Dont remove warnings[edit]

This thread at ANU most recently highlighted the incongruity between this template and Commons:Talk page guidelines, where the template warns users to "not remove legitimate warnings from your talk page", while the guideline simply recognizes the practice as discouraged, while explicitly allowed.

This seems to be a perennial issue. The template's talk page has threads about this very issue going back as far as 2007, and the template has been nominated for deletion twice in 2009 and 2013, with many of the keep !votes themselves observing that the wording was problematic and should be updated to better reflect actual community standards, rather than trying to make policy by fiat via template.

So I'd suggest we update the wording of the template, given that with the checkered history, we can predict with relative certainty this problem is going to come up again if left unaddressed. I propose we update the wording to the following:

العربية | Català | Deutsch | English | Español | فارسی | Suomi | Français | עברית | Magyar | Italiano | 日本語 | Македонски | Nederlands | Polski | Português | Русский | Svenska | Українська | +/−


Dialog-warning.svg
Hello. This is a reminder that removing legitimate warnings and notices from your talk page without addressing the identified issues is discouraged according to our community guidelines. Removing messages does not remove them from the page's history, and doing so is often seen as rude or hostile by the community. You are encouraged instead to archive past discussions. You can have this done automatically for you - simply place {{subst:User:MiszaBot/usertalksetup}} at the top of your user talk page and old messages will be archived after 1 month (see User:MiszaBot/usertalksetup for more details).
If you have received warnings for copyright issues please familiarize yourself with our policy on licensing. You can also ask for help at the village pump or the help desk if you need assistance.
Detailed changes

العربية | Català | Deutsch | English | Español | فارسی | Suomi | Français | עברית | Magyar | Italiano | 日本語 | Македонски | Nederlands | Polski | Português | Русский | Svenska | Українська | +/−


Dialog-warning.svg
Hello. This is a reminder for you to not remove that removing legitimate warnings and notices from your talk page without addressing the issues you received these warnings for identified issues is discouraged according to our community guidelines. Note that removing warnings from your talk page will Removing messages does not remove them from the page's history, and doing so is often seen as rude or hostile by the community. You are welcome to archive your talk page. You are encouraged instead to archive past discussions. You can have this done automatically for you - simply place {{subst:User:MiszaBot/usertalksetup}} at the top of your user talk page and old messages will be archived after 1 month (see User:MiszaBot/usertalksetup for more details).
Please read Commons:Licensing if you have received these warnings for copyright issues.If you have received warnings for copyright issues, please familiarize yourself with our policy on licensing. You can also ask for help at the village pump or the help desk if you need assistance.

This more closely follows the wording in the guideline, and also includes links to VPC and HD for users who may have questions but need assistance. The rest is comparatively minor tweaks in wording, in addition to piping the wikilinks, mostly because, in my personal opinion, we should be doing that anyway when dealing with newer users. GMGtalk 13:48, 6 November 2018 (UTC)

@GreenMeansGo: Perhaps we could expand it a little by adding "and notices" to form "warnings and notices" in two places?   — Jeff G. please ping or talk to me 14:02, 6 November 2018 (UTC)
Why don't see split the difference in the interest of flow? Added "and notices" in the first instance, but then simply downgraded to the more generic "messages" in the second. GMGtalk 14:04, 6 November 2018 (UTC)
@GreenMeansGo: Thanks, that works.   — Jeff G. please ping or talk to me 14:26, 6 November 2018 (UTC)
  • Symbol support vote.svg Support.   — Jeff G. please ping or talk to me 14:26, 6 November 2018 (UTC)
  • Symbol support vote.svg Support I would have said "identified issues is strongly discouraged" but let's not sweat the small stuff. This is certainly an improvement. -- Sixflashphoto (talk) 21:21, 6 November 2018 (UTC)
  • Symbol support vote.svg Support --Packa (talk) 21:34, 6 November 2018 (UTC)
  • Symbol delete vote.svg Delete I still see the wording of this template is not supported by any of our policies. Our "user talk page" policy is very brief. No wonder as people expect much freedom on handling it as it is not a serious business of others. The same in one of our sister project is more clear. "Although archiving is preferred, users may freely remove comments from their own talk pages. Users may also remove some content in archiving. The removal of a warning is taken as evidence that the warning has been read by the user." This is exactly the situation here too, and I see no reason to think other way. And if we read this which made some storm here too in the past, we can easily understand that a civilized community never wish to interact with a fellow using a Badge of shame. When we go through the history of this template, we can see several attempts to mark it as {{deprecated}}. When we go through the talk page, we can see that Lar (one who created this) had a strong feeling that it should be deleted. In the deletion request, Denniss stated that "this template is typically only used for copyvio uploaders resistant to talk page notices and is used as a higher level warning". But it is nowhere documented and we have examples that this template is recently used on the talk page of very legitimate uses which is the immediate reason of this discussion. In my opinion, we work in several Wikimedia projects simultaneously, and having entirely contradicting behavioral policies in different project are difficult to remember and deal with. For all these reasons stated above, I will not support the use of this template. Jee 03:33, 7 November 2018 (UTC)
  • Symbol support vote.svg Support, although I can't see that I often see this template being used. Often I see users who let a wall of warnings pile up, but honestly I think that there should be a special project to help educate users more on copyright rules and what applies to what (including geographically) and that these automated templates will probably get ignored because of their inpersonal nature, but I strongly encourage this form of archiving over the "archived into page history" "archiving". --Donald Trung 『徵國單』 (No Fake News 💬) (WikiProject Numismatics 💴) (Articles 📚) 11:45, 7 November 2018 (UTC)
  • Pictogram voting comment.svg Comment I agree with "discouraged" but have reservations about using this template where it may cause escalation. I have seen users blocked for repeatedly blanking warning notices from their talk pages. As per the guidelines, this may be discouraged, but should not be taken as so hostile as to warrant administrator action. So long as a notice can be presumed as read, there is no need to hound a user who may in truth be annoyed or confused by very large angry sounding notices. If anyone doubts the notices are understood, they can discuss and if necessary take the discussion to a noticeboard. -- (talk) 12:13, 7 November 2018 (UTC)
  • Symbol delete vote.svg Delete per Jee. Agreeing to this wording is agreeing to it being used. Administrators are no different to other users except for some extra buttons, and any user may choose to edit their talk page, remove edits, refuse to engage with individuals as they see fit. I support a shift to the Wikipedia practice/viewpoint where removal of a notice or message is simply taken as acknowledgement that it has been read. If there are certain messages/templates that the community feels absolutely must be retained on talk pages (e.g. serious copyvio notices) then let's have a discussion about them. -- Colin (talk) 12:31, 7 November 2018 (UTC)
I'm fine having a discussion also about updating the guideline. In the meantime, this seems like a comparatively easy fix to at least bring the template and the existing guideline in line with one another. If some folks really are blocking users solely for removing talk page warnings (rather than for the reasons they're getting those warnings to begin with, like repeated copyvio), then I'd say they're out-of-bounds as far as existing policy goes.
Other than that, I'm not really sure I understand the relevance of Administrators are no different to other users. This is mostly to do with our copyvio and vandalism patrollers leaving a template that doesn't conform to existing guidelines. I'm not sure our administrators are really the ones doing most of the front-line patrolling in those areas. GMGtalk 13:37, 7 November 2018 (UTC)
GreenMeansGo, yes some admins have indeed blocked users simply for removing a talk page message (not even an official warning message). And community agreement on this template would be taken as de facto guideline that talk page messages must not be removed. While you may consider this template wrt copyvio/vandalism there are other admins who view anything they do and say as holy and requiring of obedient respect from the proles. -- Colin (talk) 11:53, 10 November 2018 (UTC)
@Colin: Such behavior is unprofessional.   — Jeff G. please ping or talk to me 11:57, 10 November 2018 (UTC)
Jeff G., who's behaviour? The admin or the user being blocked? -- Colin (talk) 12:02, 10 November 2018 (UTC)
@Colin: The admin, of course.   — Jeff G. please ping or talk to me 12:14, 10 November 2018 (UTC)
Well, I'd say that if we have admins and other experienced users who are misapplying current guidelines, that probably isn't helped by a template that itself misunderstands guidelines. Keeping in mind also that for many or even most users here, this isn't their home project. So it would maybe make even more sense to just take a template at its word, and assume that the community did its due diligence in making sure they two align.
But it's looking like the solution here is to 1) update the wording of the template, and then 2) immediately nominate it for deletion. If it survives deletion, then keep the updated wording, and if it don't then it don't. GMGtalk 12:22, 10 November 2018 (UTC)
  • Symbol support vote.svg Support. Whatever our policy is, this is de facto policy. It can't be enforced, but users can be informed about our customs. This template can help with that. - Alexis Jazz ping plz 13:03, 7 November 2018 (UTC)
  • Symbol delete vote.svg Delete the entire template. "The removal of a warning is taken as evidence that the warning has been read by the user". Users should be discouraged from watching other users' talk pages. This will decrease a lot of drama and tensions. 4nn1l2 (talk) 15:38, 7 November 2018 (UTC)
    Exactly! Jee 15:55, 7 November 2018 (UTC)
  • Symbol delete vote.svg Delete the template and add to our guideline something in the kind "Removing legitimate warnings and notices from your talk page without addressing the identified issues is discouraged. Removing messages does not remove them from the page's history, and doing so is often seen as rude or hostile by the community. You are encouraged instead to archive past discussions. The removal of a warning is taken as evidence that the warning has been read by you." Christian Ferrer (talk) 17:33, 7 November 2018 (UTC)
  • Symbol delete vote.svg Delete the template, per Colin and Jkadavoor. "The removal of a warning is taken as evidence that the warning has been read by the user". -- Begoon 12:23, 8 November 2018 (UTC)
  • Symbol support vote.svg Support Deleting it does not seems reasonable. --Steinsplitter (talk) 12:29, 8 November 2018 (UTC)
  • This is not a deletion discussion. To request it's deletion, please start a COM:DR discssion. Per policy it is not speedie-able, and the only other way to delete a page is after a deletion request discussion - something this venue is not. This dicussion is about chaning the wording of the template. --Jonatan Svensson Glad (talk) 13:32, 10 November 2018 (UTC)
As mentioned above, opining upon a wording change implies that using the template is ok, something which those in favour of deleting the template obviously do not believe. I see no problem with expressing that opinion here. Since no feasible changes to wording will fix the problem I cannot support or oppose them, as there is no wording I would support - the template should not be used. Supporting implies the template would then be ok, opposing implies there is other wording I might support - neither is true. Fiddling with the wording may actually encourage people to use this officiously offensive, bitey, big scary warning which is unsupported by policy, believing it to be "fixed" in some way, continuing to needlessly upset users and cause further issues such as those causing this discussion in the first place. I oppose that. I hope that's clearer. Face-smile.svg. -- Begoon 10:01, 11 November 2018 (UTC)
  • Symbol oppose vote.svg Oppose. I propose the following text: "Dear user, this is a reminder that you are in almost complete control over your user space. If anybody attempts to harass you for removing any templates or any other text from your user page, talk page, or their subpages, please report them to COM:ANU." ℺ Gone Postal ( ) 19:44, 10 November 2018 (UTC)
  • Symbol delete vote.svg Delete the template - Unblock requests aren't allowed to be deleted, Other than that you can add and remove anything else as you please. –Davey2010Talk 20:14, 10 November 2018 (UTC)
  • Symbol support vote.svg Support.Vulphere 05:03, 11 November 2018 (UTC)
  • Symbol support vote.svg Support I support the changes as it is less harsh than the current wording. However, I would be fine with deleting the template if it came to that. While archiving is preferred there is absolutely nothing wrong with just removing the section as it is still in the history. Warnings and such should never be used as a mark of shame that can never be removed. I take removal as an acknowledgement of the notice. If the behavior continues after the fact that just increases the likely of sanctions as, to me, they have acknowledged the problem and chose to ignore requests to change course. --Majora (talk) 04:01, 19 November 2018 (UTC)
  • Symbol support vote.svg Support re-wording, but prefer Symbol delete vote.svg deletion, as per many others above. --El Grafo (talk) 13:27, 20 November 2018 (UTC)

RfC: Musical notation files[edit]

Should MusicXML, MuseScore or LilyPond files be allowed to be uploaded to Commons? Jc86035 (talk) 10:55, 16 November 2018 (UTC)

(Note: MEI was added after the RfC opened. MNX is presumably not an option because it doesn't exist yet.) Jc86035 (talk) 10:14, 26 November 2018 (UTC)

Background[edit]

It may be desirable to integrate musical notation files into MediaWiki, most likely by allowing their display as thumbnails and possibly allowing audio playback from them. In November I made a Community Wishlist Survey proposal to propose the technical development of such capability. Even though the proposal has not succeeded, it may still be advantageous to host these files on Commons without using them directly in other projects.

The benefits of having notation files (as opposed to PDFs/images or audio) include their being machine-readable and that they can be converted to audio and more easily reformatted and edited. The benefits of uploading them to Commons, instead of using the notation directly in articles (with the Score extension), include easier transclusion and sharing and better visibility.

The three file formats each have advantages and disadvantages, and it may be thus beneficial for Commons to support more than one format (or even all three of them). LilyPond and MuseScore are the primary libre notation software programs, although they are quite different in their approaches.

  • MusicXML (.mxl) is an open standard codified by a W3C community group (although not an official W3C standard), and export and import to it are widely supported by both proprietary and libre notation programs; but most programs have better support for their own notation file type than for MusicXML, resulting in minor display issues upon import (as well as playback issues for MuseScore specifically). Some editors may wish to upload the source file rather than be forced to perform a conversion. The MusicXML format specification is no longer being actively developed (although not abandoned) as the W3C community group have decided to focus on a new format called MNX, which is intended to overcome many of MusicXML's limitations. However, MNX is still at the early development stage and is not ready for implementation.
  • MuseScore (.mscz) is quite versatile and is focused on MIDI-style playback as well as score display, and the software is a graphical scorewriter (like Sibelius and Finale). It is possibly the world's most popular scorewriter, surpassing even Sibelius and Finale. However, in the past it has had compatibility issues between versions (related to opening files created with older versions of the software), and the file format is defined entirely by the software's source code (which is open source). Due to its lack of a separate specification, a full MediaWiki integration with editing capabilities would potentially be more difficult than for MusicXML. However, a MuseScore viewer would be simple to create using MuseScore's existing open source renderer to generate synced SVGs and audio files on the backend.
  • LilyPond (.ly) is already supported in MediaWiki as mw:Extension:Score. Due to being much like a programming language, it is easy to write as source code, but more difficult to learn than MuseScore for most people. It does not have a complete GUI, although Frescobaldi (advanced code editor) and Denemo (limited GUI) are helpful in this regard. LilyPond is the only major scorewriter not to support exporting to MusicXML, so files created with LilyPond cannot be opened in other score editors, such as MuseScore, Finale or Sibelius.
  • MEI (.mei) is a community-developed application-agnostic format. It was developed as a comprehensive format for encoding all kinds of music notation (not just common-practice, post-1750 notation), so it supports other notation styles (tablature, mensural, neumes). It also supports a wide range of editorial features (multiple source readings, annotating corrections, additions, etc.). As a result, it is quite an extensive and, perhaps, complex specification. MEI lacks widespread desktop notation editor support, but it is supported by a JavaScript-based engraver, Verovio. Verovio supports MEI playback (MIDI), PDF export, and can also render MusicXML, **kern, and Plaine & Easie notation.
    — Preceding unsigned comment added by Ahankins (talk • contribs) 12:56, 28 November 2018 (UTC)

MIDI, incidentally, is already supported as a file format on Commons. However, playback requires passing the files through the Score extension, and MIDI is audio-only and may not render very nicely.

Survey: Allow MusicXML files on Commons[edit]

  • Symbol support vote.svg Support Jc86035 (talk) 10:55, 16 November 2018 (UTC)
  • Symbol support vote.svg Support --Donald Trung 『徵國單』 (No Fake News 💬) (WikiProject Numismatics 💴) (Articles 📚) 11:19, 16 November 2018 (UTC)
  • Symbol support vote.svg Support.   — Jeff G. please ping or talk to me 12:10, 16 November 2018 (UTC)
  • Symbol support vote.svg Support Abzeronow (talk) 23:07, 16 November 2018 (UTC)
  • Symbol support vote.svg Support -Theklan (talk) 17:53, 17 November 2018 (UTC)
  • Symbol support vote.svg Support - De728631 (talk) 18:07, 17 November 2018 (UTC)
  • Symbol support vote.svg Support W3C standard, should be a solid format. HLHJ (talk) 03:30, 18 November 2018 (UTC)
  • Symbol support vote.svg Support --Wesalius (talk) 22:53, 18 November 2018 (UTC)
  • Symbol support vote.svg Support (Hoping that it becomes like MathML with rendering natively supported by some browsers) Ebe123 (talk) 02:39, 19 November 2018 (UTC)
  • Symbol support vote.svg Support per HLHJ. Vulphere 12:03, 20 November 2018 (UTC)
  • GA candidate.svg partial support The MusicXML format specification is no longer maintained as the W3C community group have begun development of a new format called MNX. MusicXML support is valuable for the back catalogue of scores available and for interchange with proprietary programs, but it is not to be encouraged for new scores or for existing scores that are available in other open formats. (OpenScore Project Manager) --Peter Jonas (shoogle) (talk) 21:53, 20 November 2018 (UTC)
  • GA candidate.svg partial support I’d have voted fully for this if it were not “no longer maintained” as most scorewriters can export to this. mirabilos (talk) 23:29, 20 November 2018 (UTC)
  • Symbol support vote.svg Support Peter Jonas's comments about MusicXML above are not correct. MusicXML is indeed maintained by the W3C Music Notation Community Group, and MusicXML was designed just for the type of score usage being proposed here. Its use for new and existing scores is highly encouraged. I apologize for any misunderstanding caused by unclear communication in the Community Group. Mdgood (talk) 22:33, 26 November 2018 (UTC)
  • Symbol support vote.svg Support We need to open up to many more file types. —Justin (koavf)TCM 22:53, 26 November 2018 (UTC)
  • Symbol support vote.svg Support Most of the existing tools will generate MusicXML format most easily of these choices, and most of the existing symbolic notation content will likely be in MusicXML content. Per Mdgood, MusicXML is supported and appropriate for this use. JimDeLaHunt (talk) 03:15, 27 November 2018 (UTC)
  • Symbol support vote.svg Support Can be rendered with a client-side JavaScript (Verovio) making it easier to integrate than server-side rendering. --Ahankins (talk) 23:16, 27 November 2018 (UTC)
  • Symbol support vote.svg Support many score writers are able to export MusicXML files, and being XML it's fairly easy to read and understand (both for humans and machines) --Stadlerpeter (talk) 07:22, 28 November 2018 (UTC)
  • Symbol support vote.svg Support Agathoclea (talk) 08:51, 29 November 2018 (UTC)
  • Symbol support vote.svg Support widely used --KriSte Mühe (talk) 21:49, 5 December 2018 (UTC)

Survey: Allow MuseScore files on Commons[edit]

  • Symbol support vote.svg Support Jc86035 (talk) 10:55, 16 November 2018 (UTC)
  • Symbol support vote.svg Support --Donald Trung 『徵國單』 (No Fake News 💬) (WikiProject Numismatics 💴) (Articles 📚) 11:19, 16 November 2018 (UTC)
  • Symbol support vote.svg Support.   — Jeff G. please ping or talk to me 12:10, 16 November 2018 (UTC)
  • Symbol support vote.svg Support -Theklan (talk) 17:53, 17 November 2018 (UTC)
  • Symbol support vote.svg Support - De728631 (talk) 18:07, 17 November 2018 (UTC)
  • Symbol oppose vote.svg Oppose MuseScore's online WYSIWYG player/editor seems to be closed-source and freemium, so Wikimedia might wind up writing one, which might not wow MuseScore as it would kill their profit model. The MuseScoreformat is apparently not formally specified and seems to have had some version problems in the past, meaning we could have compatibility problems. I assume that it is also higher-bandwidth than the markup alternatives, making it less accessible.(Mirabilos says it's comparable, so I assumed wrongly) I think there may be better format options. I would support upload-and-convert support for MuseScore. See also wishlist discussion. HLHJ (talk) 03:27, 18 November 2018 (UTC)
  • Symbol oppose vote.svg Oppose per HLHJ's rationale. Abzeronow (talk) 17:27, 18 November 2018 (UTC)
  • Symbol oppose vote.svg Oppose Going one step further from HLHJ, upload-and-convert would defeat the purpose of allowing MuseScore files on Commons in my view. I support any initiative for a WYSIWYG editor for the Score extension: phab:T49528. Ebe123 (talk) 02:39, 19 November 2018 (UTC)
  • Symbol support vote.svg Support OpenScore uses MuseScore file format (reasons in discussion). MuseScore would not view Wikimedia hosted files as a threat. (OpenScore Project Manager) --Peter Jonas (shoogle) (talk) 21:44, 20 November 2018 (UTC)
  • Symbol support vote.svg Support This is very active, Open Source, has decent import format from MusicXML and a plethora of other formats, and an immensely active community, especially in the area of Free Music. mirabilos (talk) 23:30, 20 November 2018 (UTC)
  • Symbol support vote.svg Support We need to open up to many more file types. —Justin (koavf)TCM 22:54, 26 November 2018 (UTC)
  • GA candidate.svg Weak support From a tools and expressiveness point of view, MuseScore probably does the best job of the supplied options of hitting a sweet spot for Commons. MuseScore is a freely-available notation editor which can tweak notation to look correct. The format has been used successfully for high-quality notation typesetting. However, it's still primarily an application's private data format, rather than a content representation and interchange format. Long-term, I expect MEI and/or MNX will get closer than MuseScore to the sweet spot for Commons. JimDeLaHunt (talk) 03:20, 27 November 2018 (UTC)
  • Symbol oppose vote.svg Oppose The MuseScore format is an internal ("proprietary" in the sense that it is software-specific) format that can change according to the needs of the software. --Ahankins (talk) 23:18, 27 November 2018 (UTC)
  • Symbol oppose vote.svg Oppose per HLHJ's, Ebe123's, JimDeLaHunt's & Ahankins's rationale
    — Preceding unsigned comment added by KriSte Mühe (talk • contribs) 22:00, 5 December 2018 (UTC)

Survey: Allow LilyPond files on Commons[edit]

  • Symbol support vote.svg Support Jc86035 (talk) 10:55, 16 November 2018 (UTC)
  • Symbol support vote.svg Support. --Donald Trung 『徵國單』 (No Fake News 💬) (WikiProject Numismatics 💴) (Articles 📚) 11:20, 16 November 2018 (UTC)
  • Symbol support vote.svg Support.   — Jeff G. please ping or talk to me 12:10, 16 November 2018 (UTC)
  • Symbol support vote.svg Support Abzeronow (talk) 23:08, 16 November 2018 (UTC)
  • Symbol support vote.svg Support -Theklan (talk) 17:54, 17 November 2018 (UTC)
  • Symbol support vote.svg Support - De728631 (talk) 18:07, 17 November 2018 (UTC)
  • Symbol support vote.svg Support Lilypond is integrated with a lot of things, including MuseScore (above) and Mediawiki (renders audio and score images). Lilypond markup fits with the markup-based MediaWiki, but it would also be good to have a Visual Editor version. HLHJ (talk) 03:28, 18 November 2018 (UTC)
  • Symbol support vote.svg Support--Wesalius (talk) 22:58, 18 November 2018 (UTC)
  • Symbol support vote.svg Support Ebe123 (talk) 02:39, 19 November 2018 (UTC)
  • Symbol support vote.svg Support. Vulphere 12:03, 20 November 2018 (UTC)
  • GA candidate.svg partial support Lilypond's text syntax is nice but not for everyone. Unlike MuseScore, Lilypond does not support exporting to MusicXML, so Lilypond files cannot be opened in popular music notation programs like MuseScore, Finale and Sibelius. (OpenScore Project Manager) --Peter Jonas (shoogle) (talk) 23:05, 20 November 2018 (UTC)
  • Symbol oppose vote.svg Oppose GNU LilyPond’s format is basically a dead-end; some can convert to it but almost nothing from it. (And lilypond itself depends on a version of GNU Guile that’s so old that Debian’s is too new ☺) It’s also obfuscatable to the highest degree: you can embed a Scheme/GNU Guile program in it which is then run, generating music on the fly. This is also a possible security issue. For simple, user-supplied scores, the existing support is enough; for complex scores and fine-tuned layouts, MuseScore would be better and much more interchangeable. mirabilos (talk) 23:42, 20 November 2018 (UTC)
    • See reply in discussion on Debian update and third-party GPL LilyPond import/export to MusicXML. HLHJ (talk) 06:16, 23 November 2018 (UTC)
  • Symbol support vote.svg Support --Phil Holmes (talk) 14:02, 26 November 2018 (UTC)
  • Symbol support vote.svg Support -- Lemzwerg (talk) 17:24, 26 November 2018 (UTC)
  • Symbol support vote.svg Support We need to open up to many more file types. —Justin (koavf)TCM 22:54, 26 November 2018 (UTC)
  • GA candidate.svg Weak support Lilypond's text syntax will do the best job in the short term of providing notation excerpts to serve as illustrations to text-based articles. However, I agree with mirabilos that it is a dead-end content format. All of MusicXML, MEI, MuseScore, and (eventually) MNX will be better choices for full-score content. JimDeLaHunt (talk) 03:24, 27 November 2018 (UTC)
  • Symbol support vote.svg Support would be great to see this latex'ish format to be supported --KriSte Mühe (talk) 21:49, 5 December 2018 (UTC)

Survey: MEI/Verovio[edit]

  • Symbol support vote.svg Support as belated proposer (see table in discussion below). This format would be a tie to a whole additional community of users, academics (who work a lot on public domain music, and might even have funding). HLHJ (talk) 05:15, 23 November 2018 (UTC)
  • Symbol support vote.svg Support.   — Jeff G. please ping or talk to me 05:31, 23 November 2018 (UTC)
  • Symbol support vote.svg Support Abzeronow (talk) 16:26, 23 November 2018 (UTC)
  • Symbol support vote.svg Support. --Donald Trung 『徵國單』 (No Fake News 💬) (WikiProject Numismatics 💴) (Articles 📚) 13:06, 24 November 2018 (UTC)
  • Symbol neutral vote.svg Neutral; although I would be fine with it I would be worried that not a lot of people would actually use it (and it would require more extra work to integrate it into MediaWiki, seeing as the MusicXML converter is still in development). Also, given the number of file formats, this. Jc86035 (talk) 13:24, 24 November 2018 (UTC) (edited 13:40, 26 November 2018 (UTC))
  • Symbol support vote.svg Support We need to open up to many more file types. —Justin (koavf)TCM 22:55, 26 November 2018 (UTC)
  • Symbol support vote.svg Support Long-term I think MEI will be a better content format for Commons than MusicXML, MNX, MuseScore, or LilyPond, because the academics and musicologists who are building MEI have similar goals to express music notation and works built on music notation. Short-term, there is the problem that MEI doesn't have much support from notation editors. That will change. JimDeLaHunt (talk) 03:27, 27 November 2018 (UTC)
  • Symbol support vote.svg Support A number of emerging digital editions have chosen MEI as their file format for long-term preservation and new digital editions. For example, it is in use by the Mozarteum, The Bach Archiv, the Beethoven House Bonn, and a number of others. MEI is actively developed (4.0.0 released this fall) with a broad community behind it. It has elected governance[1], supports an annual conference[2], and is provided under a liberal license[3].--Ahankins (talk) 23:14, 27 November 2018 (UTC)
  • Symbol oppose vote.svg Oppose MEI is a specialized format highly tailored for music research. Its intended audience of music historians, librarians, and theorists does not appear to match the Wikimedia audience very well. Multiple formats likely makes things more difficult for both users and implementers than choosing one standard format. Mdgood (talk) 15:57, 27 November 2018 (UTC)
    • I'm not sure what to make about this -- MEI can be used for specialized tasks, but only because it has the features to support them. It also has bog-standard music encoding features. So the fact that it can be used for more than simple encodings, but that it still *can* do simple encodings, doesn't really seem to be a reason to oppose it? --Ahankins (talk) 23:21, 27 November 2018 (UTC)
    • Commons loves GLAM, and scholars! Our audience is anyone interested in disseminating knowledge. HLHJ (talk) 04:03, 28 November 2018 (UTC)
    • The problem is that MEI is not designed for anyone other than music researchers. It was created as a specialized format for music scholars without compromise for other needs. That is totally appropriate but has also made it notoriously difficult to implement in software. MusicXML has a much better track record of collaboration and support for different musical communities. And again, I think you will find that multiple formats make things more difficult and confusing for everyone, not easier. Mdgood (talk) 19:48, 28 November 2018 (UTC)
    • It's not true that MEI is designed for scholars only – it just offers multiple perspectives on music notation, including, but clearly not limited to musicological ones. Feature-wise, it's a superset of MusicXML, with lots of additional aspects for all kinds of needs (syncing with AV files, anyone?). For Commons, the featureset of MusicXML is probably sufficient, but MEI is currently working on a customization / profile / subset called "MEI Basic", which allows full roundtripping between MusicXML and MEI. It's also not true that there is no data available in MEI – for instance, all of Mozart is going to be published as MEI files by the Mozarteum in Salzburg over the next year(s), and the Bach-Archiv offers the catalogue of Bach Digital as MEI (metadata). Sure, the number of files is smaller than with MusicXML, but those files are often of very high quality and free to use. (disclaimer: I'm deeply involved with MEI) Jkepper (talk) 22:35, 28 November 2018 (UTC)
  • Symbol support vote.svg Support MEI is a well documented, open standard, aiming at supporting all sorts of notational features. --Stadlerpeter (talk) 07:17, 28 November 2018 (UTC)
  • Symbol support vote.svg Support MEI and Verovio are both developed with long-term perpectives in mind, and for that reason the seem quite appropriate for Commons - disclamer: developer of Verovio. --Lpugin (talk) 14:16, 28 November 2018 (UTC)
  • Symbol support vote.svg Support MEI supports very rich metadata, so besides rendering scores, it could feed the metadata sections of Mediawiki (although I don't know how that works). There is also an API (EMA [4]) in the MEI world (though not technically restricted to MEI) that allows to identify sections of an MEI file. This could be very useful to have full encodings stored on Commons, and only specific snippets shown in articles. (disclaimer: deeply involved in MEI…) Jkepper (talk) 22:35, 28 November 2018 (UTC)
  • Symbol support vote.svg Support It includes modules for neumes and mensural notation, so the supported time period covers about 1000 years of music history. Possibly useful if the scope of Commons is not to be restricted on notation betwenn 1600 and 1950 only --KriSte Mühe (talk) 22:03, 5 December 2018 (UTC)

Survey: Braille music[edit]

  • Symbol support vote.svg Support as belated proposer (see table in discussion below). This format would be a tie to a whole additional community of users; blind musicians, partially-sighted musicians, and sighted music transcribers. It is technically a fairly simple format, and rough-but-usable automated conversion looks like one of the easier technical challenges here. Being able to display the LilyPond excerpts in articles as Braille would be a good step for accessibility. HLHJ (talk) 02:45, 28 November 2018 (UTC)
  • Symbol support vote.svg Support Would be great to have an accessible format supported. --Ahankins (talk) 11:26, 28 November 2018 (UTC)
  • Symbol support vote.svg Support Abzeronow (talk) 16:22, 28 November 2018 (UTC)
  • Symbol support vote.svg Support. --Donald Trung 『徵國單』 (No Fake News 💬) (WikiProject Numismatics 💴) (Articles 📚) 17:29, 29 November 2018 (UTC)
  • Symbol support vote.svg Support in terms of increased accessibility --KriSte Mühe (talk) 22:04, 5 December 2018 (UTC)
  • Symbol support vote.svg Support.   — Jeff G. please ping or talk to me 06:25, 8 December 2018 (UTC)

Survey: ABC notation[edit]

Discussion[edit]

Consensus summary table[edit]

This table attempts to summarise a community consensus from the discussion below. Please edit and improve.

Format Advantages Disadvantages Technical difficulty Lacks
MusicXML
  • common interchange standard, widely supported by notation editors and processing software
  • large user community and W3C Community Group
  • proven ability to represent advanced level notation
  • uses XML standards (XSD and DTD validation, XSLT)
  • different editors export it differently
  • XML standard implementation is not complete (no 'xml:id' support, no namespace)
  • can be imported by almost any player or notation editor
  • can be displayed in browser by Verovio (see example)
  • plug-and-play open source web renderer (many web renderers exist, but they are proprietary; LGPL Verovio (below) has beta rendering[5])
MEI
  • wide and growing range of music cultures (including historic ones) encodable
  • demonstrated ability to represent advanced level notation
  • uses XML standards extensively, making it suitable for Linked Data applications, schema validation, XPath/XSLT, etc.
  • focuses on 'semantic' markup of notation (i.e., what the symbols mean) rather than visual markup (i.e., what the score looks like)
  • active international community; elected governance
  • really sophisticated engraving (making pretty scores for print) requires custom software integration (Verovio)
  • difficult for spec-complete software implementations unless subsetted as in Verovio or MEI Basic (under development)
  • focus on semantic markup makes it more difficult to encode precise engraving (e.g., 'this slur must be so far past this note head')
  • Verovio rendering library might be a useful free-libre component
  • liberally-licensed formal specification and source code
  • (With Verovio) client-side rendering means less ongoing server-side rendering software support necessary
  • (With Verovio) SVG output can be used for dynamic highlighting and interaction with rendered scores (CSS, JavaScript, XPath)
  • extensive software editor support (programs? licensing? web-based editors?)
  • large corpus of existing material
MuseScore
  • large community
  • proven ability to represent advanced level notation
  • online/mobile freemium elements
  • essentially an editor-specific private data format
  • online visual editor and playback software exists, including mobile version, but is proprietary; would have to either duplicate and compete, or partly rely on third-party freemium service
  • copyleft online player/editor
  • formal specification
LilyPond
  • already integrated into MediaWiki, fits markup orientation, unicode
  • proven ability to represent advanced level notation
  • natively imports from, but does not natively export to, MusicXML (or any other format) (LilyPond->MusicXML export via several other GPL programs)
  • open-source visual editor and playback software (not web-based) exists, but see phab:T49528 (mobile might be harder)
ABC
  • already integrated into MediaWiki
  • ...
  • ...
  • ...
  • ...
MNX
  • should be perfect in every way :)
  • automatic conversion from MusicXML to MNX-Common should be possible
  • MNX-Generic provides a way to display and play any repertoire using SVG and audio files
  • does not yet exist, though MNX-Generic has a prototype web player
  • a full specification and an MNX-Common implementation
Braille music
  • usable by the blind, large digitized collections exist,[6] active community of users and transcribers
  • well-defined format[7]
  • format is somewhat culture-bound and may not be able to represent all musical traditions
  • copyleft visual/tactile non-markup editor exists[8] (the format resembles ASCII).
  • copyleft webpage display software exists (Braille or a colour- and text-coded visual isomorphism thereof).[9]
  • copyleft conversion tools exist from MusicXML and to MusicXML and LilyPond.[10][11]
  • Mediawiki integration

Discussion text[edit]

@HLHJ: There are good reasons for not supporting MuseScore as much as MusicXML, but I don't think these are enough to prevent users from merely uploading them. Upload-and-convert would sort of miss the point, since anyone who has the MuseScore application can already convert to MusicXML with the application. The main purpose of allowing MuseScore uploads at all would be to allow users to share files with MuseScore-specific notation that is erased when converting to MusicXML (e.g. pizzicato/harmon mute MIDI controls, some styling and positioning).

MuseScore 2 is much more mature than MuseScore 1 was at the time MuseScore 2 was released, so I doubt that there will be any further major compatibility issues. MuseScore 2 is also more than five years old now, so the vast majority of scores in existence wouldn't suffer from those issues. However, this is purely speculation, because we don't know how stable MuseScore 3 will be when it's released.

I don't think the difference in file size is enough to disqualify MuseScore files, given that orchestral scores with parts are usually more compact than average JPEGs (even the largest files probably wouldn't exceed a megabyte in any of the three formats).

I wouldn't worry about Commons cannibalizing the MuseScore online player or MuseScore Pro. MuseScore is free and open-source software, and there's nothing wrong with using it in MediaWiki. If Commons's infrastructure becomes able to compete with theirs (i.e. if it's good enough that someone starts a freely-editable MediaWiki wiki that accepts arbitrary non-free scores), then MuseScore Pro becomes an unsustainable way to earn revenue (and they do still have their mobile apps, and are now part of Ultimate Guitar as well). Do you know of any other file sharing service which limits users to five files? Jc86035's alternate account (talk) 09:28, 18 November 2018 (UTC) (edited 13:47, 24 November 2018 (UTC))

Regarding LilyPond, MuseScore does not natively export to LilyPond, and LilyPond does not natively export to either MuseScore or MusicXML. So, (a) any conversion requires going through MusicXML, and (b) there is no "official" way to convert from LilyPond back to MusicXML, as far as I know. Jc86035's alternate account (talk) 09:37, 18 November 2018 (UTC)

"MuseScore-specific" notation? Don't think there are any; MusicXML and LilyPond are much more open with what they do support, and I've already shown you that your listed examples are included. Further, MusicXML and LilyPond support more notations and customizations, and LilyPond has better output which may save the need for tweaking styles :)
Due to the unclear format specifications of the MuseScore format and the errors/"corruptions" found in MuseScore files (even in version 2), the maturity of the software is not at issue here. I have also stated the problems with the format before.
Don't really care about the file-size or cannibalization arguments. Please refute my claims (which I have substantiated) before repeating them elsewhere. Thanks! (Maintainer of the Score extension) Ebe123 (talk) 02:39, 19 November 2018 (UTC)
@Ebe123: I should have worded that differently (I've struck out "MuseScore-specific", since this is incorrect).
Presumably most MuseScore users, of which there are a not-insubstantial number, don't really care about the format being problematic, or at least tolerate it. At the very least, almost all .mscz scores made in MuseScore 2 shouldn't have any corruption issues when opening them in another instance of MuseScore 2, so it's not like every single MuseScore file uploaded to Commons would have corruption issues.
Regardless of the issues, if scores are being sourced from musescore.com (and there are a lot of scores with appropriate licensing there, although it would probably be necessary to cherry-pick uploaders without a track record of copying other people's scores) then keeping the original file on Commons would be beneficial (mainly for re-use and for PDF/SVG/OGG/MIDI export) if the Commons editor has modified the MuseScore file in any way. I would still prefer to allow MuseScore uploads, even if it's solely to compensate for MuseScore's limitations. Jc86035's alternate account (talk) 08:33, 19 November 2018 (UTC)
I don't think you understand my point about corruption. I am saying there are incompatibilities between the application and its own file format (in the same version), which leads to potential corruption. You're right that it "shouldn't" but it does (the frequency of occurrences is besides the point here but you're right that not all would) and you acquiesce to this. Furthermore, even if MuseScore users can "tolerate" this (do not believe that, but okay), the file format and its flaws must be taken into consideration if we are planning to support public upload and conversion of said files.
I do not understand the next paragraph. We do have bots like User:Flickr Upload Bot to source files from the web, but sourcing is outside the scope of this discussion. File versioning would be the same, again out of scope. "I would still prefer to allow MuseScore uploads, even if it's solely to compensate for MuseScore's limitations." doesn't make sense. Clarify? It seems to suggest creating a new MuseScore renderer. Ebe123 (talk) 18:00, 19 November 2018 (UTC)
@Ebe123: Sorry, I meant that it would be beneficial for users using the MuseScore app to convert files, not that Commons should be supporting automatic conversion to those formats. I'm not expecting anything more than uploads being allowed (for all three formats), particularly given that the wishlist proposal does not have anything near enough support to succeed.
Regarding sourcing, I meant that it would be a fairly obvious course of action to make use of at least some of the appropriately-licensed MuseScore files hosted at musescore.com (OpenScore scores: 1, 2). "Versioning" isn't really what I meant; what I was trying to get across was that if someone would want to re-upload or otherwise reuse a score in MuseScore format, it would be beneficial to them for the original file to be uploaded as a MuseScore file.
Not that I think the corruption issues are trivial, but file corruption is not unique to MuseScore or the MuseScore format, and corruption isn't a security risk in this case so (at least from the WMF's point of view) it shouldn't really affect whether or not uploads should be allowed. Furthermore, even if LilyPond is objectively better, it shouldn't prevent Commons from hosting MuseScore files; Commons still allows five different audio formats, and allows TIFF "as a courtesy" even though PNG is preferred. And MuseScore is, by every metric that I know of, more popular than LilyPond (almost twice by English Wikipedia page views, 25 times by Google searches, Alexa rank, questions asked on StackExchange…). In any event, file corruption was not enough to get me or anyone I know to switch from MuseScore to LilyPond/Frescobaldi/Denemo, although of course others may have had different experiences.
Incidentally, Wikimedia Belgium and/or its members have already had some sort of collaboration with MuseScore. Romaine proposed MuseScore ID (P4097) on Wikidata as part of this, although it's only used for ten OpenScore transcriptions so far.
@HLHJ, Abzeronow: In summary:
  • MuseScore is almost certainly more widely used than LilyPond by a significant margin;
  • All other scorewriters are either less popular than LilyPond or proprietary;
  • Because LilyPond doesn't have conversion to MusicXML, a large proportion of those uploading MusicXML files would have created the files with MuseScore anyway;
  • Commons is not limited to supporting only two more file types instead of three;
  • There is precedent (in MIDI) for allowing uploads for files that cannot be displayed on pages;
  • There is precedent (in TIFF and perhaps in MP3) for allowing uploads for common file formats even where better alternatives exist;
  • The file corruption issues do not occur especially frequently (especially not with newer scores), do not usually result in total data loss and shouldn't be problematic for security; and MuseScore is incomparable to LilyPond in this regard because LilyPond files are normally edited as source code (so corruption can only be caused directly by the user);
  • Only allowing MusicXML files to be displayed on pages and not MuseScore files may still be beneficial for re-users of the files because of import/export inconsistencies (regardless of whether or not MuseScore itself causes the problems);
  • Users may be deterred by only being able to upload files as .ly or .mxl, and people may find it easier to re-use Commons files which are uploaded twice as both .mscz and .mxl.
Jc86035's alternate account (talk) 08:29, 20 November 2018 (UTC)
(I would also note that MusicXML is not actually a Web standard yet – the status just means that some people formed a community group and put the specification on the W3C's website. It's not insignificant or a bad thing, but I think it's important to make the distinction to avoid making it seem like the other formats aren't legitimate.) Jc86035's alternate account (talk) 09:15, 20 November 2018 (UTC)
The MusicXML format specification is no longer maintained and the W3C Music Notation Community Group has switched to developing a new format called MNX (MNX is not yet ready for implementation). The reason this is happening is MusicXML was only ever intended as an interchange format for passing musical information between proprietary notation software. Its design makes it unsuitable for use as a native format, or for storing a working copy of a file, and attempting to use it as such results in loss of information compared to other formats. It is suitable for use when there is no other open format available (such as when exporting from a proprietary program), but should not be used for scores that were created in another open format, such as MuseScore or Lilypond. MuseScore is not without its flaws (what program is?) but it has improved greatly over the years. It's format is free and open source, and provides layout as well as semantic information, which is one of the main reasons why it is used by the OpenScore project (the other reasons being that MuseScore's desktop editor is free and open source, translated into over 40 languages, and is the most popular notation program according to Google Trends). Lilypond currently does not have a MusicXML exporter, so files created with Lilypond are not usable by other notation programs. (OpenScore Project Manager) --Peter Jonas (shoogle) (talk) 19:18, 20 November 2018 (UTC)
Thanks to Mirabilos for replying to me on m:Community Wishlist Survey 2019/Multimedia and Commons/Musical notation – files on Commons (rendering and playback), graphical editing. To keep the discussion together, I'll quote that reply:

“online WYSIWYG player/editor seems to be closed-source and freemium”: They use an OSS renderer internally on some places.

I’m also pretty sure that, if Wikimedia were to approach MuseScore/Ultimate Guitar, a Free solution could be found for code reuse suitable enough to make it usable here. (An online editor, though, is likely out of scope.) The profit model is to sustain development. Wikimedia adoption would rather drive more people to use it, so I’d think that would be more welcome. The MuseScore developers are very approachable and like the Free Software scene.

“I assume that it is also higher-bandwidth than the markup alternatives”: no. MuseScore files come in two flavours. *.mscx and *.mscz. The latter is the default, and it’s a PKZIP container with the mscx in it as well as any images etc. embedded into the score. The mscx is XML and comparable to MusicXML, but in my personal experience, MusicXML is more verbose. (MusicXML also has a compressed container format, *.mxl, but that’s rarely used.) I keep all my own scores as *.mscx in git and just batch-convert to other formats occasionally, and have written an MML to MusicXML exporter, so I have experience with both formats.

I'm happy to take your word on files sizes, thank you for the info. Do you mean that an online editor might not be doable by the Community Tech Team, or that MuseScore/Ultimate Guitar wouldn't relicense theirs? There is clearly demand for a visual editor of music markup, with an aural player. People would want it to run on mobile, too. MuseScore's profit model is selling freemium use of their online and mobile software; their desktop software is fully open-source. If Wikimedia adoption of MuseScore made more people buy closed-source software from Musescore, that might be a problem for Wikimedia, especially if it meant lower-quality tool access for people with less money, as it would be a source of systemic bias. In the developing world, desktops are rarer, but mobiles are becoming more and more common (while bandwidth is expensive, hence my concern with size). If, on the other hand, the visual editor and player on Commons became favoured, and Commons competed with MuseScore's online and mobile products, MuseScore would no longer have a profit model. It could be a bit sticky either way. I'm not in any way condemning the MuseScore developers, who have written a great piece of free software that a lot of people clearly really appreciate. And if what I've seen of Peter Jonas (shoogle) is any guide, they are very nice people. His statement that "MuseScore would not view Wikimedia hosted files as a threat" is reassuring.
It looks to me as if the trade-offs run:
Format Advantages Disadvantages Technical difficulty Lacks
MusicXML *common interchange standard *being replaced by MNX *can be imported by ~any player, designed as an interchange format, not for display or editing *ongoing support
MuseScore *large community *online/mobile freemium elements *online visual editor and playback software exists, including mobile version, but is proprietary; would have to either duplicate and compete, or partly rely on third-party freemium service *copyleft online player/editor
LilyPond *already integrated into Mediawiki, fits markup orientation, unicode *does not export to MusicXML, only imports *open-source visual editor and playback software (not web-based) exists, but see phab:T49528 (mobile might be harder) *MusicXML or MNX export (NoteEdit, Rosegarden, Frescobaldi, all GPL, have code for this)
MNX *should be perfect in every way :) *does not yet exist *starting from scratch (can be an advantage) *a full specification and an implementation
MEI/Verovio *browser-based, fast, wide and growing range of music cultures (including historic ones) encodable *does not do really sophisticated engraving (making pretty scores for print) *should be easiest, as open-source software exists, browser-based, SVG output (not sure if it does visual editing or just rapid previews) *non-academic users

See updated collaborative table above. HLHJ (talk) 02:19, 28 November 2018 (UTC)

In other options, Verovio is LGPLv3-licensed, and is a lightweight engraver that creates SVG scores in the browser. It uses the en:Music Encoding Initiative's format. It also has a large community, but they are music scholars, interested in representing music from a wide range of periods and cultures. It is currently more editing/encoding than display-oriented. Verovio's display is designed more to be fast than print-quality elegant, but is quite functional.
It seems to me as if the easiest path on LilyPond might be to develop an MusicXML export function for Lilypond. This might not be too hard, as apparently NoteEdit (GPL) did it, and Rosegarden and Frescobaldi do it; import MusicXML and export Lilypond.[12][13][14] Developing a format conversion (accepting that you'd lose a lot of Lilypond's finer display points) seems easier than an entire WYSIWYG editor, subject to correction by those who know more than me; it seems the LilyPond community is not opposed (they even have some bounties on it). LilyPond also has some people very interested in MEI integration (LilyPond dev on relation between MusicXML and MEI), though they don't seem to have been active on it since ~2015.
On LilyPond being a dead end, I rather think that is its niche. It does not aspire to be a visual editor, but a markup and display format. It makes really nice, highly customizable music engraving. It is the music equivalent of a typography program. I believe it's also been used for sound and follow-the-beat highlighting.[15] For import/export abilities, see Comparison of scorewriters (which is incomplete). Apparently MuseScore used to export to LilyPond,[16] but no longer does.[17]
A LilyPond file is not literally a dead end; it can be imported and converted to MusicXML via other open-source programs (and thence to MuseScore). We could offer MusicXML download on Commons, directly and via auto-conversion from LilyPond. This would integrate the MuseScore community, which I agree is an important priority. Regardless of our choice of display and editing format, I think MusicXML is an obvious format to be able to import from and export to on Commons (at least until MNX is ready). Are the "pizzicato/harmon mute MIDI controls, some styling and positioning" you mention, Jc86035's alternate account, not exportable to MusicXML in principle, or does MuseScore not currently support it?
I see that MuseScore was/is heavily involved in MusicXML/MNX development;[18] more MEI-focussed groups, including LilyPond, are not. Perhaps someone might also contact the LilyPond devs and MEI devs ask them to participate in this discussion, so we can get a broad range of viewpoints? HLHJ (talk) 05:20, 23 November 2018 (UTC)

@Mirabilos: are you sure about Debian? I simulated it on an up-to-date Debian system, LilyPond was happy with a circa two-year-old Guile, but whinged that a font configuration library was too old. It has been four years since the last stable release, but this is a mature project and development is active (and apparently perfectionist). LilyPond looks as if it will have 2.20 stable for the next Debian release next summer-ish (according to one of the Debian package uploaders[19]), so everything earlier is probably moot here anyway :). HLHJ (talk) 06:13, 23 November 2018 (UTC)

@HLHJ: According to Ebe123 (on Meta) and Marc Sabatella (on the MuseScore forum), the instrument channel issue is fixable but would require development work in MuseScore (and I don't know how MusicXML works so I don't know if the import could retain it). I don't know if the styling issues are fixable but MuseScore 3's auto layout would act as a workaround for the export use case.
I agree that it would be appropriate to contact more people, given the nature of the RfC and that shoogle and mirabilos (both being contributors to the MuseScore source code) have already posted, although contacting the W3C community group via email and/or via their mailing list(s) would also help. Jc86035's alternate account (talk) 13:37, 24 November 2018 (UTC)
Also, given that Commons inherently doesn't accept non-free content, it probably wouldn't be possible for this website to cannibalize MuseScore anyway, even if the Commons software becomes better. Another website accepting non-free content with the same software would presumably be hindered by not initially having a massive pre-existing user base or any sort of publicity. (I've edited my comment above because I didn't realise that when I was writing it.) Jc86035's alternate account (talk) 13:47, 24 November 2018 (UTC)
Canvassing expert views[edit]
I've notified the MuseScore forum and the four mailing lists: W3C, MEI, lilypond-user, lilypond-devel. Jc86035 (talk) 10:28, 26 November 2018 (UTC)
No mailing list replies so far except one on mei-l, although one person has emailed me privately to say that they think MuseScore should be supported (presumably through score display/playback) over the other formats. I replied by asking them to post here because I myself have no power to make technical changes. Jc86035 (talk) 14:43, 26 November 2018 (UTC)
I have received another two private replies: one from the RNIB's music adviser, expressing support for .brf (Braille) scores; and the other from Mdgood, the creator of MusicXML, who has also commented below. (I have replied to both emails.) Jc86035 (talk) 11:00, 27 November 2018 (UTC)
I've also received an email about using ABC notation (.abc), which is already supported by the Score extension and would be presumably be as easily supported as LilyPond. I haven't added a survey section yet because the person who wrote the email said that they would post their own comments. Jc86035 (talk) 00:54, 30 November 2018 (UTC)

Member of the MEI Technical Team here. We would love to work with this community to get support for MEI into Wikimedia. I'd be happy to answer any questions about that. It's also worth noting that Verovio also supports MusicXML and **kern input[20] as well, so you get three formats for the price of one. :) Ahankins (talk) 10:49, 26 November 2018 (UTC)

Also (replying to self) it supports client-side MIDI playback, SVG, and PDF export. --Ahankins (talk) 11:38, 26 November 2018 (UTC)
@Ahankins: Hi, thanks for commenting. It would be nice if MEI would work with Commons/MediaWiki developers, although I'm not really a developer so I can't speak for them. I've corrected some of my earlier posts regarding MEI. (You might also want to add comments in the survey sections above.) Jc86035 (talk) 13:18, 26 November 2018 (UTC)
Also noting that I can't figure out how to send an email through to the LilyPond mailing lists. I have subscribed to both lilypond-devel and lilypond-user, but the server is not even sending back confirmation that I sent the emails (and I have confirmation emails turned on). Jc86035 (talk) 13:18, 26 November 2018 (UTC) For some reason all five of the emails to the two LilyPond lists were posted at the same time. Never mind. Jc86035 (talk) 14:37, 26 November 2018 (UTC)

I am a co-chair of the W3C Music Notation Community Group and editor of the MusicXML 3.1 Final Community Group Report. MusicXML is indeed maintained by the W3C Music Notation Community Group. The latest 3.1 update was released in December 2017. MNX is an exploratory project for a next-generation notation format that addresses use cases that have emerged since MusicXML's initial release. MNX does not make MusicXML either obsolete or unmaintained. Anyone may join the Community Group to participate in the development of the MusicXML, SMuFL, and MNX projects. Mdgood (talk) 22:21, 26 November 2018 (UTC)

@Mdgood: I will be editing the description to emphasize that MusicXML is not abandoned, although to me it seems that the statement "no longer maintained" would be correct (although somewhat misleading) if MusicXML 3.1 is to be the final version of MusicXML and MNX is to be a continuation of it. Jc86035 (talk) 11:08, 27 November 2018 (UTC)
MusicXML 3.1 is not intended to be the final version of MusicXML. There will be an update when the community decides that an update is needed. Mdgood (talk) 15:51, 27 November 2018 (UTC)

What is the objective of uploading music notation files to Commons? That isn't clear from this discussion. Is it to allow small excerpts of music notation to be included in Wikipedia articles as illustrations? Is the concept that those illustrations will have high-quality layout for their music notation, or merely adequate legible notation? Is the concept that the illustration could also playback the notated music? Or, is the concept that a full-length score for an entire piece be a free-standing document, which the reader sees as primarily a music score rather than as an illustration to something else? Most of the existing content will be full-length scores. Making an excerpt of a score to serve as an illustration will be an editing task with specialised tools, soft of like making a map or drawing a chart is. Many of the the discussion points here seem to waver back and forth between "illustration" and "full-length score". JimDeLaHunt (talk) 03:12, 27 November 2018 (UTC)

@JimDeLaHunt: In brief, Wikipedia articles currently use excerpts for most pieces longer than national anthems, whereas for Wikisource (and perhaps for Wikipedia) it would be beneficial to have the whole piece transcribed for both viewing and listening. In the currently available format (LilyPond) it would also be quite disruptive and against English Wikipedia guidelines to insert a full score into the middle of a page, whereas a thumbnail would be less so. There are some full scores (fewer than 2,000, most being from books of traditional songs) on Wikisource, of which the longest I've found is this transcription of Haydn's Surprise Symphony. Jc86035 (talk) 10:37, 27 November 2018 (UTC)

Adding a community summary table: I liked the table which @HLHJ: used to summarise their view of the pros and cons of adopting each format. It is a compelling and attention-grabbing format. However, I think the discussion has come up with a different consensus than what HLHJ's table expresses. I regard that table as HLHJ's own opinion, and I don't want to hijack it into a community-edited table. So, I inserted a table at the top of this section, explicitly to be community edited in an attempt to express a consensus from this discussion in concise, tabular form. Please edit and improve it. JimDeLaHunt (talk) 04:22, 27 November 2018 (UTC)

Thank you, JimDeLaHunt. You think better of my table than I; I wouldn't have called it anything so definite as my opinion, as I'm still trying to figure out all these formats. Thank you for the community version; I have collapsed my own, and will edit the collaborative one. HLHJ (talk) 02:19, 28 November 2018 (UTC)
I'm not sure where the point on 'SVG Animation' came from in the consensus table; there seems to be no mention of that in the text. It's worth noting that Verovio doesn't use SVG Animation (just SVG, which is supported by most, if not all, modern browsers). The MEI Community is also working on "MEI Basic" which is an 'official subsetting' of the MEI Spec, to address its complexity for the majority of use cases. --Ahankins (talk) 23:36, 27 November 2018 (UTC)

@HLHJ: Would it be appropriate to add .brf/Braille to the summary table? No one has brought it up yet, although I think it would be inappropriate not to consider it. Jc86035 (talk) 11:00, 27 November 2018 (UTC)

I'd very much favour this. I think that the RNIB's music adviser is referring to Braille music formats; will add, feel free to modify. HLHJ (talk) 02:19, 28 November 2018 (UTC)
I should have read that article more carefully. Modified from its last section:
So it seems we have a converter between MusicXML and Braille music already written, and a Visual (Tactile) Editor! I've written to inquire about the license of the latter. HLHJ (talk) 04:59, 28 November 2018 (UTC)
I am the developer of Braille Music Notator and Braille Music Viewer, and I'm flattered to hear that you might find them useful here. They are meant to be free and open, and in fact I have recently made both projects open source. I had always intended to license them under CC and realize now that I hadn't specified that on the site; if any of you can provide me with guidance regarding the license variation which works best for this type of usage on Wikimedia Commons, I will add appropriate license information to the project. (And thanks for the invitation to the discussion, @HLHJ:!) TobyRush (talk) 06:18, 28 November 2018 (UTC)
That's great! I hadn't spotted that GPL license, or the Github page, by some bizarre oversight. Thank you for the information. I don't think a Creative Commons license is needed for the software; CC is more designed for things like music score files. The GPLv3 is entirely suitable for software (the Mediawiki software is GPLv2 or later), so you don't need to add any licenses to make it compatible. This is looking as if it might be surprisingly easy to do. I'm delighted. HLHJ (talk) 01:49, 29 November 2018 (UTC)
No wonder you didn't spot the Github project; I created it within the last month or two and I don't think I've linked it to the original site yet anyway. Glad to hear the GPL license will work; please don't hesitate to let me know if there's anything I can do to make things work better with Wikimedia commons; I'm happy to do so. TobyRush (talk) 17:06, 29 November 2018 (UTC)

I think the MEI conversation is getting conflated with the use of Verovio, the renderer. While I wholeheartedly support the inclusion of MEI in Wikimedia commons, I can understand that its lower software support may be an issue. However, I think there is a very strong case for the inclusion of Verovio in Wikimedia as a score renderer, independent of whether MEI is accepted as a 'blessed' format:

  • It's open-source, liberally-licensed, actively developed, and constantly improving
  • It supports MEI, MusicXML, **kern, and Plaine and Easie. (The latter format is useful for rendering millions of incipits found in MARC files, since P&E is the only format approved for inclusion in MARC, and is used by the RISM project.[25])
  • It may not be up to the same engraving "beauty" standards as Lilypond or MuseScore, but it is by far the best client-side engraving library out there with broad open format support. This is a big win if you don't want to support complex server-side rendering setups.
  • It supports MIDI playback and client-side export to PDF
  • It supports Common Western (i.e., "Common Practice Period") notation, Mensural notation, and Neume notation (the latter two requiring MEI encodings, as there are no other supported formats for this type of music)
  • The SVG output supports CSS styling, and XPath queries into a data, meaning it can dynamically render portions of scores and highlight particular passages on-the-fly. This means that the same full encoding can be rendered in smaller snippets. (Use case: A full encoding of Wagner's "Ring Cycle" can be encoded, but individual Leitmotifs can be extracted from the encoding and described separately.

(Full disclosure: I am part of the Verovio project) --Ahankins (talk) 10:56, 28 November 2018 (UTC)

That's probably my fault, I put them in one table row and survey question. I'll try to clarify, and thank you for the alert. The extractability functionality sounds really useful for this application, as files no-one can modify are really not Commons's thing. Commons is used on a broad variety of hardware (increasingly, mobiles), and over connections which vary over the full range of speeds and costs. It also gets downloaded and mirrored, making file size important.
MARC standards are used in library catalogs, according to the Wikipedia article. The RISM music catalogue looks great, and it's CC-BY![26] I found Kernscores, which seems to use yet another format, the ASCII-based **kern you mentioned and its relatives.[27]
I think I should explicitly say that this is not a decision about what formats will be approved in some way, nor will the most favoured format(s) be instantly implemented. Obviously including MNX is a nonstarter until it exists, but we are still discussing its possibilities. This being a sort of informal democracy, we are trying to decide where effort would be best spent; any and all decisions are subject to revision at any time.
Last but not least, welcome to all the new and returning editors here. Having such varied expertise contributing to this discussion is very useful. I also really appreciate the spontaneous conflict-of-interest statements, it's great to see such sympathy with community norms. HLHJ (talk) 01:49, 29 November 2018 (UTC)
Sorry, I should have noted that MARC is a library standard! Kernscores is part of the work happening at Stanford University, where they've been collecting encoded music since the 1990s in various formats. Craig Sapp (at Stanford) has been the lead for developing **kern support in Verovio, and he maintains Kernscores as well as works on projects like the Josquin research project, The Chopin Institute, the New (new) Complete Mozart edition, etc. He's done some pretty cool stuff with his Verovio Humdrum viewer (http://verovio.humdrum.org/ -- click 'play' and it's pretty cool), and automatic conversion between encoding formats (Kern <-> MEI <-> MusicXML <-> MIDI).
— Preceding unsigned comment added by Ahankins (talk • contribs) 10:29, 30 November 2018 (UTC)

A humble request to the editing community[edit]

Ic phone android 48px.svg Since the voting phase of the Community Wishlist Survey 2019 has started, I would like to invite all interested editors (particularly those who edit from mobile devices) to vote for the proposal to add a revert functionality to the mobile interface, so that a strong message may be sent out to the Wikimedia Foundation and its employees to take its mobile user-base seriously and eventually bring the mobile interface at par with other major websites of the internet. Regards.FR30799386 (talk) 05:32, 19 November 2018 (UTC)

A mock-up of how the revert functionality may look on the mobile interface

A mock-up of how the revert functionality may look on the mobile interface
— Preceding unsigned comment added by FR30799386 (talk • contribs) 05:32, 19 November 2018 (UTC)
@FR30799386: I've been pushing the Wikimedia Foundation to improve the mobile interface for quite some time to no avail, I hope that your proposal succeeds as the inability to undo edits using the mobile interface is something I run into seeing vandalism. Anyhow as I'm not allowed to vote I won't be able to help you, but you could alternatively open a Phabricator ticket. --Donald Trung 『徵國單』 (No Fake News 💬) (WikiProject Numismatics 💴) (Articles 📚) 11:31, 21 November 2018 (UTC)

Templates Anonymous-EU and PD-France[edit]

I have been moving Anonymous-EU licences to PD-France for old French postcards.(see Category:Postcards of Ardèche) The 'Anonymous-EU' is included as one of the conditions for PD in France. I would like to see the same text used in Anonymous-EU, for considering the template PD-UK-unknown for files of UK origin. Later on, one can consider scripts to change french postcards with the licence Anonymous-EU to PD-France.Smiley.toerist (talk) 00:10, 25 November 2018 (UTC)

Normalize file extensions for new uploads[edit]

Automatically normalize file extensions for new uploads (existing files may be dealt with by another proposal, but not this one). For example, ".JPG", ".Jpg", ".JPEG" and ".jpeg" all become ".jpg". For TIFF, ".TIF", ".tiff", ".TIFF" all become ".tif". (of all the TIFF files on Commons, 83.5% has the ".tif" extension) The guidelines from the Library of Congress also support this (See Commons:Village pump#Normalize file extensions for new uploads). To realize this, I will create a Phabricator task if this proposal is accepted.

See also the related proposal directly below this one. Please vote on both.

Advantages:

  • Avoids confusion, like accidentally attributing File:Dedekind.jpg when one meant File:Dedekind.jpeg (this actually happened)
  • Makes life slightly easier for those who make bots/scripts/use VFC for certain tasks.

A related proposal from enwiki can be found at w:Wikipedia:Village pump (proposals)/Archive 74#Several changes to file naming.

Some statistics on current use (stats from July, includes redirects, current percentages may vary slightly. data provided as is):

Total jpg .jpg .JPG .Jpg .jPG .JPg .jpG .JpG .jPg
42453481 35229847 (83.0%) 7222882 (17.0%) 649 50 26 20 4 3
Total jpeg .jpeg .JPEG .Jpeg .JPeG .jPeG .JpEg .jpEg .jpeG
585411 576892 (98.5%) 7605 (1.3%) 891 (0.2%) 8 8 5 1 1
Total png .png .PNG .Png .pnG .pNG .pNg .PNg .PnG
2528817 2397286 (94.8%) 131488 (5.2%) 37 2 1 1 1 1
Total svg .svg .SVG .Svg .SVg .SvG
1377496 1376872 (100.0%) 618 4 1 1
Total tif .tif .TIF .Tif
1030310 1026794 (99.7%) 3515 (0.3%) 1
Total tiff .tiff .TIFF
199233 199213 (100.0%) 20

Votes[edit]

Please vote on both this proposal and the one below. - Alexis Jazz ping plz 23:25, 4 December 2018 (UTC)

  • Symbol support vote.svg Support as proposer. - Alexis Jazz ping plz 23:25, 4 December 2018 (UTC)
  • Symbol support vote.svg Support Obvious and useful. Pi.1415926535 (talk) 23:29, 4 December 2018 (UTC)
  • Symbol oppose vote.svg Oppose JPEG to jpeg vice versa (including JPG/jpg) but Symbol support vote.svg Support JpG, jpG, Jpg ect. Bidgee (talk) 00:16, 5 December 2018 (UTC)
  • Symbol support vote.svg Support Great idea, hopefully can be implemented easily Abzeronow (talk) 03:51, 5 December 2018 (UTC)
  • Pictogram-voting-question.svg Question, would this prohibit files from being uploaded with "the wrong extension" or just automatically change them to lower case as part of the process? --Donald Trung 『徵國單』 (No Fake News 💬) (WikiProject Numismatics 💴) (Articles 📚) 08:17, 5 December 2018 (UTC)
  • Symbol support vote.svg Support Correcting all file extensions of new uploads to lowercase improves readability of the file names, in my opinion. This is why I support this proposal. Most likely, the only reason why Commons can not normalize all previously uploaded files to have file extensions with lower case, would be that the Mediawiki software is based on Linux, which operating system, unlike Windows, actually differentiates on a file path according to upper- or lower case. Linux does not care whether a newly uploaded file has a file extension in upper- or lower case. So normalizing during upload should not have any impact on the usability of the file. --oSeveno (talk) 11:15, 5 December 2018 (UTC)
  • Symbol support vote.svg Support, with the caveat that "tif" be considered "normal" based on the statistics above, instead of the current bizarre "tiff".   — Jeff G. please ping or talk to me 12:23, 5 December 2018 (UTC)
  • Symbol oppose vote.svg Oppose As no external standard has been agreed. The community should not be agreeing an off the cuff set of "best" extension types when external standards exist, and the Wikimedia underpinning database relies on file types being easily mapped to an accepted standard MIME type dictionary. For example the MIME types standard indicates the simplest mapping is to use .tiff and .jpeg, yet the majority of people would not be happy with these being forced. The proposal must include a full list of MIME types from the current database (not just "image") and give a firm rationale as to why we must discourage multiple mappings of extensions to a given MIME type. Harmonization by itself, is not a clear rationale from a developer's standpoint when the status quo is already the most widely accepted practice elsewhere on the internet, and is defined as accepted in external standards. -- (talk) 12:50, 5 December 2018 (UTC)
  • Symbol support vote.svg Support Good idea. --Yann (talk) 13:03, 5 December 2018 (UTC)
  • Symbol support vote.svg Support A file should be unique by name and not by extension: example.jpg, example.jepg, example.Jpg, example.JPg and so on. --Amada44  talk to me 15:09, 5 December 2018 (UTC)
  • Pictogram voting comment.svg Comment I support what User:Krd said at COM:VP. Yes to jpg and JPG. No to jpG, jPg, Jpg, jPG, JpG, JPg. In other words, yes to the first three columns of the table, no to the rest of columns. I sometimes feel that JPG is a sign that indicates the image has been captured by a camera, and has not been transferred from the internet. However, I cannot back up my feeling with any evidence. 4nn1l2 (talk) 20:57, 5 December 2018 (UTC)
  • Symbol support vote.svg Support. Good idea --AntonierCH (d) 19:58, 6 December 2018 (UTC)
  • Symbol support vote.svg Support and while there may be no official standards, the statistics clearly speak for themselves as to what are the preferred extensions. --HyperGaruda (talk) 05:35, 7 December 2018 (UTC)
  • Symbol support vote.svg Support for mixed case extensions, but Symbol oppose vote.svg Oppose for extensions in only upper or lower case: It does not help for the millions existing files. Also, it seems to me in Unix and Linux the upper case is the default. Users are at least in German Wikipedia still uploading files with upper case extensions, so on moving to Commons there are frequently new files created here with what you want to avoid. So, your basic aim is actually not at all reachable, you can only teach users that they know about this fact. (I dislike the upper case file extensions myself.) — Speravir – 22:12, 7 December 2018 (UTC)
  • Symbol support vote.svg Support The file extensions are not important, just standardize them across the board and go with that if it helps simplify things. Thanks. Mike Peel (talk) 23:42, 7 December 2018 (UTC)
  • Symbol support vote.svg Support. Vulphere 04:10, 10 December 2018 (UTC)

Discussion[edit]

"would this prohibit files from being uploaded with "the wrong extension" or just automatically change them to lower case as part of the process?"

@Donald Trung: They should be automatically changed. - Alexis Jazz ping plz 20:25, 5 December 2018 (UTC)

"I sometimes feel that JPG is a sign that indicates the image has been captured by a camera, and has not been transferred from the internet. However, I cannot back up my feeling with any evidence."

@4nn1l2: If this proposal is accepted, I will ask in the Phabricator request if it's possible to add a note containing the original extension to the upload/file comment. Like "User created page with UploadWizard, file extension automatically changed from jPg to jpg". I don't know if that will be technically complicated/possible, but I'll request it. - Alexis Jazz ping plz 23:43, 5 December 2018 (UTC)

"Support for mixed case extensions, but oppose for extensions in only upper or lower case: It does not help for the millions existing files. Also, it seems to me in Unix and Linux the upper case is the default. Users are at least in German Wikipedia still uploading files with upper case extensions, so on moving to Commons there are frequently new files created here with what you want to avoid. So, your basic aim is actually not at all reachable, you can only teach users that they know about this fact. (I dislike the upper case file extensions myself.)"

@Speravir: A few things:
  • Uppercase is not the default for Linux. In fact, Linux doesn't give a rat's ass about file extensions. (some desktop environments and applications do though)
  • This proposal doesn't deal with existing files, this is true. This is a first step: stop new files with odd extensions from being created. If accepted, we can look into the best solution for existing files. There are several possible approaches for that.
  • There are 7222882 .JPG files (which indeed qualifies as "millions"), but all the others combined (those from the table, there are a few more like pdf and webm) total 921067. 576892 (more than half) of those are .jpeg.
  • When moving a file from dewiki to Commons, the filename should also be automatically normalized. MediaWiki afaik doesn't differentiate between new uploads and files moved from other projects. - Alexis Jazz ping plz 00:09, 8 December 2018 (UTC)
Dewiki to Commons: If so, then this is quite new. I did not observe this myself until now. This did not happen with NowCommons and on the page for the beta-feature FileImporter this is not mentioned, either. — Speravir – 00:19, 8 December 2018 (UTC)
@Speravir: I meant: if this proposal is accepted, the filename for files imported from other projects should also be automatically normalized. - Alexis Jazz ping plz 00:39, 8 December 2018 (UTC)

When moving files, move them to the normalized file extension[edit]

Related to the proposal above: when a file with a non-standard file extension is renamed (.JPEG, .pNG, etc) it should be renamed to the normalized extension. The only exceptions for this will be if the file is renamed for reason 4 (harmonize the names of a set of images) or if normalizing would de-harmonize the names of a set. In such cases, file movers should judge on a case-by-case basis if it's better to rename and normalize the whole set or keep the non-standard extension.

File movers won't have to normalize file extensions themselves, the rename gadget would be adjusted for this. - Alexis Jazz ping plz 23:25, 4 December 2018 (UTC)

  • Note: if you are concerned about the (unlikely) theoretical possibility of this proposal being accepted without the first proposal being accepted, you can support it on the condition the first proposal (normalize extensions for new uploads) is also accepted. That's how it was meant to be anyway. I'm sorry, I should have thought of this. - Alexis Jazz ping plz 05:33, 5 December 2018 (UTC)

Votes[edit]

  • Symbol support vote.svg Support as proposer. - Alexis Jazz ping plz 23:25, 4 December 2018 (UTC)
  • Symbol support vote.svg Support Seems useful. How would the opt-out functionality be available - something like a pre-checked tickbox that the file mvoer can uncheck if needed? Pi.1415926535 (talk) 23:32, 4 December 2018 (UTC)
  • Symbol oppose vote.svg Oppose Lets wait until the above section is sorted before we move on to this. Bidgee (talk) 00:23, 5 December 2018 (UTC)
  • Symbol neutral vote.svg Neutral leaning support. Would like to wait to see how above section works out like Bidgee Abzeronow (talk) 03:52, 5 December 2018 (UTC)
  • Symbol support vote.svg Support The implementation of this proposal should not have any negative impact on the usability of the file. For the reason why I support it, please read my motivation at (the) proposal Normalize file extensions for new uploads. I am not worried that this proposal could create a technical conflict, since if it would, the technicians of Phabricator would block it from implementation. What I would not recommend though, is that we leave the decision to file movers, to judge the casing of the extension on a case-by-case basis, since a) That option would create a need for a more extensive GUI to execute the choice. b) It leaves room for interpretation based on personal preference. This proposal should be about uniformity, in my opinion. --oSeveno (talk) 11:32, 5 December 2018 (UTC)
  • Symbol support vote.svg Support, with the caveat that "tif" be considered "normal" based on the statistics above, instead of the current bizarre "tiff".   — Jeff G. please ping or talk to me 12:21, 5 December 2018 (UTC)
  • Symbol oppose vote.svg Oppose As no external standard has been agreed. The community should not be agreeing an off the cuff set of "best" extension types when external standards exist, and the Wikimedia underpinning database relies on file types being easily mapped to an accepted standard MIME type dictionary. For example the MIME types standard indicates the simplest mapping is to use .tiff and .jpeg, yet the majority of people would not be happy with these being forced. The proposal must include a full list of MIME types from the current database (not just "image") and give a firm rationale as to why we must discourage multiple mappings of extensions to a given MIME type. Harmonization by itself, is not a clear rationale from a developer's standpoint when the status quo is already the most widely accepted practice elsewhere on the internet, and is defined as accepted in external standards. -- (talk) 12:50, 5 December 2018 (UTC)
  • Symbol support vote.svg Support Please note that there is already some normalisation (JPG to jpg), but more is better. --Yann (talk) 13:19, 5 December 2018 (UTC)
  • Symbol support vote.svg Support --Amada44  talk to me 15:10, 5 December 2018 (UTC)
  • Symbol support vote.svg Support Ronhjones  (Talk) 19:56, 5 December 2018 (UTC)
  • Symbol support vote.svg Support if the above proposal is accepted. Otherwise, Symbol oppose vote.svg Oppose --AntonierCH (d) 19:59, 6 December 2018 (UTC)
  • Conditional Symbol support vote.svg Support, provided the above proposal is accepted. --HyperGaruda (talk) 05:37, 7 December 2018 (UTC)
  • Symbol support vote.svg Support The file extensions are not important, just standardize them across the board and go with that if it helps simplify things. Thanks. Mike Peel (talk) 23:42, 7 December 2018 (UTC)
  • Symbol support vote.svg Support. Vulphere 04:11, 10 December 2018 (UTC)

Discussion[edit]

"How would the opt-out functionality be available - something like a pre-checked tickbox that the file mover can uncheck if needed?" @Pi.1415926535: That's a possibility. Perhaps the person requesting the move could also be warned (but not blocked) if they try to request a move to a non-standard file extension. The exact technical implementation should be looked at together with file movers. - Alexis Jazz ping plz 00:18, 5 December 2018 (UTC)

I created this as a separate proposal as to not overcomplicate the first proposal. The first proposal (for new uploads) doesn't depend on this one, but this one (normalize when renaming) kind of does depend on the first proposal. After all, it wouldn't make much sense without it. I don't believe there would be anyone who would oppose the first proposal while supporting this one. - Alexis Jazz ping plz 05:33, 5 December 2018 (UTC)

Pictogram voting info.svg Info This proposal can be closed, because the normalisation on file moving is already in effect. BTW this creates tiff from tif/TIF. — Speravir – 22:15, 7 December 2018 (UTC)

@Speravir: you appear to be right. But as you can see on File:Hugh Riminton 2011 (cropped).jpeg, the yellow {{Rename}} box still shows the current extension, even though it will be renamed to jpg. Bug probably. - Alexis Jazz ping plz 23:31, 7 December 2018 (UTC)
@Alexis Jazz: Template {{Rename}} uses Module:FilePage. This is not the file renaming script. Nonetheless, this looks to me like a bug, too, given that you explicitely ask for .jpg. — Speravir – 23:48, 7 December 2018 (UTC)
So, I just moved the file according to your request, but it has been renamed to File:Hugh Riminton in 2011 (cropped).jpg. The renaming form suggested the new name, and even although I manually changed to .jpeg this has not been granted (as expected by me). — Speravir – 23:54, 7 December 2018 (UTC)

Proposal for Bot[edit]

I been playing with my bot (user page writes only! - see last five files listed here) looking at new images that source from Facebook, Instagram, Google, Twitter, Flickr, and have not been reviewed. It nicely finds plenty of images every day which need "no permission". It then seems logical that this could be extended to any new image with an external source and no unreviewed banner - and then rather than just make a list to a user page - add a category to the image to show it's in need of checking. Anyone think this might be good? Ronhjones  (Talk) 20:12, 5 December 2018 (UTC)

Pictogram-voting-question.svg Question So the bot would only categorize? Not tag for deletion? - Alexis Jazz ping plz 21:41, 5 December 2018 (UTC)
Tagging for deletion would need better AI than I could provide. I planned categorised only - something like "Files with external source not yet reviewed" Ronhjones  (Talk) 17:50, 6 December 2018 (UTC)
I would lean towards supporting that, but I'm afraid of the slippery slope. - Alexis Jazz ping plz 19:38, 6 December 2018 (UTC)

Give autopatrolled users more upload options[edit]

Note: this proposal depends on phab:T89131 (implement server side Flickr reviewing). If accepted, it will not be implemented before {{FlickrVerifiedByUploadWizard}} has been replaced/complimented by {{flickrreview}} and/or MediaWiki is given the ability to perform server side reviewing. They are, in a way, waiting for us. If we accept this, T89131 won't take long.

The actual proposal is below the intro, you can skip the intro if you already know what this is about.

Intro/background[edit]

Extended uploaders is a user group for users who are trusted with the following:

  • Uploading files from a URL (limited to this whitelist)
  • Uploading MP3 files (regular users need to convert their MP3 files to Vorbis, Opus or FLAC before uploading, a process in which quality is lost for Vorbis and Opus)
  • Uploading files from Flickr using UploadWizard (nothing you couldn't do without Extended Uploader, it's just more convenient.. not to mention Flickr2Commons)

There are currently 67 extended uploaders. From those, only 3 (three) have ever uploaded more than 6 (six) files from Flickr using Upload Wizard: Koavf (39 files), GreenMeansGo (45 files) and Ixocactus (79 files). You will understand in a moment why that's important.

But first, let's think about autopatrolled users. Autopatrolled users (we have 5953 of them right now) would be deemed smart enough not to upload albums from Madonna and Robbie Williams. Else they wouldn't be autopatrolled. So allowing autopatrolled users to upload MP3 files, I don't see why not. Uploading files from a URL, with the whitelist restriction which is already in place, has minimal potential for abuse. Even less so for autopatrolled users.

Leaving one thing: uploading files from Flickr using UploadWizard. There has been some controversy around Flickr2Commons with some users suggesting to put a ratelimit on Flickr2Commons. Which didn't happen. But for Flickr import using UploadWizard, I will include an option for that. This is why I had to tell you about the three users: if we decide to do this with a ratelimit, only three users will be somewhat affected. And even they probably won't notice if we don't pick some extremely low number.

On to the votes. If there are not enough supporters for this proposal without ratelimit, we will see if there is enough support with a ratelimit. - Alexis Jazz ping plz 10:31, 9 December 2018 (UTC)

The actual proposal[edit]

Allow all autopatrolled users to upload files in MP3 format, upload files from a whitelisted URL and upload files from Flickr using UploadWizard.

Support, with or without UploadWizard-Flickr-specific ratelimit[edit]

Support, with UploadWizard-Flickr-specific ratelimit, don't forget to include a maximum number of UploadWizard-Flickr uploads per minute![edit]

  • (none yet)

Leave everything as it is (oppose)[edit]

  • (none yet)

Discussion[edit]

Umm...Not particularly fond of the idea of extending the Flickr url upload until its fixed. Did we ever figure out why you have to button mash the upload button and get 9000 license review errors when you do? It was still broken as of 6 December. GMGtalk 12:12, 9 December 2018 (UTC)

Or maybe opening it up to lots of people will give an incentive to actually fix it? GMGtalk 12:14, 9 December 2018 (UTC)
@GreenMeansGo: the abuse filter has been fixed for extended uploaders by Kaldari on 6 December. Replacing/complimenting {{FlickrVerifiedByUploadWizard}} with {{flickrreview}} is not technically complicated, and if this proposal is accepted there will be a clear incentive to patch it. - Alexis Jazz ping plz 16:49, 9 December 2018 (UTC)
Okay. Well, as long as we're not getting a zillion files that need fixing, or a zillion comments from people who haven't figured out yet that if you just button mash you can bypass the initial error you get. (I now realize why no one on wiki or on IRC knew the answer to the problem, because only three of us have ever actually used this feature apparently.)
I'm still a little wary about MP3s though. From the perspective of monitoring latest files...yeah...if someone uploads Aaron Copland...obviously that a copyright violation. If someone uploads whatever is on Tamil or Mandarin radio, there ain't no google image search for that. GMGtalk 17:28, 9 December 2018 (UTC)
When UploadWizard adds {{flickrreview}}, nobody will get any error. And there are actually more people who use the feature, but they are license reviewers and administrators. So they never got any error. As for MP3s: it's still only autopatrolled users. And any user (even without autopatrol) can upload .ogg, .opus, .wav and .flac. We just use the description, license, source and have people who understand the language look at it. If somehow huge problems arise (which I don't believe) that can't be solved by taking the autopatrol status for problem users away, another proposal to take MP3 away again will be here soon enough, I'm sure. - Alexis Jazz ping plz 18:27, 9 December 2018 (UTC)

Not sure how you got your stats. I am convinced I have used the upload wizard to upload from Flickr albums. -- (talk) 13:34, 11 December 2018 (UTC)

@: so have several administrators. You are not listed as an extended uploader, therefore you are not in the stats. - Alexis Jazz ping plz 23:41, 11 December 2018 (UTC)
Oh, I vaguely remember the discussion where this was created, but I no longer remember the logic of why I am not an extended uploader and presumed I was one. Can't keep in all in my head. :-) -- (talk) 00:01, 12 December 2018 (UTC)

So just to be clear on what is being proposed here. The following flag is going to be added to the autopatrolled user group, upload_by_url as well as modifying the abuse filter to include autopatrolled. Just so you know, flickr is already included on the upload_by_url whitelist so granting that flag should allow that ability via the upload wizard. If this passes, what then would be the point of the extended uploader group? Really the only specific flag that is granted by that group is the upload_by_url and the abuse filter exemption. Both of which are being proposed here to be added to the autopatrolled group. Therefore, this proposal would make extended uploaders obsolete. Correct? --Majora (talk) 22:39, 13 December 2018 (UTC)

@Majora: absolutely 100% correct. The original title (which I changed before publishing) was something like "Make all autopatrolled users extended uploaders". But stating it so bluntly directly in the title without any nuance didn't seem quite right. And there is the option of supporting this proposal with a ratelimit, that could technically be a difference for extended uploaders who wouldn't have that limit. Although only three users (Koavf, GMG and Ixocactus) would realistically be somewhat affected.. and two of them have already supported the proposal.
Another thing (well, related): does what you say mean that anyone who has upload_by_url has the "Share images from Flickr" button? Who else (that isn't a license reviewer) has upload_by_url? Gwtoolset I guess? - Alexis Jazz ping plz 01:57, 14 December 2018 (UTC)
@Alexis Jazz: It is my understanding from a cursory look at the extension that yes, anyone with the upload_by_url flag should have the "share images from Flickr" button. Per Special:ListGroupRights, image reviewers, bots, extended uploaders, GWToolset users, and administrators have that flag. If this proposal comes to pass I really don't see the need to have two user groups that have the same basic function so retiring extended uploaders would probably be for the best. --Majora (talk) 19:32, 14 December 2018 (UTC)

Better handling of file copyright releases with source linkrot[edit]

After this deletion request raised by a Commons Bureaucrat, I have been nudged to reconsider that our norms need better formalization into guideline, so that administrators and others interested in deletion requests can act consistently when sources are suffering from linkrot, so there is a practical presumption of good faith, for files where there is no prior pattern of related challenge or demonstrated copyright violations.

It is a statement of fact that every website on the internet has a limited life, and the scope of this project should include the preservation of useful educational content that might otherwise vanish or become much harder to access. It has to be accepted that independent license reviews are not expected or enforced for all Commons uploads.

Proposal: For an in scope file where the uploading account has a reasonable history of good uploads, but the source is no longer available, or the release at source has changed to a non-free license:

  • A file which is part of an upload project of more than 1,000 files and where there is consistency of licensing, will be presumed to have been uploaded with a valid release.
  • A file which has been automatically license checked, and this can be verified in the upload code, or confirmed via a project page explanation, will be presumed to have been uploaded with a valid release.
  • Uploaded files that have been hosted for more than 2 years, where the uploader account has made significant contributions and there is no related pattern of deliberate copyright violation from the source, or by the uploader account, will be presumed to have been uploaded with a valid release. This will apply regardless of the uploading account having since retired.

These guidelines can be added to Commons:License review, unless there is a better place for them to be added.

Please vote on the principle of having this guideline. The numbers (1,000 files, 2 years) may be adapted depending on comments or evidence raised, or even later copyvio related case evidence. -- (talk) 12:54, 11 December 2018 (UTC)

  • Symbol support vote.svg Support as proposer. -- (talk) 12:54, 11 December 2018 (UTC)
  • Could we not simply have a discussion rather than a vote? Currently the issue and criteria seems to be narrow to the point of involving one user: Fae. Are there other cases? It seems to be saying simply that your uploads should all be trusted and never speedy deleted and default keep at DR unless demonstrated otherwise. I'm sure you do take care but are only human and the source organisations/people are also only human too. We have had cases where apparently trusted organisations uploaded and tagged-as-free images they shouldn't. I recall one notable example where NASA on Flickr uploaded an Annie Leibovitz photo to their PD stream, which was uploaded by an "account [that] has a reasonable history of good uploads". Wouldn't it be better to have ensured the images were reviewed four years ago? Do we have a means to bulk-review a batch of images from one source? If we don't have a means to review images as quickly as some users upload them, shouldn't we fix that, rather than just create exceptions at DR? This seems to be a tail-wagging-the-dog solution. -- Colin (talk) 15:20, 11 December 2018 (UTC)
  • Pictogram voting comment.svg Comment I have suggested that these files should be license reviewed. Fae opposed that. So not surprised that this comes up here... Yann (talk) 15:31, 11 December 2018 (UTC)
    Please provide a link to that discussion. Thanks -- (talk) 16:09, 11 December 2018 (UTC)
  • Pictogram-voting-question.svg Question Why were these not marked for license review when they were uploaded? If you are that concerned about it they should have been checked by another person. This is an image uploaded from an external site by someone who was not the photographer. That pretty much screams either LR or OTRS. Requesting such a thing would have solved all the issues. To presume anything is to throw out COM:EVID and I'm not really all that comfortable with that. I've found copyvios that have been here for 10 years (using archive sites to verify that they were indeed a copyvio at the time of upload). Putting a year limit on it in the guise of COM:GOF is also not a good idea in my opinion. I'm not entirely familiar with "automatic license checks" as stated in your second bullet. So if you could explain how that works I'd appreciate it. --Majora (talk) 22:36, 11 December 2018 (UTC)
The example DR is such an automatically license checked batch upload, the metadata of the file was checked as having a specific free license ("ATTRIBUTION_SHARE_ALIKE") and if it did not match it was skipped. You can imagine this is the same sort of check that we get with the automatic flickr reviews. There are many example projects that do exactly this type of specific automatic license checking at User:Fæ/Project_list (most recent example). In these cases, investing programming time writing a unique bot script customized for each batch upload, to do exactly the same check, adds zero value, and, even worse, marking literally millions of files for manual checks would be a bizarre waste of volunteer effort.
If it were policy, I could backdate my uploads with {{LicenseReview}} where they do not have one. This would probably mean more than two million files needing manual review as there are no automated bots to handle these, further I would not be allowed to write a bot to do the reviews, even where I have code available, as this would not be independent. This number is large enough to pretty much guarantee the backlog would never be resolved, yet I am hardly the only uploader on this project, and if we add LicenseReview to all our previously uploaded and not-independently-reviewed files, the backlog would be a magnitude or two larger than that. -- (talk) 23:02, 11 December 2018 (UTC)
Hmm...I'd be interested to learn more about this automated checking process. If something is going to come from this your second bullet is the best way forward in my opinion. Where in the metadata is this information stored? I did a more in depth search of the metadata using a third-party viewer and I'm not seeing it. I'm guessing by "file metadata" you really mean page source? That appears to be what User:Fæ/Project list/OpenBenches is working off of as well but please verify. Also, what does the checking? Is it part of the GWToolset? Is it coding that you do yourself when you create one of your projects? Obviously when we are talking about millions of images there has to be some give and take and I'm ok with that. But I just want to make sure all avenues are being explored here and that I fully understand what is going on during these uploads to make an informed comment. --Majora (talk) 23:20, 11 December 2018 (UTC)
The OpenBenches project has metadata available as JSON queries, see the project page. The way the site works is completely unique to that project and the uploads are via a customized Pywikibot script. Were someone to try to use GWT, it would mean extracting the source data from Github (where it happens to be published) and converting it to XML, which would be more complex (in my view) than writing a custom script. The license verification looks at the JSON data for the photograph and does exactly the license string check as explained on the Commons project page. The OpenBenches programmer made the image license data available this way, specifically because I discussed the Commons upload project with him. BTW, in terms of my projects, this is a rather small one, compare with User:Fæ/Project_list/PAS which is "similar" because the metadata is in JSON format, but is fifty times larger. However in the same way as OpenBenches, there is no Commons bot that can be easily tailored to include it to automate independent license reviews. -- (talk) 23:29, 11 December 2018 (UTC)
(Edit conflict) Let me first just apologize for all these questions. Obviously what you are doing is above my level of knowledge so I appreciate you bearing with me while I wrap my head around this. If I'm understanding the above correctly, you write unique queries that pull information off of sites based on how that information is stored (JSON for OpenBenches, etc.). If this is true would it be possible to publish this query so that it can be part of the public record on these images? I see part of the code you used on the OpenBenches information page but not the whole thing. If the entire thing is available it allows others to check your work that way. I feel like that would be a massive step in the right direction towards some sort of verification process. --Majora (talk) 23:41, 11 December 2018 (UTC)
I am prepared to publish the license checking part of my code, but that pretty much is exactly what is on the example project page, it's a cut & paste of a couple of lines from 800 lines of code, and I could continue to do that with future projects, but not retrospective projects especially as in some cases I have probably lost the original code. I do not publish all of my code, that is not a criteria for contributing to this project, nor would it be genuinely useful (in 8 years, there are probably 2 people than have used my code already published on Github for anything, and they did not simply reuse my code; the Github link is even published at the top of my user talk page). However, keep in mind you are only addressing me, there are many other uploaders with customized batch upload projects, and there are no criteria on them for how to do it. -- (talk) 23:50, 11 December 2018 (UTC)
Obviously this is an exploratory proposal that would not just apply to you. Unless the suggestion above made by Colin that this is really a narrow proposal for one was fact. In any case, I really only care about the way licenses are checked. The rest of the code is rather useless in terms of this discussion. As for github, digging through there should not be a requirement. If you could publish the license checking part of your code so that it can be verified that would be preferable in my opinion. This goes for others as well. The way I see this is, essentially, these batch uploads are close enough to bots without actually being bots. Most bots have some sort of code published that can be reviewed to make sure it is doing what it says it is doing. I could see, and probably support, a policy that allows for blanket acknowledgement of acceptable licensing of batch uploaded images provided source code for checking said licenses is published and retained in an easily seen format (such as a project subpage like you have listed above). I'd support this going forward and as for older images we can talk about some sort of grandfathering. --Majora (talk) 00:00, 12 December 2018 (UTC)
  • @Majora: it's not only Fæ. Category:Images from lasvegasvegas.com for example is also complicated. Most probably all the photos in that category are fine, but very few were tagged for license review. - Alexis Jazz ping plz 23:37, 11 December 2018 (UTC)
  • @: this kind of sounds like license-review-assumed?
  1. "A file which is part of an upload project of more than 1,000 files" is such a large number not many uploads will be able to apply for that. It also seems arbitrary. Why not 2,000? Or 500? And what does "where there is consistency of licensing" mean? If none of the files were reviewed and all the source links are dead, but there are more than 1,000 files we keep?
  2. "A file which has been automatically license checked, and this can be verified in the upload code, or confirmed via a project page explanation, will be presumed to have been uploaded with a valid release." should be better defined (like you're now explaining it a bit more above). I would suggest splitting this proposal and make it more clear what each bullet point stands for. - Alexis Jazz ping plz 23:37, 11 December 2018 (UTC)
The vote is for the principle, the criteria can be hammered out. With regard to "1,000", the point is really "the uploading account has a reasonable history of good uploads", so yes if the source vanished for all 1,000 uploads, the debate would still be why do we presume bad faith for the demonstrably good uploader. Deletion in those cases should require some real evidence of a copyright problem, rather than a theoretical concern because someone realized that the source website has been knocked offline, or as has already happened with the very large PAS project, the links no longer function as the database has had a redesign. -- (talk) 23:41, 11 December 2018 (UTC)
  • I'm not sure if we can have a clear-cut guideline on this. It would involve different factors - how mature the semi-automation system/procedure was at that time of the upload in question (earlier uploads may be more prone to mistakes and bugs), how standardized the markup across pages at the source web site were, etc, beyond the general trustworthiness of the uploader. It seems taht the discussion here focuses on sources that already suffer from linkrot, but going forward, I'd encourage mass uploaders to archive evidence before linkrot happens (ideally, before every upload). Services like web.archive.org and archive.is can be used to store evidence semi-automatically. whym (talk) 14:10, 14 December 2018 (UTC)
I believe this is unrealistic, unless the IABot starts including Commons. If you can show/demonstrate how I can add, say, all of finds.org.uk images to IA, currently 440,000 images on Commons, and return those archive links so they can be added on Commons image pages, I would be grateful. Thanks -- (talk) 14:24, 14 December 2018 (UTC)
Here is a quick and dirty code to call web.archive.org in Ruby:
require 'open-uri'
require 'uri'

def run_web_archive_org(uri)
  open("https://web.archive.org/save/#{URI.escape(uri)}") do |f|
    if f.meta['content-location'] then
      puts "<https://web.archive.org#{f.meta['content-location']}>" # location of the archived page which can be fed to an file-page editing script
    else
      puts "Error? #{f.meta.inspect}"
    end
  end
end
Does this help? (You will somehow have to throttle the HTTP requests if you have thousands of pages to be archived, though.) --whym (talk) 14:50, 14 December 2018 (UTC)
I could do something similar in Python, but extending IABot makes a lot more sense as I can piggyback on whatever error traps exist there, rather than reinventing the wheel, plus it can act retrospectively on my past millions of uploaded files and that housekeeping can be run by anyone not just me. I'll think about it, it's very close to Christmas to do anything right now. Anyway this proposal was generic, not just me solving how to do my own Library of Congress or Finds uploads a bit better. -- (talk) 14:56, 14 December 2018 (UTC)
There are quite a lot of images uploaded every day with no request for a license review. I've been running User:RonBot Task 1 every day, and then adding "no permission" tags to lots of images. Which is why I posted the suggestion above for a new bot Commons:Village_pump/Proposals#Proposal_for_Bot, to slowly expand on that Task, and putting the results in a sensible category for license review to be done. I think the idea of allowing a time limit on review will result in copyright images being accepted - I too have found bad images way more than two years old. Ronhjones  (Talk) 00:40, 17 December 2018 (UTC)
  • I would generally Symbol support vote.svg Support this proposal, but to me this could leave room for mistakes to happen and although this policy will only apply to otherwise highly trusted users (users such as Fæ), however there should be a better system where external links are automatically archived, a couple of months ago I suggested that the InternetArchiveBot or a similar bot would run on Wikimedia Commons as it preserves web pages on Wikipedia's however the discussion lead to nowhere and while the operator of the InternetArchiveBot showed that they weren’t interested in this, another user was interested but eventually no action was taken. Originally I was also planning at proposing the archiving of web links at the 2018 Community Wishlist but as I wasn't unblocked on the Meta-Wiki that clearly also didn't happen, as the Community Tech Bot actually runs efficiently for these kinds of tasks it's a shame. As Wikimedia Commons doesn't really have any direct support from the Wikimedia Foundation it's clear that the ageing software on this platform rarely gets updated and very few new features (if ever) get implemented, Fæ’s proposal isn't perfect but in the current environment it's the best we have, unless “the Technical community” would actually start to provide tools for the maintenance of files on Wikimedia Commons rather then just let everyone take both Wikimedia Commons and its files for granted then nothing will change, in fact the only time the Wikimedia Foundation seems to openly operate here is to delete things, not create them. I agree with the principle that these files should be preserved indefinitely and that we should actively try to discourage the deletion of files solely on the precautionary principle if the license from the historical website can’t be found. To give a recent example “[[]]” is a deletion request where a user (presumably) uploaded a free file from a website five (5) years ago but today no evidence of this license exists, yes this could've also been simply solved if the file’s license had been reviewed when it was uploaded, but if an archive of the website existed today of how it looked half a decade ago any license reviewer would've been able to debunk/confirm the license today. It would probably be best to try and solve the underlying (technical) issues rather then try and play these policy gymnastics, it seems to be a common trend among Wikimedians to build policy based on the limitations of the system rather than try to improve the system itself, let’s say the cycle works like this “The Wikimedia Foundation implements a half-baked feature (there may or may not have been a demand from “the community” for this), a better more complete feature is “promised for the future” but seemingly never comes to fruition, the users of the website start using this half-baked feature and starts creating (mostly unwritten) rules around this feature, these rules seem to be inconsistently applied by only the admins who believe in them and remain largely a secret for the out-group unless they would seek out the pattern, votes on solidifying these rules rarely seem to work and the preference for these vague unwritten rules seems to prevail” (this is probably because it gives admins more leeway, but this will inevitably confuse new users and makes the system looks unfriendly, there’s always an unwritten rule somewhere waiting for you to bite you in your hindquarters). It would also be wise if the state of these unwritten rules would be brought for scrutiny by the community rather then be dependent on the whim of the admin. Free within scope files get deleted everyday because the uploader didn't know how to do something properly, I would also suggest adding more manual links to the Internet Archive however the Internet Archive website won't load on my device anymore so this could too negatively impact users if such a rule was implemented. The best solution would still be to automatically archive links, and this is something that should be done ASAP but there doesn't seem to be any support for actually fighting the linkrot that’s so rampant on this project. --Donald Trung 『徵國單』 (No Fake News 💬) (WikiProject Numismatics 💴) (Articles 📚) 13:26, 18 December 2018 (UTC)

Any use for a "rare character" index?[edit]

Hello! There was recently a discussion at Extension:CirrusSearch about creating a new search index for "rare" characters that are currently not indexed by the on-wiki search engine. The three examples of difficult-to-find characters given were (Ankh), (ditto mark), and (ideographic closing mark). (Note that you can currently do an insource regex search like insource:/☥/, but on large wikis this is guaranteed to time out and not give complete results, and it is extremely inefficient on the search cluster.)

We can't index everything—indexing all every instance of e or . would be very expensive and less useful than , for example. So, in English, we would ignore A-Z, a-z, 0-9, space, and most regular punctuation (exact list TBD) and index pretty much everything else.

The most plausibly efficient way to implement such an index would only track individual characters at the document level, so you could search for documents containing both and , but you could not specify a phrase like "☥ 〆" or "〆 ☥", or a single "word" like ☥☥ or 〆☥.

I've opened a Phabricator ticket T211824 to more carefully investigate such a rare character index, to get a sense of how big it would be and what resources it would take to support it. If you have any ideas about specific use cases and how this would or would not help with them, or any other thoughts, please reply here or on the Phab ticket. (Increased interest increases the likelihood of this moving forward, albeit slowly, over the next year.)

Thank you! TJones (WMF) (talk) 16:29, 14 December 2018 (UTC)

  • I would Symbol support vote.svg Support this, although I am genuinely curious how and why these characters are being used, 〆☥ seem to be supported by my device while Tangut characters aren't so maybe they're not as rare as they receive mainstream support. --Donald Trung 『徵國單』 (No Fake News 💬) (WikiProject Numismatics 💴) (Articles 📚) 13:34, 18 December 2018 (UTC)
    • "Rare" is definitely a relative term. We haven't nailed down the specifics—that's part of what would need to be investigated—but it might well be the case that on English-language wikis, Cyrillic and Greek characters might count as rare, or ñ or ß might be rare, though they are all well-supported by most devices, and are not at all rare in Russian, Greek, Spanish, or German, respectively.
      The specific examples of ☥, 〃, and 〆 came up in the original discussion as characters that are ignored by the current language-processing, so they aren't being used in any particular way. Also, whether or not your device supports particular characters, you can usually still use them, you just can't see them. For example English Wiktionary has an entry on Tangut 𗌜 (which I can't see as I copy it here), and you can find it by searching it on English Wiktionary. However, it is discarded by the normal language analysis on Wiktionary, so the only result is the one exact title match. That's where the rare character index would come in. Some sort of search like char:𗌜 would find all documents with that character in it.
      Another good use case would be limiting a complex insource: or intitle: regex search by limiting the regex scan to only documents that contain a particular character (or possibly characters from a Unicode block), making the regex scan feasible.
      Commons is, unfortunately, an extra complicated case because the default language analysis for search here is English, but there is lots of text that is not in English, so how "rare" characters are best defined here might be harder to figure out. TJones (WMF) (talk) 17:26, 18 December 2018 (UTC)
      • @TJones (WMF):, that does sufficiently explain why such an index would be desirable. I do hope that it will be implemented as it won't hurt to have some extra features, even if it will have a rather limited (or niche) audience. --Donald Trung 『徵國單』 (No Fake News 💬) (WikiProject Numismatics 💴) (Articles 📚) 19:04, 18 December 2018 (UTC)
        • I hope so, too, because I'm part of the niche audience that would really appreciate it! It's a matter of scoping and prioritization, and there is always more work than can get done, but I'm hoping that I can get to the investigation of how big such an index would be next year, and if so, that it turns out to be not too costly and relatively straightforward to implement. My biggest concern is that determining what is "not rare" for every language will be both necessary and time-consuming. TJones (WMF) (talk) 19:18, 18 December 2018 (UTC)

Split up Freedom of Panorama country list[edit]

Commons:Freedom of panorama contains a list of FoP rules for specific countries, now transcluded from sections in recently-created articles giving the copyright rules for each country, such as COM:CRT/Algeria#Freedom of panorama. The page is large, with a post‐expand include size of 1,290,255, and may load slowly. During clean-up of the country-specific articles I have been adding FoP information to country articles where it was missing, but have held off on adding the countries to the Commons:Freedom of panorama countries list, which may get close to the breaking point.

This is to propose dropping the list from COM:FOP, leaving links to smaller lists for Africa, Americas, Asia, Europe and Oceania. The smaller lists will have entries for all countries, so will pick up new ==Freedom of panorama== sections in the country pages automatically. See Commons:Freedom of panorama/Africa for an example.

Inbound link issue[edit]

The main issue seems to be the many links, mainly from archived deletion discussions but also from files etc, to sections in the present FOP country list, often using shortcuts. E.g. 1,314 occurences of links to "COM:FOP#France". One way to resolve this would be to replace the country list with a multi-column list of anchors and new shortcuts with entries like

* {{Anchor|Australia}}[[COM:FOP Australia]]

which would display like

Shortcuts to country-specific rules

COM:FOP#Australia, would jump to Australia's entry in this list, and over time people would learn to use the new shortcut. It is not a particularly elegant solution, but the problem is creeping up on us, so will need some sort of solution sooner or later. Aymatth2 (talk) 19:46, 17 December 2018 (UTC)

Comments[edit]

  • Symbol support vote.svg Support, this is both easier to maintain and update and also easier to link to, idiosyncracies of the copyright laws of a certain territory could be much better contained within these specific articles rather than creating one huge page which is difficult to load. If only the software would support the option to allow people to watch edits made to transcluded sections without having to individually watch all of them, but I've been observing these proposed changes for a while now and can say that I'm a big fan of them. --Donald Trung 『徵國單』 (No Fake News 💬) (WikiProject Numismatics 💴) (Articles 📚) 23:30, 17 December 2018 (UTC)
  • Symbol support vote.svg Support, good idea.   — Jeff G. please ping or talk to me 07:02, 19 December 2018 (UTC)

Update padlock icons that Wikimedia Commons use to the ones that Wikipedia uses[edit]

The pad lock icons that Wikipedia uses are actually somewhat more descriptive on the image itself of what type of protection has been put into place. Even a user that does not know the meaning behind a particular protection name, the Wikipedia padlock icon gives away somewhat of a hint. The current old style(what Wikipedia used to use), only differentiates the padlock icons of different types of protection by color. Not only is that not much descriptive, it could also be deceiving to a color blind person. The updated Wikipedia padlock design has somewhat little bit taken care of this issue as well, a majority of the low level padlocks are black in color (thus causing no issues to color blind people).Aceing Winter Snows Harsh Cold (talk) 06:45, 19 December 2018 (UTC)

It's not clear at all to me what exactly what you are proposing here. Please give links and/or examples: Which set of files should be used instead of which other set of files for which purpose? Thanks, --El Grafo (talk) 09:25, 19 December 2018 (UTC)
The English Wikipedia changed to use the icons in Category:Page Protection Padlock Redesign (2018) to denote levels of page protection. GMGtalk 11:51, 19 December 2018 (UTC)