Module talk:Fallback

Good idea

Zolo, I like the idea of redesigning those templates and I an glad you have tacked it. --Jarekt (talk) 14:10, 20 November 2013 (UTC)

Thanks, that was not super-easy for me, but I think it finally works ! I have left a message at template talk:Fallback. --Zolo (talk) 16:50, 20 November 2013 (UTC)

Very nice, indeed. Just if you feel bored: Niklas made a nice graphic: File:MediaWiki fallback chains.svg. I would welcome if we could put the data into a separate module and load it with fallbacks = require('Module:Fallbacks') or similar so we can re-use this in Module:Languages, for example. -- Rillke^(q?) 18:26, 21 November 2013 (UTC)

Thanks, that seems reasonable. Actually, it seems that the current list is incomplete and I am wondering if there is a way to make use of some mediawiki built-in feature. It seems that there is a working fallback mechanism independent from getFallback for messages in the mediaWiki namespace or translate through Translatewiki (note that it seems incomplete too: [artwork in Corse] shows the message in English instead of French even though the fallback to French works for [Tahitian]). --Zolo (talk) 07:45, 22 November 2013 (UTC)

gerrit:96981 -- Rillke^(q?) 11:54, 22 November 2013 (UTC)

Thanks, but I am not sure Italian is the right choice. The language seems to be closer from Italian than French, just about everyone who can speak Corsu speaks French fluently, and I do not think most of them can speak Italian (we already have Tahitian -> French because Tahitian people can speak French, even though the languages are completely different). --Zolo (talk) 12:14, 22 November 2013 (UTC)

Interesting, after a quick glance, I had the feeling, the language is a lot closer to Italian than to French but this is maybe not the only criterion for a fallback, as you say, one has to consider what people are actually also speaking. Feel invited to comment over there! -- Rillke^(q?) 12:19, 22 November 2013 (UTC)

I have left a message at their main talk page, but it seems that there is at most one registered native-speaker active at the moment ([1], [2]).

Looking at File:MediaWiki fallback chains.svg we have may be half of the fallback connections mentioned there. Shall we add the rest? Also it looks to me that a lot of connections are between "xx-yyyyyy" languages and "xx" base languages may be such connection should be a default. I would also propose that Module:Fallbacklist would not be just a list but also a short bare bones function to create list of fallbacks for each input language. --Jarekt (talk) 19:19, 22 November 2013 (UTC)

Hmm. I like if data is completely separated from functionality. This allows you, for example creating bots regulary updating this list or such. -- Rillke^(q?) 19:53, 22 November 2013 (UTC)

OK, although I can not think of need for updates since languages should not be changing much, unless we are synching it with some other list, but than may be we can just use the original. How about synching with File:MediaWiki fallback chains.svg connections? --Jarekt (talk) 20:54, 22 November 2013 (UTC)

This seems to be the easiest way, unless someone wants to write a parser for all files in that directory. The $fallback variable defines MediaWiki's behaviour. Should we also file a bug/feature request for Lua/Scribunto for an API that we do not have to maintain? -- Rillke^(q?) 00:15, 23 November 2013 (UTC)

Ifexist

What's the issue with

mw.title.new('Module:Fallback').exists

except that it is expensive, but #ifexist is also… -- Rillke^(q?) 08:54, 25 November 2013 (UTC)

Defintely better. It's just that I had seen somewhere that "ifexist" was not supported in Lua and did not investigate further... --Zolo (talk) 09:35, 25 November 2013 (UTC)

Customized fallbacks

I have no knowledge of Javascript, but I am wondering if it would be possible to have a tool that would allow user to create their own fallback list instead of retrieving in from Module:Fallbacklist. --10:28, 25 November 2013 (UTC) That said, it would have to work for message from the mediawiki namespace as well otherwise, that may look a bit strange. --Zolo (talk) 10:31, 25 November 2013 (UTC)

JavaScript is processed 100% client side in your browser while LUA-processing is done 100% server-side. I cannot imagine how this should work, especially in regard of how MediaWiki with its parser-cache works: Each page is cached for every language but not per user. JavaScript would have to switch the language at the client side, if required, which in turn means either outputting all languages through the LUA-module or having to make an extra XHR with JS, resulting in a notable delay until the translation appears. -- Rillke^(q?) 10:36, 25 November 2013 (UTC)

Standards

See Commons talk:Lua --Jarekt (talk) 19:46, 25 November 2013 (UTC)

User:MIT Harvard Observatory/Treemap-index "no fallback page found for autotranslate"

User:1Veertje made a very useful color index for graphics in Category:Treemaps on Export, but it shows only "no fallback page found for autotranslate". I have no idea how to fix this. Can you help? --Atlasowa (talk) 11:29, 6 December 2013 (UTC)

Same for {{/Header}} at COM:UH. -- Rillke^(q?) 11:30, 6 December 2013 (UTC)

I wonder whether it was ever supposed to work with something not being in the template namespace. -- Rillke^(q?) 11:39, 6 December 2013 (UTC)

The former version of {{Autotranslate}} used "{{base}}"rather than {{Template:base}}. It means that, even though "Template:" was added when the base name did not contain any namespace, it also supported other namespaces. I do not think that was intended and I am not convinced allowing that would be a great idea, but if needed the Lua function can be adapted to make it work. --Zolo (talk) 12:52, 6 December 2013 (UTC)

I just hit the exact same problem by chance. I was trying to create a autotranslated template in my userspace only to find out that Module:Fallback assumes the pages to reside in template namespace. I think it should be checked if base contains a namespace. If yes, use it without change; if no, prefix base with "Template:". --Patrick87 (talk) 16:06, 7 December 2013 (UTC)

I think the simplest solution would be to check first if Template:base exists (as it is by far the most common case), and if not check it base exists. But if a page sues autotranslate, that is presumably because it is supposed to be seen by a variety of users, and then wouldn't it be better to move it to the Template namespace ? --Zolo (talk) 16:45, 7 December 2013 (UTC)

The use case I have in mind is developing testing and debugging templates in userspace before moving them to template namespace. I think your proposed solution would work, although checking if Template:base exists is probably more expensive than doing a quick reg-ex to check if the given base contains a valid namespace? --Patrick87 (talk) 16:55, 7 December 2013 (UTC)

The template has to check if the page exists for fallback anyway. My solution would indeed be more expensive, as it has to check two namespaces, but it would be a bit less expensive in the template namespace that is the foreseen usecase of this template. Actually I would think it is better to debug templates through sandbox templates rather than through user pages, so that other users can benefit from previous test pages and also becausee it is more reliable because some parts of the templates may turn out to be namespace-dependent. --Zolo (talk) 17:15, 7 December 2013 (UTC)

Will you implement this at some point? I just hit another scenario where it would be useful: Headers of project pages are often transcluded from "/header" subpages, e.g. Commons:Translators' noticeboard/header. When translating those using ext.translate you can't transclude those directly anymore but have to use an autotranslate-like system. I managed to use {{SuperFallback}} now for this purpose here, but I feel like a "one-for-all" template that can be used in all cases would be more appropriate. --Patrick87 (talk) 14:53, 12 December 2013 (UTC)

Ok done, I am wondering if it is a good idea to use autotranslate to transclude a page using the translate extension. I would imagine there should be more direct ways to translude it correctly. --Zolo (talk) 15:58, 12 December 2013 (UTC)

Thanks! What would you use/propose instead? I'm just getting used to ext.translate and it seems to me that this is the "default way" currently. --Patrick87 (talk) 16:07, 12 December 2013 (UTC)

Actually I cannot find any other solution. Do you know if that is intentional that transcluding pages using the translate extension show the translate tags even though they do not appear on the original page ? --Zolo (talk) 09:55, 15 December 2013 (UTC)

Yes, this is a current limitation of the Translate extension. It might change in future but currently one needs to transclude the subpages with the correct language code. --Patrick87 (talk) 15:08, 15 December 2013 (UTC)

But this is not that difficult using {{TNT}} on translatable pages and the awesome new {{Autotranslate}}. -- Rillke^(q?) 11:30, 20 December 2013 (UTC)

Actually the version of thr TNT template on Commons does not solve issues that I have fully solved on Meta. It can support all namespaces (also Meta uses a legacy check for the main namespace, if the base template is not found in the "Template:" namespace.

Soon I will had proper fallbacks on Meta too.

But the current list of fallbacks here is clearly insufficient (e.g. "sc" for Sardinian should fallback to Italian), and the current lack of support here for regional source variants (in user settings) means that fallbacks don't work correctly for them. It would be useful for example if users could set "co-it" as their user language (for the Southern Corsican dialect spoken in Sardinia) so that it will default to generic Corsican (co), then to Italian (it) instead of French, so the fallback list for "co-it" would be "co", "it"; then after that, the fallbacks of generic corsican will be used (so French will occur after Italian), and the fallbacks for Italian (none); then after that the "default" and "en" fallbacks.

As well, English can be a primary fallback with another secondary language to test before going to the "default" (such case is needed in India or South Africa). We should not assume that an English translation will always be present (the Translate tool should not assume that, we should be able to create a source in another language, such as Chinese, and then translate it to Cantonese and Gan (Chinese will be accurate, but English may be extremely fuzzy and should not be the reference, so Englihs should be considered as a derived translation created in the Translate tool and modifiable there). In summary, the translate tool should be able to track the language used in the base version. If later another translation becomes better than the source, we could swith the reference, and make this translation the new reference. The Trnalsate tool should allow switching the base language by marking a new language as the source (it will then rewrite the base page with the content of the translation (inserting there the translate tags automatically, instead of inserting only the translated items).

• My version of Fallback supports that: another base language than English, BCP47 default fallbacks, cached list of resolved fallbacks, accelerated with no recursion. My version of TNT supports alternate namespaces.
• My version of TNT also can work on page that have still not been marked for translation (the current version here on Commons fails with "Script error" becaus it attempts to transclude a "/en" subpage that still does not exist). The interest is that you don't have to worry if you have to use TNT or not when transcluding a template. You can add TNT immediately, then prepare the target for translation by adding "translate" tags. These translate tags won't appear on the page, they have no effect until that page is effectively marked for translation and subpages start being created (including the subpage for "/en").
• Finally the current version of LangSwitch lacks several improvements I made also on Meta, notably tagging properly the possible changes of directions, using "bdi" tags every time a fallback is used that does not match the content language for the current page. The bdi tag should indicate also with lang="" and dir="" attributes the language used, including for fallbacks. This is important for correct rendering of these languages as pages may become multilingual (otherwise, punctuations are mixed up, and fonts are not setup correctly to render some scripts like Burmese/Myanmarese, Tibetan, Yi, or Gotic, or the wrong font is used for the script style needed in Urdu, Farsi and Dari). In a multilingual page as well, we could have BOTH simplified Chinese AND traditional Chinese and mixing them without tagging will not allow correct reading in one of the other (where the variants are not interchangeable like in monolingual pages using ULS).
• Also I really don't like that "/en" is locked, this is absolutely not needed: the TNT template can work to render the text modified in the "/en" version via the Translate tool. The base page (used to mark the source of translations) has its own separate life. My opinion id that this source base page should be treated like an unspecified language (und) or as a variant of English (such as "en-x-source"), and the Translate admin should be allowed to specified in which language the source is really ("en-x-source" or "zh-x-source"). Here again the support of BCP47 fallbacks from a variant of a language (like "zh-x-source") to shorter codes with less subtags (such as "zh" here), means that the other transcluded templates will still render correctly (they may not find "zh-x-source" for these separate templates, but will still find "zh").

In summary, the Translate tool should not lock down any page. Translations should be available to any editor (just like any editor can work on templates based on LangSwitch), even if only a tranlate admin can mark which page to use as the source for translation or for updating the source. But even in this case, as all languages will be unlocked, the Translate tool should not just display the internally marked source, but should also display the unlocked "English" translations in "/en" remaining modifiable via this tool. The reference language for translators should not be only English, they should be able to display and use any other translations and the tool will display if this other translation is fuzzy according to the current referene language (not the one in the fully editable base page). verdy_p (talk) 10:45, 19 January 2014 (UTC)

LangSwitch fallback broken @{{Self-photographed}}

de-ch should fallback to de. -- Rillke^(q?) 11:30, 20 December 2013 (UTC)

The issue is at l.17 where new is  de-ch and ' ' .. l evals to  de. -- Rillke^(q?) 12:12, 20 December 2013 (UTC)

If you don't mind, I convert the logic to make use of a table. -- Rillke^(q?) 12:14, 20 December 2013 (UTC)

In the sandbox, an accelerated version supports not only the preferred fallbacks kisted, but also just after them the BCP47 fallbacks

This sandbox is optimized and also use cached results if the module is invoked many times with the same language when viewing the same rendered page.

I have passed time to also reduce all unneeded recursions, all is performed in the same function. verdy_p (talk) 10:04, 19 January 2014 (UTC)

Input/output

It would be nice if Module:Fallback/doc described what's the input and what's the output of this template. --Federico Leva (BEIC) (talk) 14:52, 29 September 2014 (UTC)

Lua error in Module:Fallback at line 73: attempt to index a nil value.

See User talk:Prada16. I don't understand why this error comes on this page (only?). Regards, Yann (talk) 15:10, 9 December 2014 (UTC)

I cannot see any error. Where is it, and using what language ? --Zolo (talk) 07:01, 12 December 2014 (UTC)

The error disappeared. I still wonder what happened... Yann (talk) 09:39, 12 December 2014 (UTC)

Lua error: too many expensive function calls

I thought last month this problem will maybe soon solved, but I do still see on COM:T many templates not visible through this error-msg (Module:Fallback: in function "fallbackpage"; Module:Fallback: in function "chunk" ↔ User: Perhelion (Commons: = crap?)  19:07, 2 February 2015 (UTC)

I suggest you trace the issue down and fix it, then. It's a wiki. -- Rillke^(q?) 19:33, 2 February 2015 (UTC)

User: Perhelion, What page shows that error? --Jarekt (talk) 20:47, 2 February 2015 (UTC)

@Rillke: Ok, that's a point, but I'm not so experienced with Lua and this is one of the most used modules on Commons

@Jarekt: Ok, not the first templates, after Commons:Templates#Related files the most inclusions shows this error (I mean I saw this the first time at end of December last year)!?

↔ User: Perhelion (Commons: = crap?)  22:39, 2 February 2015 (UTC)

Ah, I see. I do not see a way to fix it. Old template code had as many tests for existence of a page and it worked. Lua does not, that is a problem with Lua. --Jarekt (talk) 13:02, 3 February 2015 (UTC)

PS: But I know from JavaScript, IF in a LOOP is bad for performance, and here are two IF in a loop. Maybe this can solved in other way. ↔ User: Perhelion (Commons: = crap?)  23:00, 2 February 2015 (UTC)

The "expensive function" is mw.title.new().exists. I do not think there is any way to get around that without rewriting all templates using {{Autotranslate}}. If you ask me I would very much like to stop using autotranslate and showing clutterish list of langs inside boxes. --Zolo (talk) 06:50, 3 February 2015 (UTC)

Maybe the config of commons can be changed to fix this (changing a limit?) --Steinsplitter (talk) 13:35, 3 February 2015 (UTC)

This issue is new. I think we need to find a forum where problems with Lua are discussed and report it. May be people that know more about it will have some suggestions. --Jarekt (talk) 14:22, 3 February 2015 (UTC)

Maybe wmf can change memoryLimit and/or cpuLimit in $wgScribuntoEngineConf (mw:Extension:Scribunto), but not sure if this fix the problem. Maybe reporting the problem in phabricator:? --Steinsplitter (talk) 14:54, 3 February 2015 (UTC)

This is neither specific to Commons nor specific to Lua. There has always been a limit of 500 hundred expensive function calls in all Wikis. If you preview Commons:Templates, you see that Lua-related statistics are fine (Lua memory usage: 2Mo out of 50, Lua time uage < 1 s out of 10). The issue is that title.exists counts as an expensive function, just like the #ifexist wikiparser, and here we have 642 calls to expensive functions (for the English version, can be more in other languages). --Zolo (talk) 15:24, 3 February 2015 (UTC)

The thing is that COM:T displayed all templates with calls to #ifexist and I think I have seen it working with Lua calls, but now it is not working. Another approach is to either split or retire COM:T. We do have many thousands of templates and it is hard to show even representative sample on a single page. Also many templates did not show correctly on that page, since they might require some parameters which are hard to pass. --Jarekt (talk) 16:12, 3 February 2015 (UTC)

I did not know [Commons:Template]] but it seems relatively useful in getting a bird's eye view on important template. In any case, it requires some cleanup at the very least, and there does not seem to be any harm in splitting it. --Zolo (talk) 06:58, 4 February 2015 (UTC)

┌─────────────────────┘

Hey guys, I've now two solutions (I've traced the issue down by myself as requested)

1. Remove only the Template: tlrow seems to work, as you can see on the last version without.
2. Include only the /en subpage for all autotranslate templates, this will exclude all this Lua-Fallback.

But first I ask here for reviews, the remove of tlrow will swell a bit the source text. But IMHO in fact for this page this is ok. ↔ User: Perhelion (Commons: = crap?)  16:21, 24 February 2015 (UTC)

 Done I choosed the 2nd solution, however the template "tlrow" does seems not to have been guilty in that manner. ↔ User: Perhelion (Commons: = crap?)  19:57, 24 February 2015 (UTC)

PS: All translations of COM:T transclusion need the same fix (fr, ... @Zolo it was your hint which give me the fix. I guess it is only table copy to each headline.) ↔ User: Perhelion (Commons: = crap?)  11:38, 19 March 2015 (UTC)

@Rillke, Jarekt, Steinsplitter, and Zolo: Maybe it is good to install the ext.translation i18n here ? To end this problem here... I would do the /i18n thing (but I'm not very experienced here) !? ↔ User: Perhelion (Commons: = crap?)  11:52, 19 March 2015 (UTC)

 Done Fixed this today Special:Diff/312514946 -- User: Perhelion 19:51, 26 July 2018 (UTC)

mediawiki fallbacks

{{Edit request}} Make this change to this module.

I noticed that one of the authors of this template requested an function in scribunto to get fallbacks, but never added it to this module. I hope that asking that it will be added now to the module will fix that. This edit will also fix the errors under the "test_fallbacks" section on Module talk:Fallback/testcases.--Snaevar (talk) 22:09, 20 February 2015 (UTC)

One specificity of the Commons-made fallbacklist is that unlike MediaWiki's built-in function, it can be bidirection, like zh-cn -> zh and zh-> zh-cn.

{{#invoke:Fallback|langSwitch|zh-cn=zh-cn|lang=zh|default=default}} -> zh-cn

{{#invoke:Fallback/sandbox|langSwitch|zh-cn=zh-cn|lang=zh|default=default}} -> zh-cn.--Zolo (talk) 22:03, 25 February 2015 (UTC)

What to do with this editrquest? Fulfilling or not? --Steinsplitter (talk) 14:28, 7 March 2015 (UTC)

I am clueless. For new code, I would use the MediaWiki supplied ones; if we change this code people may start to wonder why we "broke" the fallback chain for their language. On the other hand we would have consistency with what MediaWiki does and if someone requests a change, we could point them to MediaWiki's code base/ Phabricator, thus having less maintenance efforts but less control, too - so we can't simply quickly fixing some fallback chains if we see a need for. -- Rillke^(q?) 14:53, 7 March 2015 (UTC)

Can we do both? Use MediaWiki list for most and add exceptions? we do something similar with Template:And. --Jarekt (talk) 19:54, 7 March 2015 (UTC)

 Not done No consensus to change the module. --Hedwig in Washington ^(mail?) 04:44, 2 August 2015 (UTC)

Lua Errors

{{PD-scan|PD-signature}} gives

This image is in the public domain because it is a mere mechanical scan or photocopy of a public domain original, or – from the available evidence – is so similar to such a scan or photocopy that no copyright protection can be expected to arise. The original itself is in the public domain for the following reason: This signature is believed to be ineligible for copyright and therefore in the public domain because it falls below the required level of originality for copyright protection both in the United States and in the source country (if different). In this case, the source country (e.g. the country of nationality of the signatory) is believed to be ***Tag invalid. You must provide the source country as a parameter.***. Note that this tag cannot be used on all signatures, as not all signatures are copyright-free. See Commons:When to use the PD-signature tag for an explanation of when the tag may be used. This tag is designed for use where there may be a need to assert that any enhancements (eg brightness, contrast, colour-matching, sharpening) are in themselves insufficiently creative to generate a new copyright. It can be used where it is unknown whether any enhancements have been made, as well as when the enhancements are clear but insufficient. For known raw unenhanced scans you can use an appropriate {{PD-old}} tag instead. For usage, see Commons:When to use the PD-scan tag. Note: This tag applies to scans and photocopies only. For photographs of public domain originals taken from afar, {{PD-Art}} may be applicable. See Commons:When to use the PD-Art tag.

. Anybody knows how to fix it? --Jarekt (talk) 13:12, 10 May 2016 (UTC)

mediawiki fallbacks again

I was just doing some cleanup of this module suggested in phabricator:T171392, see Module:Fallback/sandbox. I was looking at Module talk:Fallback/testcases test_fallbacks output which compares Commons fallback chain for each language (Expected column) with the global list used by all the other projects (Actual column). In many cases where the two differ I think the fallback list provided by everybody else is better than ours. I think Commons fallback list was mostly frozen a decade ago and the global list is always evolving. The Module:Fallback/sandbox is using global fallback list. Are there any objections to using that one? --Jarekt (talk) 16:33, 7 September 2017 (UTC)

I'd  Support. Long overdue --Zhuyifei1999 (talk) 03:53, 8 September 2017 (UTC)

 Done --Jarekt (talk) 11:56, 8 September 2017 (UTC)

Incorrect handling of the "default" option

The code currently incorrectly uses the fallbacks we need on Commons. Internally Mediawiki fallbback chains don't have any "default" option returned, but places "en" at end.

As well, as noted above, the MediaWiki fallback chains are almost frozen since long, while in Commons we have patiently built a more extensive and more precise list (there may remaiun some differences to handle, see the testcases about which version we want (actualkly we have chosen to keep the local list of fallbacks when they are locally tuned, and MW fallbacks are just used if we don't specify a local override).

For the fallback to work correctly, we need to remove the "en" entry, add "default" and then add "en" after it like this:

function remove(list, value)
	for k, v in ipairs(list) do
		if v == value then
			table.remove(list, k)
		end
	end
end
function has(list, value)
	for k, v in ipairs(list) do
		if v == value then
			return true
		end
	end
	return false
end
function mwlist(langcode) -- expected
	local list = mw.language.getFallbacksFor(langcode)
	if langcode ~= 'en' and langcode:match('^en%-') == nil then -- keep 'en' first, if lang is 'en' or 'en-*'
		remove(list, 'en')
	end
	table.insert(list, 'default') -- this must come before 'en'
	if not has(list, 'en') then
		table.insert(list, 'en') -- this must come last after 'default'
	end
	return mw.text.listToText(list)
end

See the Module talk:Fallback/testcases for what we really need on Commons (and which implements this strategy). In my opinion, MediaWiki should include a "default" entry which will be before "en" always at end, unless a language has an explicit fallback to "en" that must not use the default (I really doubt this is the case; for "en-GB", the fallback should still use "default" before "en" which remains at end).

So this existing incorrect code (which is still incorrect has it can list "en" twice in some cases, if it is used to query the fallback chain for "en" itself):

function p.fblist(lang) -- list the full fallback chain from a language to en
	local fbtable = p.fallbackloop{ lang:lower() }
	table.insert(fbtable, 'default')
	table.insert(fbtable, 'en')
	return fbtable
end

should become something like:

function remove(list, value)
	for k, v in ipairs(list) do
		if v == value then
			table.remove(list, k)
		end
	end
end
function has(list, value)
	for k, v in ipairs(list) do
		if v == value then
			return true
		end
	end
	return false
end
function p.fblist(lang) -- list the full fallback chain from a language to en
	local fbtable = p.fallbackloop{ lang:lower() }
	if langcode ~= 'en' and langcode:match('^en%-') == nil then -- keep 'en' first, if lang is 'en' or 'en-*'
		remove(fbtable, 'en')
	end
	table.insert(fbtable, 'default') -- this must come before 'en'
	if not has(list, 'en') then
		table.insert(fbtable, 'en') -- this must come last after 'default'
	end
	return fbtable
end

Actually the solution was more complex because this fixes only Autotranslate, but still not LangSwitch which was using the MediaWiki fallbacks only (without any "default"), so I had to reconciliate both. (see below and the sandbox version for the final solution).

verdy_p (talk) 17:13, 17 February 2018 (UTC)

This is related to template talk: LangSwitch‎‎ #Using default with en. Incnis Mrsi (talk) 19:34, 17 February 2018 (UTC)

I'm currently using this to test it in the /sandbox version of this module. verdy_p (talk) 21:02, 17 February 2018 (UTC)

So what? https://commons.wikimedia.org/w/index.php?title=Template:Maps&uselang=sv and https://commons.wikimedia.org/w/index.php?title=Template:Maps&uselang=el do not show improvements. Incnis Mrsi (talk) 21:50, 17 February 2018 (UTC)

This test is OK: Template:Maps now correctly uses the "default" (value taken from wikidata) before "en", except with "?uselang=en" or "?uselang=en-gb" where the LangSwitch does not consider the provided "en=*" text after the default. In all cases, "en" will be in the fallback list along with "default"; the translation in "en=*" will be used before "default=*" (with "uselang=en or en-*"), but the translalion "default=*" will be used before "en=*" in all other languages.

You did not look correctly ! verdy_p (talk) 21:53, 17 February 2018 (UTC)

oRLY? Let’s try lang:

{{Maps|lang=sv}} → Kartor
{{Maps|lang=el}} → Χάρτης

Do you see anything in Swedish or Greek? Whereas we have German “Landkarten”. Incnis Mrsi (talk) 22:02, 17 February 2018 (UTC)

Please, do not edit my posting! I do not want my examples to be adulterated with lang=gsw or similar, although I admit some languages didn’t function in my revision of {{Maps}}. Incnis Mrsi (talk) 12:37, 18 February 2018 (UTC)

@Verdy p: when will you accomplish your sandbox job? It was you who started to interfere with {{Maps}}. I did not ask you to fix anything, only to read discussions. Incnis Mrsi (talk) 09:37, 18 February 2018 (UTC)

Hello! Thank you Verdy p for this fix! I'm looking forward to when the change will go live (out of the sandbox) and we can revert all of the gallery headings templates back to LangSwitch. Thank you so much for your work! --Reinhard Müller (talk) 10:27, 18 February 2018 (UTC)

@Reinhard Müller: you missed the point by no less than 1 metre. {{Maps}} now depends on {{LangSwitch/sandbox}} which, in turn, invokes module: Fallback/sandbox. And default= in {{Maps}} demonstrably doesn’t work. What do you thank Verdy for? Incnis Mrsi (talk) 10:42, 18 February 2018 (UTC)

@Verdy p: yes, it seems that default is not used at all - can you please look into this? Thanks --Reinhard Müller (talk) 10:59, 18 February 2018 (UTC)

The sandbox version is still not complete. The code above does not apply to LangSwitch which still uses only the MediaWiki fallbacks (not handling the "default") and not the local fallbacks in the code above. The code above however is correct for the local fallbacks, LangSwitch requires similar update.

I'm still working on the sandbox of Module:Fallback to test the behavior (only for Template:Maps which has very small uses and is the only template in which I have activated the use of the LangSwitch/sandbox). verdy_p (talk) 11:49, 18 February 2018 (UTC)

For now I've tested that fallback lists were correctly inherited (see Module talk:Fallback/testcases which still does not test the behavior of LangSwitch, but only displays and compares builtin MediaWiki fallbacks and local fallbacks on Commons), and correctly determines where to apply the "default" option. I've commented that page to show why there are still differences, but these differences do not apply to the tests above. We just need the "default" to be handled as well in LangSwitch in the correct order (i.e. before "en", except for "en" and "en-*").

I'm thinking about reconciliate both : use local fallbacks first (recursively chaining them), then add MediaWiki fallbacks (except "en" when the requested language is "en" or "en-*"), then add "default", then finally add "en" if it's still not in the chain. As well the existing code is not optimal as it uses recursion, when it can be changed easily into trail recursion, then to a loop to avoid the recursion completely to create the fallback chain: with the loop version it can be greatly accelerated by to parsing the list multiple times on the same items. verdy_p (talk) 11:58, 18 February 2018 (UTC)

Wikimedia Commons doesn’t warrant all this wall of text. We are waiting for a solution after this unilateral action. It may a brand-new function or even a separate module, but it must do what we need for multilingual tags, not even {{Maps}} specifically. Or disengage and me together with Reinhard will make our own repairs. Incnis Mrsi (talk) 12:37, 18 February 2018 (UTC)

Not at all a wall of text, but technical details to explain what's really happening.

LangSwitch was broken sometime in the past where it did no longer support the "default" option correctly ("en" taking priority each time it is present, in which case "default" is silently ignored). Yes there's a bug, it's not as simple as what you think (and using "#switch" instead of LangSwitch is exactly because we still want a default on commons, but LangSwitch uses fallbacks since always, where #switch does not fallback at all ! What I mean is that I just want to restore this default: that's exactly why we have a sandbox version used on a very limited case (Template:Maps is for now used only very few galery pages for areas where the native languages are already handled correctly for the languages; you want the default for other languages, me too, but even before the change this default was still only English and nothing else (so even the current version does not break the Maps template at all). verdy_p (talk) 13:45, 18 February 2018 (UTC)

The fix is there: all fallbacks are honored, those from Commons first (for the local overrides we need and have always used, notably for Autotranslate), then those fallbacks that are builtin from Mediawiki (except 'en' if the requested language is not English), then 'default', then 'en' (if the requested language is not English). LangSwitch is unified in the sandbox version. See it for the Maps above.

verdy_p (talk) 15:15, 18 February 2018 (UTC)

Finally I've fixed (or added) a few testcases, and separated the group of tests of LangSwitch with "default=*" specified. And I made the module now more BCP 47 conformant for languages with variants to add automatically the base language to the list (e.g. Chinese, Serbian, English, French, German, Dutch...) verdy_p (talk) 17:51, 18 February 2018 (UTC)

Also the "/testcases" module is now also in its own "/testcase/sandbox" (it uses the /sandbox version of the base module) (with its results in its own talk page)

I've removed the "/sandbox" from the base "/testcases", but these are the same tests, only using the base module instead of the sandbox module.

You can compare the two behaviors by looking in talk pages of the two testcases versions (base or sandbox) and you'll see some differences where the current implementation fails but not the "/sandbox" version. The remaining failures in "/sandbox" (between "Actual" and "Expected" come from incorrect defaults in the "expected" MediaWiki only fallbacks which has no 'default'), and are the same failures between the two versions. In summary, we should no longer use only the MediaWiki fallbacks.

I don't see now how to improve the fallback lists more than what I did in the "/sandbox" module, but you may suggest some additional fixes if needed (for example I've excluded the automatic inclusion of "be-x" as a valid fallback from "be-x-old" during the BCP47 expansion phases, where it truncates it more to "be"; and I disabled the inclusion of "roa" as a valid fallback from "roa-tara", or "fiu" as a valid fallback from "fiu-vro").

I've not managed to add automatically some known aliases for legacy codes (they are already in the local fallbacks module, but some of them may be missing, since Wikimedia is converting legacy codes to more standard ones): adding them would facilitate the transition of existing translations, but we may need to track them. verdy_p (talk) 18:59, 18 February 2018 (UTC)

@Verdy p: I've tested a few languages with the "Maps" template, and as far as I can tell, it works perfectly. Please let me know when this updated code moves from the sandbox to the "real" template, so I'll update the "Maps" and other related templates. Thank you! --Reinhard Müller (talk) 20:59, 18 February 2018 (UTC)

@Reinhard Müller: I cannot move the sandbox to the normal version myself (the base template is currently blocked against editing by non-admins, because it is widely used on many pages). This is the reason I had to do that on the "/sandbox", so that we can create limited tests only on subsets of pages without breaking too many pages.

But now we can ask to administrators to do that (the code is ready), unless we want to do some more tests elsewhere (I think about "Autotranslated" templates, such as licence templates: we can create a fake licence, or derive one in its own sandbox to use the new code in Module:Fallback/sandbox (where it is used by "#invoke:Fallback"), and see what happens in a test file tagged with this sandboxed licence.

What you can do for now before the code is updated in the base Module:Fallback, is already to use the "/sandbox" version in similar templates (we can know the list of pages using the /sandbox version by the "what links here" in the toolbox displayed from the Module:Fallback/sandbox page, or more precisely from the Template:LangSwitch/sandbox page for pages using {{LangSwitch/sandbox|...}} instead of {{LangSwitch|...}}, so once it is deployed (by an admin), it is easy to remove the "/sandbox" suffix). Admins will appreciate that we have conducted enough tests before applying the change. You can do such edits easily, the impact will be local only to pages you have edited and we have a tracker to list the pages using the sandbox.

The sandbox module also has a version from its history: if someone modifies it and it breaks out tests, it is easy to revert to the last version I edited. This is also true for the sandox Template:LangSwitch/sandbox which is trivially almost identical with Template:LangSwitch (except that it changes the name of the module to use the sandbox module, so I see no reason why someone would need to modify it as it is espacially made to use the sandbox module without changing anything else or changing any parameters).

You'll also note that the new sandbox version seems complex, but it still runs faster (in most cases except where it can find additional translations in fallback languages or in the "default" option that were initially ignored), it uses a bit less memory in MediaWiki parser, it has no cost in terms of expansion by MediaWiki, it also uses less memory and less time in Lua/Scribunto beause the loops for looking up the list of fallback is more efficient, avoids non trailing recusions, and scan the codes much less times.

Its code is also slightly better documented, explaining what it does and why. verdy_p (talk) 22:05, 18 February 2018 (UTC)

Please check my very minor edits to Module:Fallback/sandbox which cleaned the whitespace and fixed unintended globals. By the way, function _removevalue is not used and should be removed. I looked at Module:Fallback months ago and was perplexed by the way it went over the same list multiple times, although I forget the details now. More efficient code would be good but the changes are more extensive than I have time or skill to ponder. Are you sure that deviating from MediaWiki's fallback rules is desirable? Are there some usage cases showing that MediaWiki produces a poor result which is fixed in the sandbox? Jarekt should consider this. Johnuniq (talk) 03:00, 20 February 2018 (UTC)

The original post misquotes my statement in the previous discussion. I was saying that Commons fallback list did not change in a decade and was quite off from much wider used MediaWiki list which gets much more timely updates. For a while MediaWiki fallback list was not accessible in Lua, so we wrote Module:Fallbacklist, but once it become accessible we started using it. I do not think we want to go back to the our own "Commons-only" fallback list, over the list used by all the other Wikimedia projects. As for the use of default option it was always meant for cases when "en" version is not available. I originally added it to {{LangSwitch}} after observing people labeling text which was clearly not English as English, only so it something would be visible. Default was de-facto alternative name for English. So right now the last 2 entries in the fallback chain are English followed by default. We could switch their order, but that would interfere with the current MediaWiki fallback order. The behavior is different only when both are present. So far I have not seen any examples for cases where we want someone for hum we do not have some text in their preferred language, to have the text shown in user specified default language when English text is present. --Jarekt (talk) 05:13, 20 February 2018 (UTC)

@Jarekt: did you see category talk:Multilingual tags: Gallery headings and related? There are situations where languages with no sensible backup option may not fall to English. Incnis Mrsi (talk) 07:33, 20 February 2018 (UTC)

Incnis Mrsi, thank you for pointing me to that discussion, it clarifies this one. I am still at lost for specific examples where fallback chain with default before en would make a difference. User:Verdy p I think gave the most comprehensive arguments, although I have never heard about international English and I am not sure how it differs from US and British flavors or from standard English. I also often find that English has many official spellings. Archeology and Archaeology are both valid, so is Łódź and Lodz (Lodz is almost unrecognizable to Polish speaker). That said, I am not opposed to having default before English. It just complicates the code and I see no specific examples where it would make a difference. --Jarekt (talk) 13:29, 20 February 2018 (UTC)

There's a difference notably because "default" tracks all missing translations where it is used, where we'll want to display an additional message (or icon) asking people to provide a translation instead of the displayed English, possibly emphasized with some red color (example fuzzy translations that have been disabled), but that message (and categorization) will NOT be used for English itself ("en" will be before "default" only when requesting English, but the "e" resource will be after "default" in all other languages; and even that "en" resource may be temporarily disabled or missing when the "en" resources is also fuzzy and is requested by the "default": this occurs when a message first starts from a template whose original was NOT English but another language, e.g. German; in that case you have only "de=*" and "default=*" specified containing the same message in German with a translation request markup, or an approximative transaltion in English; then someone translates it in English and add "en=*", but default=*" is kept normally unchanged and continues to be the original.). verdy_p (talk) 16:26, 20 February 2018 (UTC)

I can also give another reason for justifying "default" being distinct from "en": look at official toponyms in English of India, they are the default to use for all other languages, except possibly English itself which may keep "traditional" names still widely used in US or Britain, so that pure-English version is not to be used as a reasonable default for all other languages than English (they may still provide their own translation: "default=Mumbay" (official in India), "en-in=Mumbay", "en=Bombay" (traditional in US or UK), "fr=Bombay")...

Having a distinct "default" is the most flexible option: it is equivalent to the "root" locale in CLDR, where there are also default values for all locales, that are different from values specified specifically only for English.

Note that having "default" or "en" is not there because there may exist "synonyms" in English (like Archeology or Archaeology): this occurs also in all other languages as well (including for the "default"), we have to choose one of those synonyms in all cases, which is reasonable for that specific language or for all other languages or variant that would fallback to that specific language. verdy_p (talk) 16:43, 20 February 2018 (UTC)

That is a good point. In order to swap order from en/default to default/en, the simplest way would be to do this (single line change). Please see test at Template:LangSwitch/testcases. Would that be sufficient? --Jarekt (talk) 04:22, 21 February 2018 (UTC)

Your change is wrong and clearly insufficient, my sandbox handles this case more cleanly and more completely than this test just based on the number of elements in the fallback list where you conditionnally add English, but where on Commons BOTH "default" and "en" must be present in the fallback list but NOT necessarily at the same position. Your test for example will break when looking for fallbacks from English or variants of English ("en-*"). Look at the result testcase of my sandbox: "en" and "default" are always both present (but may be swapped in the order). And still your single line change does not handle BCP47 compatibility, and does not uniformize the different fallbacks used by Autotranslate (based on local lists of fallbacks only) and LangSwitch (based only on the "frozen" MEdiaWiki builtin fallbacks designed only for its UI and across all wikis, including outside Wikimedia). verdy_p (talk) 16:07, 21 February 2018 (UTC)

My change addressed the issue you described. It creates list of fallbacks that always has both default and en options. In most cases default is before en, except for English language for which the list is en followed by default. That is all it does. I do not know what is your desired behavior for variants of English ("en-*"), but I have never seen anybody using them with {{LangSwitch}} template before, so I am not sure it is necessary to complicate code used on 20M pages to carve out special behavior for rarely (or never) used cases. As for compatibility with BCP47, I think it is fine. Both {{LangSwitch}} and {{Autotranslate}} use the same fallback list all the other MediaWiki projects use. That list is used by many more projects and is updated through requests to phabricator. There is not need to go back to our home-brewed version we created in 2009. Choice of fallback languages, require many assumptions about what language people might want to see the text in if language in their language is not available. That choice can be very delicate. For example after Annexation of Crimea by the Russian Federation and conflict between Ukraine and Russia, we suddenly had a lot of complaints from angry Ukrainian users to not use Russian as fallback language for Ukrainian, which resulted in phabricator:T39314. I think, MediaWiki with debates on phabricator is a better way to manage this delicate issue than our home-brewed solutions that receive very little scrutiny. It also provides uniformity of behavior. --Jarekt (talk) 17:25, 21 February 2018 (UTC)

@Jarekt: can the MediaWiki solution do anything we can hardly manage ourselves? If both “their” and “our” solutions are essentially directed trees, then what would be the benefit of reliance on MediaWiki except for said uniformity? Incnis Mrsi (talk) 17:36, 21 February 2018 (UTC)

Incnis Mrsi, MediaWiki solution is what all the other projects are using, and where changes are reviewed and discussed by large cross-section of users, see for example phabricator:T39314. Changes to our {{GetFallback}}, {{GetFallback2}} and Module:Fallbacklist generate much less discussion. Unless there is a clear advantage of picking different fallback languages than everybody else I would rather use a single repository. Choice of fallback language should not differ from project to project and the only reason we had those templates was because for the longest time MediaWiki fallback was not available to templates or modules, forcing us to maintain our own copy. --Jarekt (talk) 18:33, 21 February 2018 (UTC)

Can we develop some intermediate solution? I mean, relying on the MediaWiki data in general, but having hooks for our own Commons-specific things able to override MediaWiki ones. Not only reversal of the acrimonious default—en edge of the tree, but a list of arbitrary corrections which will took priority over MediaWiki. It is not an easy task, I realize, because directed cycles may not happen even in the case of a sudden change in the MediaWiki language data. Incnis Mrsi (talk) 18:44, 21 February 2018 (UTC)

Which of the MediaWiki choices do you think are wrong? If there is a Commons consensus of language speakers that fallback for some language is wrong, and we can not convince MediaWiki gate keepers to change it, than I think it would be appropriate to have a local overwrite. However personally, I feel qualified to have opinion about only 2 languages I know. In case of all the other languages I prefer to leave it to people that speak them. --Jarekt (talk) 02:36, 22 February 2018 (UTC)

Note that my code in the sandbox already handles the priority of all fallbacks in Commons Module:Fallbacklist to override any other fallback that may be built in MediaWiki (frozen for its UI and designed for all wikis, not just those of Wikimedia), but fallbacks that are in MEdiaWiki are taken into account after that

this means that some fallbacks that we have put in Module:Fallbacklist are no longer necessary with my sandbox version; note that my code automatically adds fallbacks from "LANGCODE-VARIANT" to "LANGCODE" (with the same "LANGCODE" value), but still AFTER the local overrides in Module:Fallbacklist (what this means is that the fallback list for "zh-hans" to "zh" is automatic but will occurt after any specified fallback to "zh-hant", unless we specify in Module:Fallbacklist that "zh-hans" should first fallback to "zh" before trying another variant like "zh-cn" (in which case "zh" will not be added automatically to "zh", "zh-cn").

To make this clear: fallbacks in Module:Fallbacklist like "Lang1-variant1" to {"Lang1-variant2", "Lang2-variant2", "Lang1"} contains the superfluous fallback "Lang1": you can remove items from the end of these fallback chain as the BCP47 default mechanism is implemented and "Lang1" is automatically added from the source language "Lang1-variant1", and "Lang2" is added after from the specified fallback "Lang2-variant2"; then after this MediaWiki fallbacks are considered. The following however are useful and not superfluous:

['nds'] = {'nds-nl', 'de'}, -- Low German

['nds-nl'] = {'nds', 'nl'}, -- Low Saxon (Netherlands)

In the second line, 'nds' would be added automatically from the source 'ndsl-nl', but only after 'nl'. But in the lines below:

['kk-arab'] = {'kk-cyrl', 'kk'}, -- Kazakh (Arabic script)

['kk-cn'] = {'kk-arab', 'kk-cyrl', 'kk'}, -- Kazakh (China)

['kk-cyrl'] = {'kk'}, -- Kazakh (Cyrillic script)

['kk-kz'] = {'kk', 'kk-cyrl'}, -- Kazakh (Kazakhstan)

['kk-latn'] = {'kk'}, -- Kazakh (Latin script)

['kk-tr'] = {'kk-latn', 'kk'}, -- Kazakh (Turkey)

all 'kk' entries at end of lists are superfluous, because they are automatically added from the source language 'kk-*' in the mapped key, and we don't even need these two lines at all (they are implicit from BCP47 rules and are automatically infered by my sandbox version):

['kk-cyrl'] = {'kk'}, -- Kazakh (Cyrillic script)

['kk-latn'] = {'kk'}, -- Kazakh (Latin script)

Note that {{GetFallback}}, {{GetFallback2}} and Module:Fallbacklist are currently NOT used at all by LangSwitch which currently only uses the Mediawiki fallbacks. My sandbox reunites everything and uses the Module:Fallbacklist in all cases without forgetting any fallback from MediaWiki (but they are at low priority). verdy_p (talk) 06:48, 23 February 2018 (UTC)

This threat was originally about Incorrect handling of the "default" option and I agree that new proposed behavior when both "default" and "en" options are present would be an improvement. However I strongly disagree with rolling back to use old fallback definitions in Module:Fallbacklist module. User:Zolo who wrote that module, was the first one to propose to retire it once phabricator:T50328 was fixed. Verdy_p, You did a lot of work on rewriting and testing Module:Fallback/sandbox, however I believe you are proposing to take us in the wrong direction. If there is something wrong with the current MediaWiki fallback for some language, than we should file a ticket on phabricator, and change that language settings if the people from the community speaking it want the change. I would support local settings only if such phabricator ticket fails (or takes too long to be acted on) while there is consensus of community speaking given language to change it. --Jarekt (talk) 15:44, 26 February 2018 (UTC)

The "old fallback mechanism" used in Commons is preserved as long as these fallbacks are defined there (there were reasons for defining them as overrides, where Mediawiki did not want them at all because its fallbacks are targetting all wikis (not just Commons), including wikis outside Wikimedia. So even if changes have been requested in tickets for MediaWiki, they've constantly been frozen since years, when Commons needed to advance faster, and also manage its own local content. So yes there are overrides in the local Fallbacklist, but as I said, some of them won't be needed at all and may safely be removed if they are already present in the MediaWiki fallbacks, or if they are infered directly by BCP47 rules. This leaves only a few entries that Commons needs to manage internally with its fallback list as local overrides. We can reduce these overrides, but removing them is for not NOT an option. And we still have incoherent behavior with the current version where Autotranslate uses them (but not any Mediawiki fallback for now), and LangSwitch uses only the MediaWiki fallbacks (and not any MediaWiki fallbacks): this is exactly what I have resolved to make them converge, in exactly the right direction you want : using all MediaWiki fallbacks everywhere, but possibly at lower priorities than the local fallbacks (as long as they are defined and the content on Commons is not migrated, or as long as MediaWiki fallbacks are defective and forget fallbacks still wanted on Commons) ! I was completely coherent and made the behavior coherent. I've not taken any decision about the future of local fallbacks and those we need to keep (they were decided collectively by Commons users, independantly of the small team that manages Mediawiki for other purposes than just Wikimedia sites). verdy_p (talk) 21:47, 26 February 2018 (UTC)

I do not believe {{Autotranslate}} is using Module:Fallbacklist or Template:GetFallback where local fallback options are stored. {{Fallback}} still uses local fallback list, because it is a rarely used template which was never converted to use lua. Servers take forever to adjust transclussion tables, so it is hard to track of what page is using what. I also have not seen anybody reporting on phabricator that MediaWiki fallbacks are defective. If you know that some of them are wrong please file a ticket. Our local fallback options are not overwrites to Mediawiki ones, but a full parallel copy, which is mostly not used anymore. But can we get back to the Incorrect handling of the "default" option which was the original issue. I think we agree on that and on the topic of fallback we might just respectfully agree to disagree. --Jarekt (talk) 14:30, 28 February 2018 (UTC)

@Verdy p: thanks for writing Module:Fallback/sandbox/doc, but could you document the current sandboxed module in a more regular fashion, please? To describe what are its main differences from the “industrial” version. I am uncomfortable with reading long threads. Incnis Mrsi (talk) 18:54, 26 February 2018 (UTC)

My doc for the sandbox is jsut there to signal this is a sandbox, the actual usage is already in the normal version and will require almost no change. I cannot just overwrite the doc for the sandbox by the normal doc, as it would display incorrect usage and would not signal this is a sandbox. verdy_p (talk) 21:47, 26 February 2018 (UTC)

And if you have not followed everything above, just look at the description of the result page of the sandbox, which summarizes the rules applied: Module talk:Fallback/testcases (current broken implementation) with the results of my version in Module talk:Fallback/sandbox/testcases (more successful!). you'll see that I'm not loosing any existing translation provided in Commons, I handle all tricky cases (notable variants of Serbian, Chinese, Norwegian, and even English, as well as I preserve the compatility with legacy codes still used for translations in Commons such as be-x-old, or zh-min-nan, they will be found even if the new code is still not used!). This allows smooth transition and time to cleanup these codes used in many pages and templates without breaking anything. verdy_p (talk) 01:35, 27 February 2018 (UTC)

Module:Autotranslate

@Jarekt: Hi, I have not read anything of the complicated LUA code, the descriptions, discussions and other informations. Just a question: The Template:Infobox flag has problems. It is funny written, all parameters in UPPER CASE - and this seems not to work [since some time?]. Parameters in lower case are passed very well (and unnamed params too, of course). As another difficulty, I cannot change all the many language versions because they need to be translated instead of edited. Can you recommend some possibility how to repair that? Thank you -- sarang♥사랑 16:21, 22 September 2018 (UTC)

They are intentionally lower-cased by normalize_input_args (in Module:Autotranslate). I think no template should ever use all-upper-case parameters—minuscules have been invented on purpose, they are much easier to read. I don’t see the translation system a difficulty in this case: you have to correct the parameters only once, all the translations are automatically updated when a translation admin (like me) marks the page for translation. —Tacsipacsi (talk) 17:20, 22 September 2018 (UTC)

Thank you for swift answer. Now I have corrected the parmeters in Infobox flag/i18n which is AFAIK the source for all translations. Would you please now perform the automatic update? -- sarang♥사랑 17:45, 22 September 2018 (UTC)

 Done. —Tacsipacsi (talk) 18:15, 22 September 2018 (UTC)

Splitting Module:Fallback into Module:Autotranslate, Module:LangSwitch and Module:Fallback

Module:Fallback had several distinct heavily used functions. There were some issues with that: it was hard to track what template or module needs which functionality, also any changes to one of them triggered updates to pages that used other functions. To simplify the maintenance, I copied last June current code of two functions to Module:Autotranslate and Module:LangSwitch and redirected {{LangSwitch}} to use Module:LangSwitch. Now I did the same with {{Autotranslate}} and related templates, and they are all using Module:Autotranslate now. I believe I tracked down all the templates and modules that called Module:Fallback to access functions LangSwitch and autotranslate, so the next step will be removing of those functions from Module:Fallback, see Module:Fallback/sandbox2. --Jarekt (talk) 13:53, 8 November 2018 (UTC)

Unfortunately, you've used the Mediawiki builtin fallback list which is still not enough for Commons, and ignored completely the fact it has no support for "default" and lacks many fallbacks (exactly those discussed above where I also spoke about stricter BCP47 conformance, for which Mediawiki's fallback lists is also very defective).

This means that fallbacks no longer work as expected for Commons. Even if the change needed was not so complex to use the builtin list but still honor BCP47 rules and add correct handling of the optional "default" (before "en" which is must be added only at end, unless it is already explicitly added in the list before we add the "default").

This also means that we see English too much often where in fact there are accurately usable translations.

The MediaWiki builtin fallback list is fall of bugs/omissions, and evolves very slowly (probably because Mediawiki is not tuned just for Wikimedia projects, but still it should honor BCP47 more strictly by default for all wikis, including outwide Wikimedia). And the "default" options is also standard in BCP47 and CLDR (it is the equivalent to the CLDR "root" locale where applications will provide their defaults).

So yes there's a severe problem in MediaWiki's builtin fallback list which is completely defective (and not conforming to BCP47 at all: this is what my little change allowed to make this list more usable; for now the Mediawiki fblist is just a badly hacked list, not really evaluated, just built from limited tests from English Wikipedia with very few languages known by their authors, it just favors sites written by English-only speaking teams, but it complicates the tasks for all other sites where English is really undesirable and that should have another default, notably when the team works basically from a country where there's another national language and many regional variants that should fallback to it; and this also applies to ALL non-English wikis of Wikimedia, and that's why Commons also needs such fix: Commons must really be multilingual and serve all our wikis fairly).

In Commons we need the default for detecting cases where there's no other fallbacks available than English listed after it, to detect really missing translations. verdy_p (talk) 09:13, 21 April 2019 (UTC)