User talk:Dispenser

From Wikimedia Commons, the free media repository
Jump to: navigation, search
Welcome to Wikimedia Commons, Dispenser!
Afrikaans | Alemannisch | العربية | Asturianu | Azərbaycanca | Беларуская (тарашкевіца)‎ | Български | বাংলা | Català | Čeština | Dansk | Deutsch | Deutsch (Sie-Form)‎ | Zazaki | Ελληνικά | English | Esperanto | Español | Euskara | Estremeñu | فارسی | Suomi | Français | Frysk | Galego | עברית | हिन्दी | Hrvatski | Magyar | Հայերեն | Interlingua | Bahasa Indonesia | Italiano | 日本語 | ქართული | 한국어 | Latina | Lietuvių | Македонски | മലയാളം | मराठी | Bahasa Melayu | Plattdüütsch | नेपाली | Nederlands | Norsk bokmål | Occitan | Polski | Português | Português do Brasil | Română | Русский | Scots | Slovenčina | Slovenščina | Shqip | Српски / srpski | Svenska | Kiswahili | தமிழ் | ไทย | Türkçe | Українська | Vèneto | Tiếng Việt | 中文 | 中文(简体)‎ | 中文(繁體)‎ | 中文(台灣)‎ | +/−

Giggy (talk) 01:35, 6 November 2008 (UTC)


Ships by IMO number[edit]

Hi Dispenser,

For categories in Category:Ships by IMO number, it would be interesting to have a page that links to various websites about ships that can be queried by IMO number, similar to booksources or the coordinates page. I made a request for a page similar to booksources in bugzilla, but it seems that it's unlikely to happen. Thus I was wondering if you would set one up on toolserver. --  Docu  at 07:43, 16 June 2012 (UTC)

As you may know, I don't particularly like GeoHack. Visitors are confused by the copious links, many of them second rate, and only use five out all the links. This mean the reset of the page is a waste. And then there's an ideological argument to prompt or limit links that meet a FLOSS definition.
Back to the topic, possibly the nicest way to implement this in JavaScript using &withjs= on the links. This would make editing, adding features, and maintenance easier and more reliable. Plus, it'll be implemented a lot sooner. You might need to pass the script off as a gadget to get it into the MediaWiki namespace. —Dispenser (talk) 03:22, 19 June 2012 (UTC)
Sounds good. Is there a sample I could build on? I checked Special:Preferences#mw-prefsection-gadgets, but there seems nothing similar. --  Docu  at 05:24, 23 June 2012 (UTC)
/* [[Commons:IMO sources]]
 * Quick and dirty for generic sources pages
 */
$(function() {
    // Retrieve and escape the replacement for MAGICNUMBER
    var mn_replacement = mw.util.getParamValue("imon");
    if(!mn_replacement) {
        // Null or blank value
        return;
    }
    if(!mn_replacement.match(/^\d+$/)) {
        // Invalid value
        return;
    }
    // Replace HTML with magic number text
    $("#content")[0].innerHTML = $("#content")[0].innerHTML.replace(/MAGICNUMBER/g, mw.html.escape(mn_replacement));
    // Remove the no JavaScript notice
    $("#nojs-notice").remove();
});
Drop that in MediaWiki:IMO_magic.js and open the specially crafted link. Dispenser (talk) 21:11, 23 June 2012 (UTC)
Excellent. Looks very compact. I made the corresponding edit request at MediaWiki talk:IMO magic.js. --  Docu  at 06:48, 24 June 2012 (UTC)

Mediawiki error lists[edit]

Could you please create a pair of subpages and give the full lists of effected files for the errors you brought to the village pump? Thanks! Sven Manguard Wha? 15:27, 1 September 2012 (UTC)

All files with problems that I know of have been listed on the wiki. Unless you're referring to Image scaler problems which should be done within two weeks. Or the secret report on frozen GIFs. —Dispenser (talk) 22:52, 1 September 2012 (UTC)

Update[edit]

Could you please update the list if this is no big deal. (I cleared some percent of all files and they could be deleted from this list "automatically"). --McZusatz (talk) 17:32, 7 September 2012 (UTC)

Done. —19:05, 7 September 2012 (UTC)

404 error at File:The Naval Temple, at The Kymin, Monmouth 3.jpg?[edit]

Hi Dispenser. What happened here? The image was working fine before, and File:The Naval Temple, at The Kymin, Monmouth 2.jpg was uploaded in the same minute yet doesn't exhibit any problem? Thanks. Mike Peel (talk) 19:15, 9 September 2012 (UTC)

I don't know. Likely an issue retrieving with the new storage engine (considering the horrible hack for "long" file name). Anyway, according to Bug 39615 WMFer User:Aaron Schulz is working to fix this. Dispenser (talk) 03:33, 21 November 2012 (UTC)

Broken Videos[edit]

No. Error type
3 File:Dmitry Volodikhin.ogg
Exitcode: 1
2970   av_interleaved_write_frame()
33   Error while opening encoder for output stream
11   could not find codec parameters
3   Audio encoding failed
4   Unable to decode input.
2   Output file #0 does not contain any stream
10 Exitcode: 134
1 Exitcode: 136
34 Exitcode: 137
5 Exitcode: 139
3076 TOTAL

Do you know how to find all "broken" videos such as this one? --McZusatz (talk) 17:30, 20 November 2012 (UTC)

I don't have too much time for investigations this month. I'd try asking on IRC (#wikimedia-tech) if someone could just grep the database. Or with Commons having only 25,208 videos it would take ~8 hours to download each description page and check for class="errorlink". Whichever way, the transcoding queue is probably backed up for the next month and the complete error report with it. —Dispenser (talk) 04:46, 21 November 2012 (UTC)
Thanks for the help so far. What do you think is the best method to download each description page? --McZusatz (talk) 12:53, 22 November 2012 (UTC)
jira:TS-1570. —Dispenser (talk) 21:31, 23 November 2012 (UTC)
Is it likely to get a response? --McZusatz (talk) 13:55, 30 November 2012 (UTC)
I poked DaB on IRC and he quickly added it. Sorry about being slow in posting a report. I left 1,655 av_interleaved_write_frame() affected files (11%) as it might be an implementation issue. A few interesting tid bits: the software upscales 240p ogg to 360p webm, the transcoding timestamps can be used for load and CPU time graphs, 26,969 video files. —Dispenser (talk) 07:42, 17 December 2012 (UTC)
Thank you for the list. I will have a look at the stuff soon. --McZusatz (talk) 09:09, 17 December 2012 (UTC)
Btw. I do not think the software upscales 240p ogg to 360p webm. (See: https://upload.wikimedia.org/wikipedia/commons/thumb/1/14/Tmh.rev.test.ogv/Tmh.rev.test.ogv.360p.webm ) --McZusatz (talk) 22:16, 17 December 2012 (UTC)

Could you please do another update of your sandbox. I cleaned a lot of the files. --McZusatz (talk) 14:15, 24 May 2013 (UTC)

Done, related bug 46650. Dispenser (talk) 04:23, 26 May 2013 (UTC)
Thanks again. Jan Gerber requested the "av_interleaved_write_frame()"-errors to be reset-transcoded and I fixed the remaining files in your sandbox. I think it would be appropriate to update the list from time to time. Maybe every two weeks... What do you think? --McZusatz (talk) 14:44, 3 June 2013 (UTC)
It doesn't appear the extension author wants to acknowledge let alone address the transcoding issue. So I've moved the report to User:Dispenser/Transcode errors. The av_interleaved_write_frame() errors are still appearing despite bug 46795 claiming it was fixed. I usually run report whenever I feel like it, but I probably should implement something similar to Commons:Database reports. —Dispenser (talk) 15:06, 5 June 2013 (UTC)

Mismatched extensions[edit]

Rillke did some great progress on the cleanup. Could you please update the list? --McZusatz (talk) 11:47, 18 December 2012 (UTC)

Done Dispenser (talk) 18:29, 18 December 2012 (UTC)

Autopatrol given[edit]

Commons Autopatrolled.svg

Hello. I just wanted to let you know that I have granted autopatrol rights to your account; the reason for this is that I believe you are sufficiently trustworthy and experienced to have your contributions automatically sighted. This will have no effect on your editing, and is simply intended to help users watching Recent changes or Recent uploads to find unproductive edits amidst the productive ones. Thank you. --McZusatz (talk) 14:09, 26 January 2013 (UTC)


Coordinates log[edit]

Hi Dispenser,

Long time since I looked into this. It seems that finally someone else is working on them, see Template_talk:Location#10k_files_in_Category:Pages_with_malformed_coordinate_tags

BTW, the report at https://toolserver.org/~dispenser/logs/coord-commonswiki.log worked on the 26, but displays an error message today. Maybe it just needs re-starting. --  Docu  at 11:14, 1 March 2013 (UTC)

User:Dispenser/Transcode errors[edit]

Kannst du die Liste bitte noch einmal aktualisieren? --McZusatz (talk) 19:11, 24 August 2013 (UTC)

Danke! Ich habe heute alle Videos repariert, soweit ich konnte. Falls du Lust und Zeit hast, würde ich mich über regelmäßige updates freuen. :) --McZusatz (talk) 17:33, 25 August 2013 (UTC)

User:Dispenser/Absurd overhead[edit]

This list looks really amazing. How did you create it? It seems for me that a lot of CPU cycles were consumed to achive this. --McZusatz (talk) 15:10, 27 August 2013 (UTC)

It works similar to GIF check, except a python script downloads the images and runs zip -9, jpegtran -copy all, and unrar l before printing the results. It takes about 4 hours to run the first time. I'll publish the source code later. We still need to figure out what's bloating the other files. —Dispenser (talk) 16:17, 27 August 2013 (UTC)
Some of the use weird compression but are likely not harmful. (File:Nevtor Oy - Logo (2012).jpg)
 Exif:
 Compression scheme     Uncompressed
 Data arrangement       chunky format
 Color space    Uncalibrated
--McZusatz (talk) 13:23, 28 August 2013 (UTC)
The "harmful" ones likely have 0% zip compression and -80% to -100% jpegtran. Now a 100x26 image should be (100 wide)(26 tall)(3 bytes/pixel) = 7.6 KB uncompressed true-color bitmap. The other 560 KB appears to be data mostly unrelated to the image. That it was created with Adobe Photoshop CS3 Macintosh in 2009, Adobe XMP and Apple Property List XML junk, junk data repeated a few times, and lots of 0xFF bytes. Still a work in progress and nothing finished. Dispenser (talk) 20:10, 28 August 2013 (UTC)
So the half a megabyte of size is due to the embedded color profile, which is also copied to the smallest thumbnails. Color space support is bad outside of photography applications, even with the introduction of wide color gamut OLED phone displays. The image scalers should convert to sRGB below a certain pixel size like 64x64 (We'll need a bug for that). I've also got the report back working (didn't actually need unique img_name→page_id mapping), so expect an update soonish. Dispenser (talk) 16:43, 26 September 2013 (UTC)
Color profiles are important sometimes... File:Jigsaw-PNL.jpgDispenser (talk) 18:44, 29 September 2013 (UTC)
Yes, indeed. The automatic detection of the color profile failed sometimes. All instances should be fixed by now. --McZusatz (talk) 16:40, 30 September 2013 (UTC)
Since color profiles are copied to thumbnails I'm now recording file size in my thumbnail request script. Color profile estimates are now available for 2.1 million images larger than 48x48. This has reduced the false positives.
I've obtained a copy of Fireworks begun tagging images with multiple layers. However, I'm unsure what to do with those using it as an export tool (single layer, same as the output). --Dispenser (talk) 16:16, 16 May 2014 (UTC)

On the most recent version: What does the MDA mean? --McZusatz (talk) 19:13, 16 May 2014 (UTC)

Metadata Attributes (took a while to recall that, introduced in September). A count of metadata values extracted by MediaWiki in hopes of improving pre-selection, but MW isn't keeping it updated so likely it'll be dropped. —Dispenser (talk) 20:13, 16 May 2014 (UTC)

Uncompressed PNG[edit]

Some programs and/or users save PNG files in a uncompressed way. (BMP-like compression). I would appreciate it, if you forged a query to find all PNGs which loose at least ~20% file size by recompression. --McZusatz (talk) 18:21, 23 September 2013 (UTC)

I've partially integrated it into the Absurd overhead, but I've limited to minimum of 750 KiB of estimated overhead until we better understand what data we're throwing away. —Dispenser (talk) 04:50, 1 October 2013 (UTC)

Do you know TweakPNG? You can use this tool to look into suspicious PNG files. From what I see none of these files is harmful or dangerous. They just contain some unused chunks. For example, this file was made with Adobe Fireworks and contains all information needed to recreate the image with Adobe Fireworks (which can be good and should not be dropped). A PNG optimizer drops this stuff. Our image scaler does the same. All non-original image sizes don't contain these chunks. Therefor it's pretty much a waste of time to optimize these images. Very few users are downloading the original image size anyway. --TMg 13:54, 4 October 2013 (UTC)

Looks useful. Too few Linux ports of these PNG utilities :-(. I've created {{Fireworks PNG}} to identify Fireworks files and will excluded them from the report. It's up to the community to implement this (debate on propriety formats/FLOSS religion and should be had at Commons:Village pump#Compression). To make it easier, I've already noted Fireworks files on the list. Also, the source file is sent when thumbnail width ≥ source file, which is happening to 2,000+ PNGs. —Dispenser (talk) 05:06, 5 October 2013 (UTC)
Did you saw the discussion about progressive JPEG files? It was the more or less same: A user tried to convert all these files to non-progressive. He was stopped because what he did wasted more bandwidth, space and CPU power than it would have saved in the next 100 years. I think it's the same here. Don't waste your time with this, please. Don't "fix" files that aren't broken. --TMg 22:21, 8 October 2013 (UTC)

TIFF check?[edit]

Over 50 megapixel
Type Files
JPEG 13,849
SVG 1701
TIFF 316
PNG 237
DjVu 208
PDF 62
GIF 16

Hi Dispenser. This this is similar to your current GIF check, so I was wondering if you would be able to produce a report of TIFFs that fail to render previews because the file is above the current megapixel limit ($wgMaxImageArea is currently set to 50MP). This is affecting some of my uploads, and I'd like to see how widespread the issue is. Thanks! Dominic (talk)

Right now my thumbnail caching script reports 173 TIFFs give the The resolution of the source file is too large. No thumbnail will be generated error, with another 5 giving no error message at all. PNG and TIFF files currently ignore $wgMaxImageArea. The PNG limit has been lifted, although I still have trouble with File:WorldMap-A non-Frame.png. Tomorrow, I'll have a proper query. —Dispenser (talk) 06:32, 28 September 2013 (UTC)
filed bugzilla:54744 for the PNG file. --McZusatz (talk) 19:12, 28 September 2013 (UTC)
The caching script may have incorrectly marked some files as thumbnailable, I'll retry those above 50 MP. Do you need the file list or is a summary sufficient? Dispenser (talk) 18:05, 29 September 2013 (UTC)
I've posted the query to Category talk:TIF files affected by MediaWiki restrictions and User:Fae has tagged the remaining images. —Dispenser (talk) 04:34, 1 October 2013 (UTC)

DjVu files from archive.org[edit]

Hello. Because of [Wikisource-l] archive.org switched to ABBYY FineReader 9.0, I'd like to have a list of all DjVu files on Commons (but also Wikisources if one wanted) which link to an archive.org item as their source, ideally split in two lists: 1) DjVu files used on some Wikisource, but with some Page: pages missing (this means the old text was not yet completely copied to wiki pages), 2) DjVu files not yet used on any Wikisource. Can you help with this? If I had a list of item,filename pairs I could rerun the OCR on archive.org and then update our copy of the files. --Nemo 07:45, 8 October 2013 (UTC)

To answer your question, yes, https://archive.org/details/VocabolarioDellaLinguaItaliana2 is the pattern I'm looking for. --Nemo 16:11, 9 October 2013 (UTC)

User:Dispenser/Transcode errors[edit]

Do you know why File:Wikimania 2011 - Statistical analysis and MediaWiki development.ogv does not appear in the list? --McZusatz (talk) 22:31, 11 October 2013 (UTC)

I had it set to ignore empty errors. Its now fixed and I've reset the transcodes for the affected images. —Dispenser (talk) 06:18, 12 October 2013 (UTC)

About a tool[edit]

Hi, just wondering about something, sorry if this isn't the ideal place to ask; it seemed like this is somewhere you're relatively active.

There's a tool over on toolserver, embeddedincount, that looks like it was yours. It's used in the {{high-risk}} templates on en.wn and en.wb. We've been switching such links over from toolserver to wmflabs, of course, but as best I can tell, that tool doesn't have a wmflabs equivalent. Do you know of an equivalent that exists, or that you or anyone else plans to create? --Pi zero (talk) 20:20, 3 May 2014 (UTC)

I asked the Foundation to allocate 24 Terabytes of non-backuped storage for tool improvements and they refused. The WMF made clear it does not value me nor the community. More concerned with soliciting donations, apparently. --Dispenser (talk) 16:55, 16 May 2014 (UTC)
As a Wikinewsie, I can appreciate bitterness about WMF priorities — since they've been leaving Wikinews out in the cold (seemingly hoping it would die of exposure) for years. They apparently don't realize that, just because they don't believe in the importance of what we do, doens't mean we don't believe in it so deeply we're willing to keep going despite them. (The same goes, come to think, for the social atmosphere at Wikipedia, which has turned rancid since I first arrived there but the project continues because lots of great people care about the work enough to do it anyway.) --Pi zero (talk) 13:28, 17 May 2014 (UTC)