I spoke to Tim Starling about possibly adapting Special:Boardvote for use for Picture of the Year voting. We both came to the conclusion that it would probably be easier to install and use some other software that is better suited to the purpose. So if you know of any open source SW that can be used for multi-category approval voting (and possibly multiple rounds) with image thumbnails, randomised presentation and an option to leave a comment, please speak up. :)
And how about the verification of the voters? I don't know whether there exists any voting software exists that can be integrated with multiple Mediawiki wikis. -- Bryan (talk to me) 09:10, 8 August 2007 (UTC)
Well this is what I was thinking:
generate a list of accounts that meet the eligibility requirement, remove same username/diff wiki (i.e. with pfctdayelise@commonswiki and pfctdayelise@enwiki, remove one of them, or mark them as equivalent somehow)
get hashes for the password or something
get people to input their password and compare the hashes.
Or else we could do some 'token' trick like the flickrbot perhaps? pfctdayelise (说什么?) 13:02, 8 August 2007 (UTC)
I started writing software for this during the last one but haven't gotten it finished yet. If there is interest in using it I could go ahead and finish it. What I was implementing is software that displays two images with description text and asks the user to pick the better of the two, the pairs are randomly selected but the software makes an effort to equal views to each image, and a single user is never presented with the same pair twice. The data from this can then be used to compute rankings.
It was my hope that conducting the vote this way would be more friendly for users who don't have the patience to review lots of images at once, allowing us to have more candidates. It also should improve fairness because people will be less able to just call their friends to go vote for their image. This sort of vote would also eliminate the need to have separate votes for separate categories, the same preference data could be used to produce both the overall and the per category winners.
James F used a similar approach to make a manipulation resistant hot-or-not type voting system for another website.
I didn't solve the authentication issue yet. I could easily do an editcomment token, but I think this needs to be made as easy as possible. We had a lot of authentication issues last year. We could direct a watchlist notice to only those eligible, like the system, which would at least mean most of the people trying would be eligible to participate. w:OpenID against SUL would be ideal, but I doubt we'll be that lucky.--Gmaxwell 14:19, 8 August 2007 (UTC)
I don't like the model you suggested, because what if two crap images are presented? I want to pick the single one I think is the best, not just the best out of two randomly presented. I think rounds are better to reduce the numbers. We can promote the final round more heavily so that only super dedicated people use the earlier rounds. That should avoid voter fatigue. pfctdayelise (说什么?) 12:57, 9 August 2007 (UTC)
So really the final selection will only be picking out of the "super dedicate people's" favorites? I don't like that. ... If you get two crap images you pick the least crap, but those two crap images will each lose out whenever paired with anything else better, so the fact that they are crap will still be clear in the results. --Gmaxwell 07:49, 10 August 2007 (UTC)
The responsibilities of the organising committee should be to:
determine voter eligibility rules
determine image category eligibility rules
respond to queries about the voting process and assist voters on request
promote the competition among the wider Wikimedia community
announce the results, contact the winners
organise various archives
scrutinise votes and detect fraud, respond to allegations of voting fraud
try to ensure the competition is as 'fair' as possible and that all eligible entries are presented equally.
Members of the organising committee should not
have entries in the competition???.
Why not? The committee should be acting in a purely administrative role, and will have no discretion to grant or withold prizes. It's not a great idea to exclude all the FP regulars - many of whom will want to vote, and quite a few of whom will have at least one image in the competition. I'd consider getting involved, but not if I can't vote! --MichaelMaggs 17:11, 16 October 2007 (UTC)
All static media upoloaded between 01/01/2007 00:00 UTC and 31/12/2007 23:59 UTC ?
In theory yes. However I would like to shorten this year's eligibility time until the end of November. That would give us time to hold the competition and release a calendar. So really it would be Picture of the 11 Months Competition. Future ones can include the previous year's December. (i.e. time period for POTY08 would be Dec 07 + Jan-Nov 08.) pfctdayelise (说什么?) 13:14, 8 August 2007 (UTC)
I would strongly support limiting candidates to "own work" images made by commons contributors (or at least Wikimedia contributors, or at least derivations made from Wikimedia contriboturs). --SB_Johnny | PA! 12:58, 1 October 2007 (UTC)
Disagree. External images are eligible for FP status, and this competition should be open to them as well. --MichaelMaggs 17:12, 16 October 2007 (UTC)
Categories of voting
Overall winner, Picture of the Year (all images eligible - maybe this still needs to be done in rounds)
Wikimedian Picture of the Year (work created by a Wikimedian - needs rounds?)
Non-Wikimedian Picture of the Year (non-Wikimedia source - needs rounds?)
Encouragement Award (Quality Image of the Year - only QIs that are not also FPs are eligible here)
Most valuable picture of the year (for works that are scarce in free content, e.g. celebrity pics, non-Western society)
The bold 12 are the ones I think should go in the calendar. (Why not astronomy? Because it is likely to overlap with the non-Wikimedian cat.) pfctdayelise (说什么?) 12:10, 7 August 2007 (UTC)
How about a bucket "Rich media" category? Sounds/Video/Animation? Prior Wikimania awards have had categories for rich media. I honestly don't think yet have enough really good items in any one of those, but if we combine them? --Gmaxwell 14:26, 8 August 2007 (UTC)
Meh. I am ambivalent about it. I don't think we have enough good content in those categories either. If we do, then sure (and we need to find a way to promote it a hell of a lot better). pfctdayelise (说什么?) 15:38, 8 August 2007 (UTC)
Chicken and egg. We do have some very good audio recordings. Just not that many. I've viewed all of our videos (as of a couple months ago) and decided that we didn't have more than a couple that impressed me. Of course if we keep leaving these things out of our contests we won't get the added attention that they need. --Gmaxwell 16:01, 8 August 2007 (UTC)
Yeah but remember if its our best its still worth recognising, beside if people see this as an easy get, next year you'll have 100's and the quality would have improved. Maybe sound/video could get a section within QI as it's self made to further promote it. Gnangarra 11:31, 9 August 2007 (UTC)
I think playback is still a big problem. Nonetheless I changed 'Animation' to 'Rich media', and based on the talk page discussion removed the category 'historical' and replaced it in the calendar with 'objects'. pfctdayelise (说什么?) 12:59, 9 August 2007 (UTC)
I think that "The Picture of the Year" should be Wikimedian only. Also who decides on what image belongs in what category? Also "Most valuable picture of the year" seems very vague to me. Maybe we should do the following: Voting for the FP of the year and QI of the year go in two rounds, with a short discussion period in between them. The per-category competitions have one nominate/discuss round and [all]/[all images that receive a certain number of support] go to the final round. Opinions? -- Bryan (talk to me) 16:35, 7 October 2007 (UTC)
One vote per person, not one vote per eligible account
Allow edit combining?
Do we want cash prizes? --Gmaxwell 14:23, 8 August 2007 (UTC)
Maybe? :) Who knows what we could get? I have a huge pixel-art poster from eboy, by a weird circumstance in my life. I'd be willing to donate it as a prize. But approaching potental sponsors (camera makers???? can you imagine?!) should be a first approach probably. pfctdayelise (说什么?) 15:02, 8 August 2007 (UTC)
I'd prefer somethign substantual such as a camera or a poster or a Wikimedia Commons shirt or a package of stroopwafels. -- Bryan (talk to me) 15:10, 8 August 2007 (UTC)
latest thoughts, and LimeSurvey
Time is creeping up. These are my latest thoughts:
given that we don't really have enough time to do a proper trial with a pairwise system, and it would be a very unfamiliar and untested and new thing, for this year I think it makes more sense to stick with a familiar system.
Could have manually decided categories so that each category has 15-30 images in it. Top two in each category go to a final round, where the single winner is Picture of the Year. Highest-rated picture by a Wikimedian is Wikimedian POTY (could be the same image, no separate category).
I had a look at existing voting-style software packages and the best by far I have found is LimeSurvey. I set up a demo so whoever is interested can log in and explore:
For the admin stuff, choose a survey on the right (so far there is only one :)). Then choose the group 'everyone' (this seems to be a mandatory but useless step, for us at least). Then you will be able to edit the single question in this survey. This survey allows anonymous voting at the moment but you can easily make surveys that only let people with tokens take the survey.
So I think we could host the image galleries for each voting category here on Commons, then in the survey just refer to the image name, or a code for it like "FP001" (if names could be prejudicial).
So, problems that remain:
authentification. a problem with any solution. I was thinking we could make a toolserver thing that people apply to, it checks if they have suffrage, and if so uses special:emailuser to email them a voting token. probably the trickiest bit to figure out... LimeSurvey has this option for tokens: Import from LDAP: Allows you to import information from an LDAP query (tested on openLdap but should work in any LDAP compliant directory including ActiveDirectory)... don't know if this is continuously updated or not?
need for public voter logs (not record of votes, just voters) - probably not too hard
limit votes to 5 per category? - if we switch to approval voting then LimeSurvey will handle it. I'm not sure if there was a compelling reason for only allowing 5 votes in the first round, anyway?
I'm making and testing some auth and voting stuff. Those parts are quite easy. The only probalem is what kind of voting we use, and whether we want to use external software and shiny features or not. I'll leave that up to the rest of the people. LimeSurvey is open source, so no doubt I can hack something together with it. -- Bryan (talk to me) 20:05, 1 October 2007 (UTC)
Well, LimeSurvey is also already set up for i18n, which is another bonus. Also letting people change their vote, I think. If you think you can hack something together... can you show us within a week or so? :) pfctdayelise (说什么?) 11:37, 2 October 2007 (UTC)
I looked into some more detail, and see that there are two flaws in the software. As far as I can find, it is not possible to change your vote. Also there is no way I could find to have the images added to the questions, which is a minor detail. Bryan 11:00, 3 October 2007 (UTC)
That's why I figured we could have galleries on Commons, and just filenames (or codes) in the survey.
As for changing your vote, allowing it last time was a bit of nightmare administratively speaking anyway. Maybe we could just get people to contact the organising committee by email/wiki if they want to change their vote, and we could manually delete their old vote and arrange a new token. Would that be possible? pfctdayelise (说什么?) 11:02, 4 October 2007 (UTC)
I've found some SQL injection points in LimeSurvey, of which luckily only one is usable by malicious users. I'll be digging through the code to find more before I can do more work on this, so it may take a little longer. -- Bryan (talk to me) 14:30, 5 October 2007 (UTC)