Some great feedback - thanks.
It's a big club - early in the season, up to 120 people present. Average age about 65... I like the simplicity of a paper based solution as a differentiator for 1st, 2nd, 3rd only style voting. But we often have around 120 images being reviewed by a judge and I imagined a concurrent process whereby the user score would also be given to each image, and an automated solution would reveal the users 1st, 2nd, 3rd at the end, and individual totals provided later to users so they could assess their relative performance in the club. Which would serve to remove the inherent single person's opinion we currently get (albeit from a qualified judge). I've looked at some educational style systems, but setup and management aren't really geared up to image scoring en masse.
I've been chatting about this with some club members, and already I'm seeing resistance (to change), suggestions that peer bias would be prevalent (some people's images are very recognisable), and I received one feedback from a qualified judge who said he would never judge in a club that operated such a system. He wouldn't want his carefully considered judging to be up against a people's vote. Though one or two members think it's a great idea.
I think the accessibility of the system would be key. A simple clicker system that didn't rely on smart phones would probably be ideal. But then you've got 120+ devices to be handed out, collected in, batteries to be maintained, technical issues, cost of those devices up front, replacement devices and so on. If we went the smartphone route, the average age of the audience would be an issue I'm sure. Judges may hate it and so on. I'm beginning to think it's a terrible idea! I think I might just keep my head down and forget I ever mentioned it!
After much resistance to change, the club are changing it's scoring system this coming season to a pointless system whereby more Highly Commended and Commendeds are given out, and the rest aren't scored, only critiqued. While I think this is an improvement in some regards, for those who are middling, they never get to know if they are getting better relative to other members. And placings will still be based on the whims of an often older judge. I found the club competition to be both motivational in my first season, and hugely demotivational in my last, to the point I gave up photography for 6 months. I'm not planning on entering the competitions next season as a result and will be shooting purely for myself, but I remain keen to find a mechanism by which a shift to a peer based rating system can be gradually introduced - perhaps only on one or two nights in a season where we had a members judging night, or perhaps a website based competition. One thing I've learned though, is change has to be very carefully introduced in a camera club!!!