Camera Club - Audience Response Systems (Audience Voting)

S

Simes123

Messages
472
Edit My Images
No
I've been asked to investigate an Audience based judging system for our Photographic Society. We have over 100 members regularly attending, and as an augmentation to traditional judging, it would be nice to have events where the audience scores images, rather than an inconsistent, single fuddy duddy judge.

Anyone used such a system in their club that works consistently, is accessible for older members and so on?

I'm thinking cloud based would be lowest risk and cost, but that brings it's own accessibility issues, so maybe a clicker type hardware based system too.

It needs to be simple, both in terms of use/operation and also in terms of back end prep of the images. Don't want to have to create powerpoint of images etc.

Grateful for any experiences/thoughts,

Simon
 
I don't know of any judging systems but if you just wanted a simple poll system you could set up one with a free online poll creator;

easypolls
doodle

which create a poll then give you a weblink to share which could be accessed on smart phones or maybe a laptop set up for those without smart phones.

You wouldn't be able to photos up but people could at least vote for their favourite of the night.
 
Hi Thanks Steve - yes those would work for choosing a favourite - the main requirement here is to enable scoring - say out of 20 for each image as they were shown up one by one, with the results revealed at the end. One of the biggest complaints we have is that a single judge can be somewhat inconsistent. Some are great, and others give the impression that they've barely looked at the images and have been arbitrary in their scoring. While we probably won't move away from that entirely, we see the basis of a new kind of competition where the audience judges, Salon style, and you get a nice averaging effect from over 100 different photographic opinions to give a good basis for internal club leagues and so on.
 
We did this at Maidenhead Camera Club a few years ago. Our club has about 100 members and we usually get 50 to 80 at any given meeting, so it's comparable to yours in size and our experience is probably relevant.

The software we used is called Socrative. There are two apps: Socrative Teacher which the organiser uses to set up the polls, and Socrative Student which the participants use.

It's intended as an educational tool for teachers to set quizzes and tests for their students, hence the naming of the apps. Some of the terminology it uses is based on that, so for example students log in to join a particular "classroom", etc. But that doesn't get in the way and it did the job for us.

However it is perhaps interesting to note that, despite the fact that it basically worked, we haven't used it again since the experiment in 2014. I don't think we ever had a thorough assessment as to why, but subjectively I would think:

1. It takes a lot of preparation. The organiser needs to do a reasonably sized private test, using a mixture of Android and Apple devices, to check how the system works in practice before using it in anger.

2. It takes a lot of communication. Every single participant must install the app before the session. So the organiser must communicate that requirement, and provide detailed instructions, and make sure people read and understand and follow them. That's a challenge in itself. How many of us see emails in our inbox entitled "Next week at camera club" but don't bother reading them because we know we're going anyway? How many of us see "Camera club voting - important - please read!" but don't bother reading it because it sounds like it is neither interesting nor important? Do not underestimate this aspect of the process.

3. It totally alienates people who don't have smartphones, or who aren't comfortable loading new apps onto their smartphones, or who forgot to charge their smartphones, or have Windows phones.

4. It's difficult to manage real-time problems. For example you're displaying image 19, but one person's app says it's image 17, so apparently a couple of scores they thought they had entered weren't registered, so now a load of their scores are wrong and how do they fix it?

I'm not saying don't do it. But don't think it will be easy!

Hope this helps. Happy to discuss.
 
We did this at the last camera club I was a member of. All done manually. No electronics at all.
Every picture was given a number (we used numbered paper corners but there are other methods) and they were displayed in the meeting hall. All members were given a pre printed voting slip to state their 1st, 2nd & 3rd choices. Everything anonymous, of course.
Then after half time two "volunteers" simply went into a corner and counted up the votes. It was easy to do and AFAIK is still being used.
You don't get the critique, but then most judges don't give any - they just tell you what they like, or more often don't like, about the picture.
This method gives a cross section of photograhers views on which they think is the best picture. When the winners are announced the togs are invited to say a few words and answer a few questions.

Very low tech, but it works.
 
Feedback from many individuals can be far more useful than feedback from just one. It's also easy to strike off the views of 1 judge as simply "they're blind and therefore rubbish" if they don't like your image. With lots of votes (or lack of) it's much harder to slip into denial.

I teach evening classes in photography and regularly use the @mickledore approach. I also (even quicker) get people to vote with hands-up-eyes-closed and do a rough count. With over 100 though that might be tricky :)

Also - even though it's anonymised, I ask if anyone would like to comment on their vote (once it's cast) allowing people with strong opinions to voice them. This usually generates discussion from those who strongly agree/disagree which is always of huge benefit. Again, this is unlikely to be practical with over 100 people, although you could pick the top 3 for group discussion.

Finally - as Stuart said, never underestimate the lack of technical skills of your audience. Some may not even be able to use a computer! (I've had a few DSLR photographers who keep their images on cards and just buy a new one when it fills up) As soon as you implement a technical, whizzy way of doing things, you'll lose the interest of some - especially if it doesn't work flawlessly, and especially if they need equipment they don't have (smartphone, computer, internet access)
 
mickledore said:
We did this at the last camera club I was a member of. All done manually. No electronics at all.
Every picture was given a number (we used numbered paper corners but there are other methods) and they were displayed in the meeting hall. All members were given a pre printed voting slip to state their 1st, 2nd & 3rd choices. ... Very low tech, but it works.
Click to expand...
Definitely that could work if the images being judged were prints, and you had space to display them all with good lighting. Obviously a 1st-2nd-3rd approach isn't viable if he images are projected sequentially.

I would be concerned that it wouldn't scale very well though. At our club we might typically have 60 prints in the competition and 60 people voting, and I think the OP's club is even bigger. That would require an awful lot of space for people to mill around in, even if you had space to display the pictures.
 
Simes123 said:
... the main requirement here is to enable scoring - say out of 20 for each image as they were shown up one by one, with the results revealed at the end.
Click to expand...
In a big club, that is really going to require some sort of automated solution. We've tried doing it manually and it's really hard. Back to my example of 60 people each scoring 60 images. That's 3600 scores to process. I'm exceptionally proficient at mental arithmetic, in that I can add up a column of numbers faster than I can type them, but even so it would take me probably 30 minutes to do it all. If you had half a dozen people all inputting scores into a spreadsheet template, with someone then combining all the separate spreadsheets to calculate the results, you might cut it to 20 minutes. But that's an incredibly boring 20 minutes for everybody else.
 
StewartR said:
Definitely that could work if the images being judged were prints, and you had space to display them all with good lighting. Obviously a 1st-2nd-3rd approach isn't viable if he images are projected sequentially.
Click to expand...
Ah, but it did work!
There was always a separate digital competition. Images were arranged in numerical order and they were given - I think - three run throughs. (Or should that be runs through? )
TBH that was PIA. By the third run through everyone was getting itchy bums!!!
 
StewartR said:
In a big club, that is really going to require some sort of automated solution. We've tried doing it manually and it's really hard. Back to my example of 60 people each scoring 60 images. That's 3600 scores to process. I'm exceptionally proficient at mental arithmetic, in that I can add up a column of numbers faster than I can type them, but even so it would take me probably 30 minutes to do it all. If you had half a dozen people all inputting scores into a spreadsheet template, with someone then combining all the separate spreadsheets to calculate the results, you might cut it to 20 minutes. But that's an incredibly boring 20 minutes for everybody else.
Click to expand...
But why does every picture have to have a score out of 20 from every member present?

The only logical answer to that question is "Because we always do it that way".

You want a winner. Find a simple method. It matters not one jot if someone got 2 or 3 more votes than the next guy if they are both losers.
 
mickledore said:
But why does every picture have to have a score out of 20 from every member present?

The only logical answer to that question is "Because we always do it that way".
Click to expand...
Not true. There is another logical answer: "Because we have experimented with various methods over the years and we have actively canvassed members as to which they prefer, and this is it."
mickledore said:
You want a winner. Find a simple method. It matters not one jot if someone got 2 or 3 more votes than the next guy if they are both losers.
Click to expand...
Again, not true. Again, we have experimented with various approaches and people do care about more than just 1st/2nd/3rd. They want to know whether they are improving, and in a big club a string of top-10 finishes (for example) is pretty good even if you don't get any wins.

YMMV.
 
Ah. The camera club member.
Our system is the best because we have been using it since the days of Fox Talbot.
Change? Not while I'm on the committee.
 
mickledore said:
Ah. The camera club member.
Our system is the best because we have been using it since the days of Fox Talbot.
Change? Not while I'm on the committee.
Click to expand...
That might be what your club is like. As I explained, we've experimented with various approaches. In the 11 years I've been a member, I think the structure and approach to competitions has changed about 5 times. The system we currently use is the one that was preferred by members last time we reviewed it. It may well change again.
 
Some great feedback - thanks.

It's a big club - early in the season, up to 120 people present. Average age about 65... I like the simplicity of a paper based solution as a differentiator for 1st, 2nd, 3rd only style voting. But we often have around 120 images being reviewed by a judge and I imagined a concurrent process whereby the user score would also be given to each image, and an automated solution would reveal the users 1st, 2nd, 3rd at the end, and individual totals provided later to users so they could assess their relative performance in the club. Which would serve to remove the inherent single person's opinion we currently get (albeit from a qualified judge). I've looked at some educational style systems, but setup and management aren't really geared up to image scoring en masse.

I've been chatting about this with some club members, and already I'm seeing resistance (to change), suggestions that peer bias would be prevalent (some people's images are very recognisable), and I received one feedback from a qualified judge who said he would never judge in a club that operated such a system. He wouldn't want his carefully considered judging to be up against a people's vote. Though one or two members think it's a great idea.

I think the accessibility of the system would be key. A simple clicker system that didn't rely on smart phones would probably be ideal. But then you've got 120+ devices to be handed out, collected in, batteries to be maintained, technical issues, cost of those devices up front, replacement devices and so on. If we went the smartphone route, the average age of the audience would be an issue I'm sure. Judges may hate it and so on. I'm beginning to think it's a terrible idea! I think I might just keep my head down and forget I ever mentioned it!

After much resistance to change, the club are changing it's scoring system this coming season to a pointless system whereby more Highly Commended and Commendeds are given out, and the rest aren't scored, only critiqued. While I think this is an improvement in some regards, for those who are middling, they never get to know if they are getting better relative to other members. And placings will still be based on the whims of an often older judge. I found the club competition to be both motivational in my first season, and hugely demotivational in my last, to the point I gave up photography for 6 months. I'm not planning on entering the competitions next season as a result and will be shooting purely for myself, but I remain keen to find a mechanism by which a shift to a peer based rating system can be gradually introduced - perhaps only on one or two nights in a season where we had a members judging night, or perhaps a website based competition. One thing I've learned though, is change has to be very carefully introduced in a camera club!!!
 
Simes123 said:
change has to be very carefully introduced in a camera club!!!
Click to expand...

I hope some of the old school rules never change.

Thou shalt not take a photo of people stuffing their face.
Thou shalt not take a photo of tramps.
Etc, etc, etc....
 
Simes123 said:
After much resistance to change, the club are changing it's scoring system this coming season to a pointless system whereby more Highly Commended and Commendeds are given out, and the rest aren't scored, only critiqued. While I think this is an improvement in some regards, for those who are middling, they never get to know if they are getting better relative to other members.
Click to expand...
That was one of the things we've tried. It wasn't at all popular. The main reason was the one which you mentioned, and I mentioned earlier: people who aren't getting the awards - which is most people, in a big club - have no idea whether they are improving. An important secondary reason, however, was that many judges weren't comfortable with it. I think they're often more resistant to change than club members!
 
Our club is not as big as yours, 40 on a good night. We only use this system on projected images.

Each member gets a voting paper with the images numbered 1 - xxx. The images are projected in numerical order and each member marks them 1-5. The higher the mark the more you like the image. Then, once all the images have been seen, each member chooses their top three images and marks them 1st, 2nd, 3rd accordingly. In the tea break two or three of the committee add up all the 1st, 2nd and 3rds and a result is declared.

We usually run through the images 3 times.

We used to add up all the scores and give every member a place but this became too time consuming so we dropped it in favour of the 1,2,3 approach.

We use this maybe 3/4 times in the club season. Most of our comps are judged by external judges. The member voted comps don’t count for our Photographer Of The Year award.
 
Not at a camera club, but I've done similar with Google Forms for an audience of 500+

You can even prep the form, with one page per pic and a thumbnail of each picture - results available instantly in Google sheets, but the one caveat is they have to compete the form (ie score all pictures and hit submit) for their entry to be registered. Score can be anything from a simple yes / no to a sliding scale 1-20 etc, even score multiple factors per pic.

No need to install anything, just go to a simple URL to complete the form. No need to sign in either if you don't think it's necessary etc.

For Example

https://goo.gl/forms/kGql6yYSG4bNJTbq2

Obviously you can do a lot more with formatting, split one page per photo etc as you see fit and use a better URL shortener of you want something they can type in (or use a QR code on the main screen etc).
 
Last edited:
Furtim said:
Not at a camera club, but I've done similar with Google Forms for an audience of 500+ ....
Click to expand...
That's a clever solution. However it does require absolutely everyone to have a laptop / tablet / smartphone, so for that reason it's probably not going to be too attractive at a camera club. (Perhaps surprisingly, it's hard to overstate the level of techno-FUD that some members will have.)
 
All these electronic gizmos could have fatal repercussions at your average camera club if they interfere with pacemakers.
Feedback in hearing aids could be nasty too.
 
Last edited:
There is the, lottery style, pencil/card reader systems, where everyone is given a pencil and rubber and a card with lots of squares. You mark a pencil cross on the square with the score for each picture. These are read in at the end and the scores all instantly tallied up. Not too tricky for the oldies. But you'll need a 70's mainframe with a card reader handy. And a boffin in a white coat. Either that or you scan them with a special app on a smartphone.
scantron.jpg


What can go wrong?
 
Last edited:
ianp5a said:
There is the, lottery style, pencil/card reader systems, where everyone is given a pencil and rubber and a card with lots of squares. You mark a pencil cross on the square with the score for each picture. These are read in at the end and the scores all instantly tallied up. Not too tricky for the oldies. But you'll need a 70's mainframe with a card reader handy. And a boffin in a white coat. Either that or you scan them with a special app on a smartphone.
scantron.jpg


What can go wrong?
Click to expand...

They might wake up at the end of the presentation, see the paper/pencil and shout out HOUSE
 
You must log in or register to reply here.
Back
Top