Could artificial intelligence make awards judging fairer in the future? Introducing Pearl, the first AI jury experiment. To celebrate its 10th anniversary, the Belgian MIXX awards has asked the first artificial intelligence jury president, called Pearl, to designate the winner of a special award during this year’s MIXX Award gala. With Pearl, the organizers want to open a debate about the future of award judging, particularly in light of some recent criticism. Is it possible that AI could make judging fairer? Could it help make creativity measurable? And how might it complement human juries?
While award shows normally just look at the future of advertising, this
time the MIXX awards have decided to look at the future of judging. This
first AI experiment on judging will investigate if creativity is
measurable and how AI might be able to help the judging system. Named
after Radia Perlman, one of the prominent female Internet pioneers, the
result is Pearl.
WATCH THE VIDEO
More or less measurable
Pearl was fed with
all international MIXX case studies from the last decade and given 3
months to analyse the texts, videos, results and even music making her
surely the most experienced advertising jury on the planet. Pearl uses
state-of-the-art technology, for example: Microsoft gave early preview
access to their Video Indexer service, which extracts insights from
videos using Artificial Intelligence. She has taught herself to think
like humans and has already reached an accuracy level of about 76%. In
other words, if you feed her a new case, she is pretty good at being
able to predict whether it will be a winner or not, and even what colour
of award you might expect.
Says Jos Polfliet, head of machine learning at Faction XYZ: “This is beyond our expectations, but it seems that
creativity is more or less measurable after all.”
already giving us great insights into what is important in an award
winning idea. She has made links between narrative styles and awards,
between choice of instruments and awards, and she also has understood
that the number of impressions doesn’t mean that much, while number of
shares is far more important.
There is room for improvement
though, as she needs to be able to better recognize humour and a
creative idea. On the plus side, she can detect original combinations,
which helps, “but sometimes she has the reasoning of a pregnant woman,”
says Peter Ampe, ECD of DDB in Brussels. “She thinks Tuna with whipped
cream sounds like a great idea! But she’s still learning and her
accuracy will continue to improve.”
Back-up for pre-judging
is not quite ready yet to replace human jurors, but she can already
complement them. She doesn’t of course have the flaws human juries have
as she is able to give every case the same level of attention, never
grows tired, and perhaps most importantly doesn’t have friends to defend
or a network or country to promote.
There is no doubt AI will be
able to help award shows in different ways in the future. For example,
in larger award competitions, where much of the pre-judging is done in
small groups, it is sometimes the case that good campaigns are
overlooked. With AI, it will be possible to judge which campaigns might
be potential winners and then compare this with the shortlist the human
jury has put together. If some of these campaigns are not shortlisted,
the human jury could perhaps look at these campaigns again and maybe
reconsider their judgement.
During her first appearance, Pearl
will have the honour of picking her favourite campaign amongst the
different MIXX Award entries. We hope she will be invited to more award
shows in the future.
What PEARL learns us:
- PEARL is
not impressed by the numbers of impressions you ran. She prefers
qualitative metrics such as ‘trending topic’ and ‘earned media’.
- If your campaign has had more than 100.000 shares, don’t hesitate to share it with the jury.
- Being mentioned by dedicated ad press is no guarantee to win. Being
mentioned by BBC & CNN however, does give more credibility to your
- Upbeat feel good music is so much overused that it’s being perceived as cliché and generic.
- The use of a voice-over has a positive effect on the outcome.
- When it comes to new technologies, buzzwords have the most success in
the year they were launched, and even more success in the year
- Case movies of winning campaigns contained 25% more suits than non-winning cases.
- PEARL can identify the originality of your campaign, but is still
learning to recognize if an original idea is also meaningful.
More info via: mixxawards.be.