In view of Nelli Kim's recent interview, Lupita and I thought it timely to revisit the performance of some of the WTC President's judges over past competitions ... this article from 27th August 2012 is reposted here, as a reminder.
You will find a link to the FIG's newly published book of results at the Olympic Games here. This year, they have broken down the judge's execution scores so you can see exactly how each judge evaluated the gymnasts' performances. It makes for interesting reading - if only I had more time to analyse each judge's marking. A skim reading already highlights multiple inconsistencies in individual judges' marks and makes you wonder why they bother with the jury at all.
I have taken the time to look at the reference judges' scores for the top four in the women's all around. The FIG explains here what their role is, and how they are selected. I even used my calculator, which is a risky thing in my hands. My, how I wish we could have seen a similar document for the Tokyo World Championships.
I wonder if anyone can explain how, if the FIG's Code of Points is so objective and fair, it is possible to come up with two different results using two different sets of judges? Presumably the reference judges are expected to be highly reliable in their evaluation? Here is what the result of the women's all around competition would have been, according to the reference judges. It casts a whole different light on the competition, one which, satisfyingly or frustratingly, is reflected by the judgement of many on the gymternet.
1 Viktoria Komova 62.1
2 Gabby Douglas 61.75
3 Aliya Mustafina 60.05
4 Alexandra Raisman 59.2
One of the key differences appears to be in the bars scores, where the e-score situation for Douglas and Komova is more or less reversed by adopting the reference judges' score over the jury's. Bars scores had seemed particularly inconsistent all week.
Valentina should have a field day with this document. I am not sure what conclusions to draw, but I hope you have fun reading it. Do comment, please.
You will find a link to the FIG's newly published book of results at the Olympic Games here. This year, they have broken down the judge's execution scores so you can see exactly how each judge evaluated the gymnasts' performances. It makes for interesting reading - if only I had more time to analyse each judge's marking. A skim reading already highlights multiple inconsistencies in individual judges' marks and makes you wonder why they bother with the jury at all.
I have taken the time to look at the reference judges' scores for the top four in the women's all around. The FIG explains here what their role is, and how they are selected. I even used my calculator, which is a risky thing in my hands. My, how I wish we could have seen a similar document for the Tokyo World Championships.
I wonder if anyone can explain how, if the FIG's Code of Points is so objective and fair, it is possible to come up with two different results using two different sets of judges? Presumably the reference judges are expected to be highly reliable in their evaluation? Here is what the result of the women's all around competition would have been, according to the reference judges. It casts a whole different light on the competition, one which, satisfyingly or frustratingly, is reflected by the judgement of many on the gymternet.
1 Viktoria Komova 62.1
2 Gabby Douglas 61.75
3 Aliya Mustafina 60.05
4 Alexandra Raisman 59.2
One of the key differences appears to be in the bars scores, where the e-score situation for Douglas and Komova is more or less reversed by adopting the reference judges' score over the jury's. Bars scores had seemed particularly inconsistent all week.
Valentina should have a field day with this document. I am not sure what conclusions to draw, but I hope you have fun reading it. Do comment, please.
Conclusion: Komova should have won. I'm not saying Gabby doesn't deserve that gold medal.
ReplyDeleteWhy Ms. Nelli didn't address us about this travesty of justice for more than a year now?
ReplyDelete"I wonder if anyone can explain how, if the FIG's Code of Points is so objective and fair, it is possible to come up with two different results using two different sets of judges?"
ReplyDeleteWhat you are asking for is impossible. If you are asking human beings in a sport based on human judgements to all be identical with their scoring, than you will need to wait for cyborgs or robot judges to replace human judges.
What you should be looking at is how far apart (percentage-wise) the two different scoring groups are.
On vault, Komova got 9.066 from E jury and 9.000 from E ref; Douglas got 9.466 / 9.5. That is very very close.
On bars, Komova got 8.966 / 8.950; Douglas got 9.133 / 9.00. OK now Douglas's score is more than .1 diff. So what is the rule when the E ref scores come into play? This is the next question to ask...
On beam, Komova got 8.933 / 9.150; Dougklas got 9.00 / 8.8. Komova got her scored changed to 9.041. Ahhhh, so Komova got her E ref score averaged in on this routine. Was it because her difference was over .2? Douglas difference was exactly .2. So again what is the rule?
On floor, Komova got 9.1 /9.2 (highest score, only one to top 9.0); Douglas got 8.933 / 8.75.
After looking at this, my biggest question is when is the E ref score supposed to be factored in to a gymnast's final score? Not how can two different sets of human judges came up with different scores that are off by .0-.25?
The reference judging is somewhat complex, and it's based on the score that the reference judges give. Here are the rules that govern it:
Delete1) If the average of the reference scores is between 9.6 and 10.0, then, only 0.05 can separate the reference judge average from the e-jury average.
2) If the average of the reference scores is between 9.4 and 9.599, then, only 0.10 can separate the reference average and the e-jury average.
3) If the average of the reference scores is between 9.0 and 9.399, then, only 0.15 can separate the reference average and the e-jury average.
4) If the average of the reference scores is between 8.5 and 8.999, then, only 0.20 can separate the reference average and the e-jury panel.
5) If the average of the reference scores is between 8.0 and 8.499, then, 0.30 can separate the reference average and the e-panel average.
Bearing that in mind, let's look at the discrepancies…
DOUGLAS ON UB: 0.133 separated the reference judge average and the e-jury panel. That's less than the allowed 0.15, so her e-jury panel score stuck.
KOMOVA ON BB: The reference judges awarded Komova a 9.150, which means that the difference between the reference average and the e-jury panel must be within 0.150. It wasn't. The difference was 0.217, so they had to take the average of the e-jury panel and the reference panel.
DOUGLAS ON BB: The reference judges awarded Douglas an 8.800, so the difference between the reference average and the e-jury panel had to be within 0.200. It was exactly 0.200, so Douglas's e-jury panel stayed the same.
DOUGLAS ON FX: Douglas received an 8.750 from the reference panel, so the difference between the reference average and the e-jury panel must be no greater than 0.200. It was 0.183, so Douglas's e-jury panel stayed the same.
In the end, the rules were followed as written. Now, whether you think the calculations are acceptable, well, that's an entirely different question.
Uncle Tim, why do you know EVERYTHING? Thanks for this breakdown!
DeleteI appreciate this break down of the rules too. Thanks Uncle Tim!
DeleteI want to add one more point of clarification: This is not a WAG-exclusive rule. The same formulae apply to the men, as well. So, one cannot blame Nelli Kim for this rule.
DeleteIt's silly to blame one person anyway. I wish people realized this, and realized that the MAG and WAG Technical Committees are called "committees" for a reason.
DeleteBut thank you. You are awesome and possibly omniscient!
I think under any circumstances the gap between the two panels must not exceed 0.2
Deleteso I suggest if it happens then it's up to the apparatus supervisor to favor one of them or stamp a new score
There are two reference judges per event, and they are not immune to bias either - so I don't think averaging two scores is better than the panel of five.
ReplyDeleteHowever, if you average the five E panel scores and the two reference scores, Komova wins (62.12857, E: 36.22857) over Douglas (62.05714, E: 36.35714). Mustafina (59.82857, E: 34.92857), Raisman (59.35714, E: 34.55714). The difference between Komova and Douglas is about 0.07
If you drop the highest score for each athlete, the order is the same: Komova (61.91667, E: 36.01667), Douglas (61.9, E: 36.2), Mustafina (59.7, E: 34.8), Raisman (59.23095, E: 32.32095). The difference between Komova and Douglas is 0.016.
If you drop the highest and the lowest score of each athlete, the order changes: Douglas (62.08, E: 36.38), Komova (62.06, E: 36.16), Mustafina (59.78, E: 34.88), and Raisman (59.4, E: 34.6). The difference between Douglas and Komova is 0.02.
For reference, the official scores with the original five panel judge are as follows:
Douglas 62.232
Komova 61.973
Mustafina 59.966
Raisman 59.966
My two cents: Mustafina definitely earned the Bronze medal over Raisman. I think it's harder to make a decision about Douglas and Komova's results. Separate analyses show different outcomes - so if you choose to believe certain judges scores were superior to others, you'll only end up with your opinion in the end. I was glad to see Komova on top in different analyses, because I am a fan of her gymnastics, but I'm still not convinced who won. However, the twenty E score judges that were selected and given a set of criteria before the competition determined Douglas was the winner.
I think we can all agree that Mustafina deservedly won the AA bronze over Raisman. Mustafina is 1,000,000 times the AA'er of Raisman and always has been.
ReplyDeleteAs for Douglas and Komova, I think it's a debate that will rage as long as the sport exists. This competition was very inconsistent in judging, and quite frankly, I'm still not sure I believe the score awarded to Douglas. She had always struggled to break 60 in both domestic and international competitions, and all of a sudden, the judges in the OG AA final give her a 62+ with routines that were not significantly better... That just reeks of strange to me.
But what I"m more interested in is the fact that a 0.259 difference between gold and silver can be changed with official judges' live scores to yield a different podium. If this is the case for that gap, what would the case be for the 0.033 difference between Wieber and Komova in the 2011 WC AA? I wish FIG would publish a similar book for that competition, too, because my gut tells me that if we take a look at all of the scores in there, every other combination would yield Komova as the winner. And that means that Wieber and USAG need to thank their lucky stars and whatever God exists out there that this particular combination of judges existed in 2011 to grant Wieber that gold.
Personally, I'm still convinced that money exchanged hands before the WAG AA between USAG and FIG. Gymnasts don't with by the smallest allowable fraction of the smallest allowable deduction anymore; that shit just doesn't happen. And anyone with a brain can see that Komova's bars were way underscored in the 2011 WC AA compared to Wieber. The fact that Wieber got a 7+ E score on bars for that shitty routine while Komova only got an 8.5-ish just REEKS of corruption.
But honestly, if we can mix and match various judges' scores to give different outcomes with this, the only WAG AA Champion who has a definitive win is Mustafina in 2010 who beat her competition by 1.034. There's probably no way that you can mix and match official judges' results to change the gold medal winner there, and honestly, it just makes Mustafina's dominance in 2010 that much more admirable. Mustafina truly is a legend now. And anyone who says otherwise can stick a 12-inch barbed dildo up their ass.
Regarding Douglas' scores in 2012 - she had three competitions over 61 before the Olympics. She won Olympic Trials with 62, Runner up at Nationals with 61, and she outscored Wieber (61) at the American Cup as an exhibition gymnast. She had a great 2012, except for Pacific Rims, where she injured her ankle.
DeleteRegarding Komova's bars in 2011, she hit the lower bar with her toes on the swing following her laid-out jaeger which incurs 0.5 in deductions. Wieber's 7.3 score on bars seems like an understandable score - she had major problems: loss of form and control on the HB to LB (Bail?), missed handstand, lack of flight in tkatchev release etc, but she didn't fall off (1.0) or hit the apparatus (0.5), so her score is likely the accumulation of various 0.1 or 0.3 deductions for each element competed. I think what you mean to say, is anyone with a brain appreciates Komova's routine more than Wieber's - which I agree with.
Regarding bribery in the sport - I don't know. Gymnastics is not an incredibly popular/religiously followed sport in the US, so I kinda doubt the USAG bribed the FIG or judges at the 2011 games - since the majority of US citizens wouldn't care. The Olympics is a different matter - I'm sure bribery occurs all over the place there since everyone is suddenly a gymnastics fan when the Olympics roles around. But if that were the case, I would have thought Wieber would have been declared winner since all the commercials were about her. NBC didn't even note it was a possibility for her to miss out on finals until Raisman's floor in qualifications.
I wont argue about Mustafina's 2010 win or dominance in the sport. She's amazing!
The incompetence of the judges was (and still) obvious ...
ReplyDeletefor instance: (page 141) look at Grishina’s scores 7.9 8.1 7.1 7.9 7.4 (one point between the highest and the lowest score and after you abolish both of them there’s still half point between the highest and the lowest score!!!) and even the reference Judges’ scores are no better 8.3 7.4 ( nine tenth discrepancy!!!)
It's Nelli/FIG responsibility to prepare a high qualified judges whom scores work in unison not in such a fiasco
Looking at just that uneven bars apparatus during qualifications, 5 of 78 gymnasts had that happen to them (scores range of 1.0 or greater from 5 person E jury). That is 6.5% of gymnasts had scores that ranged 1 point or more. This means that 93.5% of the gymnasts had scores that were all less than 1 point apart. It is still humans making judgments as best they can with their eyes while its happening.
Deletehttp://ryviewpoint.blogspot.com/2008/04/how-accurate-is-human-eye.html
Line judges in tennis were found to be wrong 1 in 12 times on balls within 100mm (3.9") of the line, or 8.3%. Just some food for thought.
First when you have 6.5%(as you calculated) misevaluated gymnasts (who don’t have right to challenge their E-score) then it means that you have a lot of work and self-assessment to do, or you imply that every competition the judges have the right to screw 6.5% of gymnasts?
DeleteSecond I have given the above example because a full point discrepancy is a disastrous in the world of gymnastics, as we all know that less than 0.1 could make a real difference and 2011 AA final is a witness … bear in mind that the rules put up to 0.3 as A MAXIUM average between the two panels as Uncle Tim explained !!!
Third would you calculate the score differences that ranged 0.9 to 0.3 and tell me the percentage? I know that it will be a hell of percentage …
My point is that human judgement will always have errors. We can work to improve this with better training, with technology (like I believe the D score judges get to look at video), but with human judgement, there will always be errors.
DeleteThat was my only point.
And my point is simply that there are limits for these errors ... and if it exceeds that limit there is system/code to change or someone to be punished, specially when these errors going all the way in favor of one country only ...
DeleteI am not sure one can place limitations on human judgements. If the judges had access to instant replay then that would increase their accuracy, but it would also slow the judging process down. hehe
DeleteAnd I find this debate ultimately pathetic, childish and unsportsmanlike. Over 30 million people, either publically or privately, were screaming for the judges' heads just over two years ago, wailing "scandal, scandal, scandal" when a panel of judges draped a gold medal on Adelina Sotnikova's neck, demanding they strip the girl of the medal and award it to another. Many are still howling about it. Those 30 million, fortunately, were unsuccessful in perpetrating such a travesty of justice. I say to you as I said to them---if you expect to host the Olympic Games, you must show the sportsmanship, good will and class that is required under the rules to prove that you DESERVE to host them. Squabbling, wailing and sniping about one-tenth or two-tenths of a point, based SOLELY on the opinion of human judges, hardly fits the definition of sportsmanship, good will or class.
DeleteThe judging was clearly all over the place in London, including the reference judges. There needs to be a better system, but it will never be perfect. The judges are human and therefore view things differently. There are no robots that judge everything perfectly according to the COP, which still favors certain gymnasts. Another thing is that judges can always find ways to justify their scores by giving deductions for lack of amplitude, precision, pre-rotation, etc. It is never the gymnasts' faults, just the judges or COP. I don't think the reference judges can be used as justification of Vika deserving the AA title. The scores were very close, so it could have gone either way. They had also thought that Maroney deserved the vault title with her fall, so they are not the most reliable.
ReplyDeleteIndeed, considering that they would have given Maroney the gold with a fall(!) I say "Thank God we don't go by the reference judges."
DeleteMoney Money Money Isn´t it obvious???? USAG X FIG??? Everybody, except american, thinks Komova won AA WG 2012 and WC 2011.
ReplyDeleteBut I am gladedy shocked hearing that here.
*eye roll*
DeleteKomova beat herself. As always. Lovely gymnast but she is not a competitor.
I completely agree with the eye roll. Komova is her own worst enemy. I remember when Tim Daggett declared that she was the best gymnast in the world, I Googled Komova's routines. I saw a gymnast with a terrible habit of giving away tenths here and there with lots of little bobbles. The Oly AA was no different. She is a lovely gymnast who is just not a great competitor. Even Komova has publicly acknowledged her penchant for giving away lots of little deductions.
DeleteIt's also hard to take the Komova defenders seriously because they never acknowledge that a strong argument can be made that Komova was overscored on her vault in the Oly AA. She went off not just the mat, but the podium, and took several steps to the side.
Personally, I think the AA was pretty close and either one could have won. I'm not surprised that different panels found a different winner. As for Gabby, the only thing that surprised me was her UB score. I thought the AA was her best bars routine, but I thought it would be in the 15.5 to 15.6 range, not 15.7. That surprised me.
IMO the real winners in London 2012 are:
ReplyDeleteTeam: USA-RUS-ROM
AA: Komova-Douglas-Mustafina
VT: Izbasa-janine-paseka
UB: Mustafina-He-Yao
BB: Deng-sui-Ponor
FX: Ponor-Raisman-Ferrari
So what is your list of winners?
IMO the real winners in London 2012 are:
DeleteTeam: USA-RUS-ROM
AA: Douglas-Komova-Mustafina
VT: Izbasa-Paseka-Berger(Janine)
UB: Mustafina-He-Tweddle
BB: Sui-Deng-Raisman
FX: Raisman-Ponor-Mustafina
IMO the real winners in London 2012 are:
DeleteTC: USA - RUS - ROU
AA: Douglas - Komova - Raisman
VT: Izbasa - Berger - Maroney
UB: Mustafina - Yao - He
BB: Sui - Deng - Raisman
FX: Raisman - Ponor - Ferrari
IMO the real winners in London 2012 are:
DeleteTF: USA - RUS - ROU
AA: Komova - Douglas - Mustafina
VT: Izbasa - Berger - Paseka (but Maroney was still the best anyway)
UB: Mustafina - Kexin - Tweddle
BB: Lu - Linlin - Raisman
FX: Raisman - Ponor - Mustafina
IMO the real winner in London 2012 are:
DeleteTF: USA - RUS - ROU
AA: Komova - Douglas - Mustafina (Douglas and Raisman were "let's not even try to hide it" overscored)
VT: Izbasa - Maroney - Paseka
UB: Mustafina - Kexin - Tweddle
BB: Lu - Linlin - Raisman
FX: Raisman - Ponor - Mustafina
What I would've wished:
TF: USA - RUS - ROU
AA: Komova - Douglas - Mustafina
VT: Maroney - Izbasa - Berger
UB: Tweddle - Mustafina - Komova
BB: Lu - Ponor - Linlin
FX: Izbasa - Ponor - Raisman
My opinion of how the London 2012 podiums should have looked:TF: USA - RUS - ROU
DeleteAA: Douglas - Komova - Mustafina (Douglas had the cleanest four routines and Komova's vault was terrible and overscored. I wouldn't want an AA champion who faltered that much on one of the apparatus.)
VT: Izbasa - Berger - Maroney. (Vault was a splatfest. Maroney only gets bronze because so many others faltered.)
UB: Mustafina - Kexin - Tweddle
BB: Linlin - Lu - Raisman
FX: Raisman - Ponor - Mustafina
I can't stand dealing with that logic adopted by those people who see that Komova shouldn't get the gold because she took a couple steps aside in an "ALL AROUND" Competition but see nothing wrong that Maroney get a medal in "APPARATUS" Final while Marony failed to stand her second vault ...
DeleteIt's a schizophrenic rhetoric
IMO the real winners in London 2012 are:
DeleteTeam: USA-RUS-ROU
AA: Komova-Douglas-Mustafina
VT: Izbasa-Berger-paseka
UB: Mustafina-Yao-He
BB: Deng-sui-Ponor
FX: Ponor-Raisman-Ferrari
Uh, Komova took more than "a couple steps aside." Her vault lacked the height and distance, she landed to the side, then took three steps, could've been counted as four not just to the side, but off the podium, thus to the side and down. Her vault had more problems than just a couple steps aside. Have to agree about Maroney, though. A fall in apparatus finals doesn't deserve a medal at all. But Komova didn't deserve the gold either.
DeleteOf course it was corruption. Even Wieber was surprised with the result. Why would it be different in the Olympics??
ReplyDeleteI live in Salvador / Brazil and I was dumbfounded when FIFA approved this city to host the Games of the World Cup. I live and work near the stadium and lost a tire in the most expensive neighborhood in Salvador, BECAUSE holes in city streets.
And now I ask you, as HOW Brazilian politicians convinced FIFA members to approve this city to host the Cup? CORRUPTION.
MONEY
FIFA rotates which continent will host the world cup, the world cups was slated to be held in South America in 2014. Brazil was the only country to place a bid, therefore their proposal was accepted. No corruption there - lack of bids from other countries! This happened in 2006 or 2002.
DeleteDon't try to use logic... :) There must be a conspiracy with Olympics because some people believe the results should favor the Russians because these people prefer their classical style and balletic beauty as the only defenition of artistry. :) (ok sarcasm meter must be turned off to read this lol)
DeleteDear., do u know how many states do Brasil have to Fifa decides host a Game in Salvador insted of Pernambuco??? Aracaju?? ParaÃba??? Do you know the IDH of these states to Brasil accept to host a World Fifa Cup insted of building hospitals, schools, etc...? How many money have being spend insted of goodwill of the people??? I am from the Southeast of Brasil. So. why don´t they make the Cup in South and Southeast? Thry are building a stadium in the gate of rain forest. What for? Don´t be cinic.
DeleteAnd now, I will try to use logic... :;) I hate classical russian style ant balletic beautyas the only definition of artistry. I hate Bolshoi Ballet.
DeleteNobody likes money. Money doesn´t move the world. Corruption just exists in poor countries. Never a Olympic judge would accept money to overscore a gymnast..Never.
I am totally crazy. Now I will take some piills.
Let's take a breather. As bad as the 2012 AA final was, it was nowhere near as bad as the 2000 final. As far as I'm concerned, no one won the AA in Sydney.
ReplyDeleteWell, 2000 had a great field, the russians, the romenians, Liu Xuan, Yang Yun, Viktoria Karpenko and the underdog Lisa Skinner (she even got to the top 3 in the third rotation). It was still enjoyable IMO because the gymnasts there were more impressive, artistic and it was harder to get to the podium, It would be awesome to see Khorkina slay had the vault been set properly, though.
DeleteIn 2012 we had four/ffve (Wieber) medal contenders, Mustafina and Raisman never had a shot at gold. FIG ruined the sport and is making what is supposed to be a nice rivalry (Komova x Douglas) a boring and frustrating competition. I'd take Sidney over Beijing (the CoP here was at its worst) and London any day.
TOTALLY AGREED. God, what a fiasco. When people talk about how horrible Raducan's ordeal was I always respond "yes, it was horrible...and everyone else was treated as shamefully as she was." They're lucky none of those girls were seriously injured, like with career-ending injuries! We will *never* know who is the real AA gold medalist--frankly, I felt worst for Khorkina. Then the Chinese earn the bronze--oh wait, no they cheated with an underage gymnast. What a mess. I try to forget Sydney even happened.
DeleteI felt the worst for Elise Ray and Annika Reeder. Annika got injured thanks to that stupid vault and Elise because of the trauma of that terrifying warm-up fall. I'm willing to accept Andreea as Olympic Champion though since she did compete on the altered vault so she didn't have the advantage of waiting for the correction.
Delete