Problem difficulty rating lowered after solution edit

Post Reply
Aristotle
Posts: 5
Joined: Thu Nov 29, 2012 1:22 am

Problem difficulty rating lowered after solution edit

Post by Aristotle »

It happened a few times to me that, immediately after I edited the solution to a problem, its difficulty rating suddenly plummeted by 60 or 70 points. Of course, this might have been coincidental with many attemptees solving my problem correctly, but that event seems unlikely. Particullarly, since it didn't happen once but several times (I remember 3).

So is this a bug or is there a system to it?

Aristotle,
User avatar
adum
Site Admin
Posts: 422
Joined: Tue Sep 30, 2008 5:09 pm

Re: Problem difficulty rating lowered after solution edit

Post by adum »

hmmn, i'm not sure why that would happen. did you check the difficulty history graph?
Aristotle
Posts: 5
Joined: Thu Nov 29, 2012 1:22 am

Re: Problem difficulty rating lowered after solution edit

Post by Aristotle »

I am not sure how to find that graph. Do I need a pro account for that? By now I have a theory on how the problem difficulty rating might plummet suddenly. I may have edited some variations containig "attempt paths" by solvers. When an attempt path changes, possibly the attempt count will no longer register in the overall difficulty rating thus changing its value. Don't know, just an idea.

Aristotle.
User avatar
adum
Site Admin
Posts: 422
Joined: Tue Sep 30, 2008 5:09 pm

Re: Problem difficulty rating lowered after solution edit

Post by adum »

go to a problem page. click on the difficulty in large letters (like "20 kyu"). you'll get a popup. click "rating history"
Aristotle
Posts: 5
Joined: Thu Nov 29, 2012 1:22 am

Re: Problem difficulty rating lowered after solution edit

Post by Aristotle »

See it now! Internet Explorer 8 doesn't pop up the difficulty rating on my computer screen but Firefox does. The chart reveals that incorrect attempts add little value to the difficulty stat but incidental solutions may cause dramatic drops. I suppose this behavior is by design and just happened to coincide with my edits.

Aristotle.
Aristotle
Posts: 5
Joined: Thu Nov 29, 2012 1:22 am

Re: Problem difficulty rating lowered after solution edit

Post by Aristotle »

I finally found what is "buggy" in the problem difficulty system. "Difficulty" is lowered drastically when a problem is solved by a relatively "weak" solver. However, the system carries no memory on how weak a player really is. Anyone with a non-pro account may chose to climb the ladder every week up to his real strength. If he selects problems above his current rating - but below his real strength - he destroys the difficulty value of the problem.

The solution would be to for the system to memorize the real strength of a non-pro solver - e.g. the highest end-of-week rating in the past 3 months - and apply that to modify the difficulty rating of problems solved. Of course, this solver must still climb the ladder from 30th kyu upward to reclaim his former rating. This feels a lot more accurate in assessing problem difficulty than the current system.

Aristotle.
User avatar
adum
Site Admin
Posts: 422
Joined: Tue Sep 30, 2008 5:09 pm

Re: Problem difficulty rating lowered after solution edit

Post by adum »

well, there is some built-in protection against that. problem ratings are moved relatively little by players who don't have an extensive solving history. so a new player shouldn't do much. once they have an established rating, their attempts will have more of an effect.
Post Reply