A Quality label

Re: A Quality label

Postby kaf on Mon Nov 09, 2009 11:31 am

Well, then the author (the submitter) should at least have an option to exclude his problem from the rating system. Maybe some checkbox to mark/unmark?
Or separating real problems, made by masters, from "i-kind-of-played-a-game-and-think-that-everyone-should-know-about-that" would be nice too.
kaf
 
Posts: 47
Joined: Fri Dec 05, 2008 6:09 pm

Re: A Quality label

Postby adum on Tue Nov 10, 2009 1:35 pm

as tails said, the ratings are for all registered users, and are permanent (though of course users are free to change their rating at any time).

as the system is fairly new, i wouldn't get too concerned when a problem has only two ratings and you disagree with the average. let's see how the ratings balance out after some more time.

the Stats page shouldn't just show the problems with the highest average as Top Rated, but should do something like a confidence interval:
http://www.evanmiller.org/how-not-to-so ... ating.html
this is my fault since i always had it as an average.

excluding a subset of submissions from a rating system is probably not going to happen. if something gets submitted to digg or StackOverflow or any other social site i can think of, it suffers the same trial by fire as any other submission.
User avatar
adum
Site Admin
 
Posts: 385
Joined: Tue Sep 30, 2008 5:09 pm

Re: A Quality label

Postby kaf on Tue Nov 10, 2009 2:37 pm

Thanks, Adum. I just overreacted, as usually. That "1" set to my favorite problem tipped me off. Nevertheless, the "most high-rated problems" list is a face of the site, if you will. So, I think the idea of placing a problem in this list only after at least 10 people voted, is not bad. Evidently, any invalid problem rated "5" by its creator will top the list and deface the whole site, until somebody downrates it.
kaf
 
Posts: 47
Joined: Fri Dec 05, 2008 6:09 pm

Re: A Quality label

Postby tails on Tue Nov 10, 2009 10:18 pm

Hmm... I thought it was rather a good thing that the list is unstable, in that new problems have more chance to get listed, to be viewed, to get voted and so to settle into appropriate values. But that seems wrong.

Adum, can I re-implement the highest and lowest rated lists using the function in the page you wrote above? The detailed ideas are:
  • Calculate a lower bound for each problem, and use it to order the highest rated list.
  • Calculate an upper bound for each problem, and use it to order the lowest rated list.
  • Users can't see those values (because the values don't make much sense humanly), and users still can see average and number of votes.

Or should we have 20 highest rated problems instead of 10 highest and 10 lowest in that page?
tails
 
Posts: 90
Joined: Thu Oct 23, 2008 12:10 pm

Re: A Quality label

Postby kaf on Wed Nov 11, 2009 2:12 pm

That's an excellent idea, tails. I could never understand the purpose of the lowest rated problems list.
kaf
 
Posts: 47
Joined: Fri Dec 05, 2008 6:09 pm

Re: A Quality label

Postby adum on Wed Nov 11, 2009 5:07 pm

i agree with your suggestions, tails and kaf.

and lowest rated probably is not necessary.
User avatar
adum
Site Admin
 
Posts: 385
Joined: Tue Sep 30, 2008 5:09 pm

Re: A Quality label

Postby tails on Wed Nov 11, 2009 10:15 pm

I have removed the lowest rated problems list. Please give me some more time for ordering problems by the confidence interval function. The page shows a function for two-value votes, and I must remake it for multi-value votes.
tails
 
Posts: 90
Joined: Thu Oct 23, 2008 12:10 pm

Re: A Quality label

Postby tails on Thu Nov 12, 2009 6:03 am

I have made a test page that shows how a problem will get scored in the new ordering system.
http://www.goproblems.com/test/wilson/wilson.php
Please note that "1" voters have more impact on the new ordering system because it uses the lower bound of the interval.
I think I will wait for opinions for some days before implementing the new list.
tails
 
Posts: 90
Joined: Thu Oct 23, 2008 12:10 pm

Re: A Quality label

Postby kaf on Thu Nov 12, 2009 11:48 am

What I'm trying to recommend is excluding from the list problems with only one vote. That should take care of garbage. Although I've learned statistics while ago, I remember that one cannot make statistics out of a single point.
kaf
 
Posts: 47
Joined: Fri Dec 05, 2008 6:09 pm

Re: A Quality label

Postby tails on Thu Nov 12, 2009 12:31 pm

A single 5-star vote will give only 1.83 Wilson score to a problem, which is lower than the score three 3-star votes give, so the problem should be unlikely in the list. Isn't that satisfy your request?

I think the good point of Wilson score (compared to the normal approximation) is that it can give a good score even for a small number of votes.
tails
 
Posts: 90
Joined: Thu Oct 23, 2008 12:10 pm

PreviousNext

Return to Suggestions

Who is online

Users browsing this forum: No registered users and 5 guests

cron