A Quality label

[spook]

I have the impression that the current collection of problems contain a LOT of problems.

But I also have the impression that a lot of problems have become more like discussions than problems.
Go is a game with such many possibilities that very often there are unmentioned variations.

Do not take this as criticism, because I am thankful for the fact that everybody has the ability to discuss the problems on the website.
But sometimes it is very hard to say a move is "correct" or "wrong", even though it could be the best available move.

Usually when I solve go books I only disagree with 2% of the problems.
But on this website I have different opinions about almost 50% of the problems.
I think this is because books try to avoid publication of problems with answers which are only correct in "certain situations".
Usually the problems presented in books have many wrong answers which fail miserably and at the same time only 1 correct answer which is undeniably good.

The point of my topic here is: maybe we can achieve the same quality as the ones in such go books?
I believe there are ways to obtain this. Here are just some ideas.

QUALITY LABEL IDEA:
When 50 high dan players (4d or stronger) say a problem is of "HIGH QUALITY" the problem gets a special keyword "High Quality Problem".
This still can be rejected again when only 5 high dan players disagree again.
The idea is that much effort is required to make something of high quality while only very little disagreement can take it back down.


QUALITY SCORE
Maybe the "coolness" factor was originally designed for this (?) , I don't know. But currently it is being used more for scoring problems with "unexpected answers which look really amazing and unexpected = cool". So it is not really used for the quality. But maybe adding a score for "quality" is a good solution?

These are just suggestions. I hope they are useful.

[adum]

hi spook. i completely agree that problem quality is an important concern, and that striving to improve the quality should be a major goal.

i haven't tried to do a survey myself, but it seems like the fraction of 50% of problems being problematic seems rather high. what do other people think?

i quite like the idea of a quality label for problems. wikipedia has something a bit like this called Featured Articles:
http://en.wikipedia.org/wiki/Wikipedia: ... d_articles

they only accept about 1/1000, which is a bit lower than i think we want for goproblems =)

about a year ago (six months? i forget) i instituted the flagging system to help with this exactly problem. the idea being that people could flag problems as either good or problematic, and over time this would help people improve quality. (this especially targeted the issue of long neglected issues.) if enough people target a problem as needs fixing, editors can find it and fix it:
http://www.goproblems.com/prob/flagged.php

conversely, people can flag a problem as good and with enough of those flags, it's similar to the quality label you speak of. perhaps a problem could actually get a star when it reaches 5 good labels or so.

so the question i have for you is: how does this system not work right now?

thanks,
adum

[BartTM]

I have been thinking about a quality label for GoProblems too, for a long time actually. The fact that anyone can (or could) rate coolness or set flags meant that those ratings suffered from unfair input, as some users just won't use those earnestly, and also the terms have not been defined (What is 'Cool', really, and what makes a problem a good one?).
I feel a quality label should only be awarded by a select group of users (mods and pro account holders?). Also there should be a clear definition of rules to which a qualified problem should adhere, like:
- The problem should be original. Copies from books, other problems and problem sites, etc. should only be awarded if the source is mentioned in the problem and explicitly allows copying it.
- The problem should be correct. The path(s) listed as solutions should be better than those listed wrong. If the result depends on circumstances then those should be very explicit indeed.
- The problem should be complete. All path leading to a result equal to the solution must be covered. Paths may only be blocked to avoid overcomplexity and when that is both enforced and explained in the problem.
- The problem should be understandable. Wrong variations should be in place for all reasonable moves and played out until the outcome should be clear even to unexperienced go players.
- The problem should not be offensive to anyone. Rude language, religious, political, sexual or discriminating statements, symbols, etc. have no place in a (e?)quality problem. Humor, well, only if not offensive.
- If a problem is changed it loses its quality rating. It may be put in a special category to be re-evaluated.
- Users cannot rate their own problems.
- I don't think the list should end here but you'll get the idea.

If 1000 problems on this site would qualify under these rules we're doing very fine indeed. Looking back at my own problems I think less than 10 would qualify.
Not saying that this is the only way to go, I admit being very interested to find out if and how a quality system for Goproblems evolves.

[BartTM]

Since my last post the rating system came active so please ignore my previous suggestion, at least for a while to see how the rating system works out.

I saw I can rate my own problems, which I think should not be possible.
Also the 'Cancel rating' button is available to users who have not rated the problem yet. Nothing happens so there is no harm, but it would be nicer to hide it until a rating is set.

[tails]

Thank you for the suggestions.

For self rating, I'd like to ask for more opinions (and finally, adum's decision). I personally think it's ok to rate your own problems.

For hiding the cancel button, I'll do it soon later.
EDIT: Done. (Clear the cache if you still see the button.)

[tails]

By the way, for the flagging system, what about to label the first flag as "Well Constructed" or like that, instead of "Good Problem"?

I sometimes encounter the situation that the problem is very good itself but it needs fixing because e.g. some important variations are missing.

[adum]

that's an excellent point on the 'Good Problem' flag. i didn't think about it, but 'Well Constructed' is much more specific. i have made that change in the English.

on not letting problem authors rate their own problems, hopefully it doesn't matter too much either way, because there should be enough ratings on each problem to drown out one author's vote (i'm sure there will be once the system is in place for a while.) i know on some social sites, like i think reddit comments, when you post something it automatically gets your own upvote, so maybe people are used to that. in any case, i don't have a strong opinion on this one.

for the big issue of starred problems, let's do as bart suggests and see how the new rating system works out in combination with the Well Constructed rename.

[kaf]

And so the "rating war" has began.... again. I really don't like the idea to allow any person rating problems created by the best go minds. For instance, I have just noticed that some a-hole (sorry, but I can't find better word) has rated "1" my favorite problem by GoSeigen (#8541) . This is ridiculous at the least and very disrespectful toward the creators of these beautiful problems (and I mean the real authors, not submitters).

[santa c]

about voting on our own problems - i just voted most of the problems on my fav list as 5, (unfortunately.... )i don't have many of mine there but the ones i did have i treated like any other...

i'm a little curious how many ppl can effectively vote? are registered non pro users valid voters? can they revote every couple of months? maybe everybody should be able to vote then? (guests included) if the votes aren't singular, anyway... and as for the quality label, maybe only registered users (/pro) votes would count (just a separate hidden counter)?

also would be nice to add now that daily (or 12 hourly) problem so that many ppl can vote (well and discuss) on one problem everytime...

kaf maybe in some time when there're enough votes it'll turn into at least several votes to count for the best table (so that a single 5 or 1 vote wouldn't land a rpbolem ther)...(?)

[tails]

kaf, I guess the problem was too hard for the voter to understand, and the voter just didn't like it. And I think you don't have to be embarrassed, you can just say to your students: "Oh it's just a rating that everyone can vote."

santa c, at least for now, registered non-pro users are valid voters, and votes are valid permanently regardless of whether the voter is pro or non-pro. So a user can't vote twice to a problem (unless the user has multiple accounts). If adum decides in the future that non-pro votes should get expired, then the votes should be treated just as they are canceled, so a user can't vote twice anyway.