Voting, Karma and Vaults - Oh my!

[Legacy_Gruftlord]

               

FunkySwerve wrote...

Short thought: IMO all voting systems are messed up in some way and open for abuse or even just ruined by differing standards, and it's in the nature of the thing, as it's all about opinions.

This is demonstrably wrong. Throwing your hands up in the air and saying, hey, no matter what we do it'll have problems, is a false leveling. As an attorney who studied economics and law, I can assure you that there are better and worse ways of structuring systems of rules. If you decide that it doesn't matter, you're likely to wind up with one of the worse ways, since you're deliberaately ignoring the pros and cons.

Here is a thought you missed: What you're saying is, if you want a system, that thrives to be as less abusive as possible, you need a system with a legislation, lawyers and judges. And the System needs to evolve constantly based on the actions of these 3.

Here's the point: online voting Systems only have a legislation, and the rules seldom change. And; no offense; nobody wants lawyers in an online voting system (well, we do like the lawyes, but nobody wants them to do the lawyering in the voting system :-P).

which concludes in the condensed statement that has already been given: you can not have a voting system online, that is not abusable.
and i still think the Facebook voting system is one of the most stable. give a thumbs up, or don't. compare to the number of people that (in our case) downloaded a file, and you get the general satisfaction of the downloaders.
overall numbers of thumbs/votes will anyway be low, but that doesn't change the information content of the votes.

if you start to make the system more complex, lets just say by introducing a thumbs down button (see Youtube) it will be abusable and will be abused to no end (see Rebecka Black, Justin Bieber. Just to clarify abuse goes in both direction :-D).

You can add further complexity to the sytem, but that only makes it more complex to abuse it. you never get rid of the abuse, unless you introduce some attorneys and judges, you handle the abusers.
               
               

               
            

[Legacy_Tarot Redhand]

               So now we are arguing about one sentence that was taken out of context anyway? The above argument completely ignores the immediately following sentence

I still think they can be useful though.

and thus denigrates the original poster. This may be acceptable in the courtroom, but is dubious behaviour outside. The discussion on here is about how we want the new vault to be structured, not the irrelevant minutiae (in the context of the purpose of this thread) that you have chosen to concentrate on. Please keep it on topic guys.

TR
               
               

               

                     Modifié par Tarot Redhand, 30 avril 2013 - 10:17 .
                     
                  

            

[Legacy_Zarathustra217]

               Ack, the vault is closing?! Glad people are doing something - if you need financial support, I'd be happy to chip in and I'm sure it'll generally be easy to gather donations.

In terms of a voting system, here's a random idea:

The way I see it, a major part of the problem is people's differing voting habits. Some people consider carefully whether to vote a 7 or 8 (assuming a ten point scale), whereas others throw 10s at everything they like (and sometimes, 1s at things they don't like).

A way to overcome this is to weigh each person's score against the past scores the person has given to other things (their voting habit). There's many considerations you could use to shape out this. A vote of 9 from a person with an average score-giving of 5.0 could weigh more than a 10 from a person with an average score-giving that was much higher (e.g. 9.0). A vote of 9 from a person that rarely votes outside the 3-7 range could weigh more than a vote of 10 from a person that has voted exclusively 1s and 10s.

Such a system might be a bit resource intensive, but it could be overcome by only updating scores once a day or so.

</random idea>
               
               

               
            

[Legacy_FunkySwerve]

               

Gruftlord wrote...
Here is a thought you missed: What you're saying is, if you want a system, that thrives to be as less abusive as possible, you need a system with a legislation, lawyers and judges. And the System needs to evolve constantly based on the actions of these 3.

No, that isn't even remotely similar to what I said. I was talking about rulemaking, and used an example from my field of expertise. And no, you don't need attorneys to reduce the abusiability of a system, only better crafted rules. And it's 'strives', not 'thrives'. I would continue at greater length, but you clearly didn't spend any time trying to understand my last post, so I see little point in doing so, when I could do something far more productive, like banging my head against a wall.

Funky
               
               

               
            

[Legacy_FunkySwerve]

               

Tarot Redhand wrote...

So now we are arguing about one sentence that was taken out of context anyway? The above argument completely ignores the immediately following sentence

I still think they can be useful though.

No, it doesn't. He went on to talk about the usefulness of the COMMENTS, not the voting system. You're the one reading things out of context. The fact that his post was poorly worded, poorly thought through, and, by your curious reading of it, self-contradictory does not render what I quoted the least bit out of context.

and thus denigrates the original poster.

I was very careful NOT to denigrate him, since I was aware that people are seldom fond of having their ignorance pointed out. It is possible to disagree with someone without denigrating them.

This may be acceptable in the courtroom, but is dubious behaviour outside.

And now who is denigrating who? Resort to ad hominem arguments is both a clear indication that you don't have a leg to stand on, and worse, is the very behavior you're trying to call me out for. Please stop the disingenuous nonsense.

The discussion on here is about how we want the new vault to be structured, not the irrelevant minutiae (in the context of the purpose of this thread) that you have chosen to concentrate on. Please keep it on topic guys.

You don't get to decide which 'minutiae' are irrelevant. My post was entirely on topic - the structuring of the Vault's rules systems, and the problem with your quotee's views on that topic. Don't try to play moderator - you're terrible at it, and worse, you're doing it while attempting to drag the thread offtopic. I have a serious interest in this thread, as I've offered to help code the new Vault, so I would really appreciate it if you could refrain from the sniping.

Funky
               
               

               

                     Modifié par FunkySwerve, 30 avril 2013 - 06:18 .
                     
                  

            

[Legacy_MagicalMaster]

               

Gruftlord wrote...

Here is a thought you missed: What you're saying is, if you want a system, that thrives to be as less abusive as possible, you need a system with a legislation, lawyers and judges. And the System needs to evolve constantly based on the actions of these 3.

Since Funky would rather bang his head into a wall, let me give it a shot.

Funky's point was that you need to be careful when setting up the system.  Analyzing the pros and cons of different options is important and you should try to set up the system in a manner that avoids things prone to abuse.  He said absolutely nothing about needing lawyers and judges - solely that you should make sure you write the "legislation" as best you can versus throwing your hands up in the air and saying "Well, nothing's perfect, so who cares?"

I honestly have no idea why you thought he was advocating "laywers and judges."

Gruftlord wrote...

which concludes in the condensed statement that has already been given: you can not have a voting system online, that is not abusable.

But not all forms of abusable are equal.  And sometimes picking a system that's more abusable may still be a better option if the benefits are superior on average.  But you have to think about it and craft the system as best you can.

Gruftlord wrote...

.and i still think the Facebook voting system is one of the most stable. give a thumbs up, or don't. compare to the number of people that (in our case) downloaded a file, and you get the general satisfaction of the downloaders.
overall numbers of thumbs/votes will anyway be low, but that doesn't change the information content of the votes.

Except that gives next to no information.  If it gets a thumbs up, that doesn't indicate whether it's a decent module, an amazing module, or the new standard by which all modules should be judged by.

On top of that, comparing the download count to the thumbs up also gives little information - plenty of people will download the module but not bother to "vote" no matter what.  So yes, it doesn't change the information content of the votes - because there IS no information content of the votes.

Gruftlord wrote...

if you start to make the system more complex, lets just say by introducing a thumbs down button (see Youtube) it will be abusable and will be abused to no end (see Rebecka Black, Justin Bieber. Just to clarify abuse goes in both direction :-D).

It's already abusable in your "simple" system.  Just download the file a bunch of files so the thumbs up:download ratio is very low.  And it becomes harder to recognize abuse when you stick to thumbs up/thumbs down.  Let's say player A and B player module C.  A likes it, B does not.  A gives thumbs up, B gives thumbs down.  All right.

Now let's say player D and E download module C as well.  Player D likes it and gives it a thumbs up.  E is a troll and just gives it a thumbs down.  There's no way to easily distinguish player B from player E.  On a points system, if most people are giving the module something around 8.00 out of 10 and someone gives it a 0 or 1, you can easily recognize that and remove the vote.  Blatant trolling becomes more difficult.

In short: thumbs up/thumbs down (and even just thumbs up) is a terrible system for something like NWN modules.

Gruftlord wrote...

You can add further complexity to the sytem, but that only makes it more complex to abuse it. you never get rid of the abuse, unless you introduce some attorneys and judges, you handle the abusers.

You don't need to get rid of the abuse - you just want to try to create ways to limit it.  And sometimes opening it up further to abuse is worth it because, on the whole, the system is much better.

Tarot Redhand wrote...

So now we are arguing about one sentence that was taken out of context anyway? The above argument completely ignores the immediately following sentence

No, it doesn't.  He basically says "all systems are messed up because they're about opinions" and then the sentence after it says "voting systems are still useful, though."

That reads like he thinks it does not matter very much what system you use because they're all messed up - but that it's worth using a system regardless.

Our point is that it definitely does matter what system you use - some are clearly better than others. 

Think of it this way - what is the point of mentioning that all systems are messed up and all are open to abuse and all are about opinions *unless* he was arguing that the choice of system wasn't very important?
               
               

               

                     Modifié par MagicalMaster, 30 avril 2013 - 06:25 .
                     
                  

            

[Legacy_Gruftlord]

               

FunkySwerve wrote...

Gruftlord wrote...
Here is a thought you missed: What you're saying is, if you want a system, that thrives to be as less abusive as possible, you need a system with a legislation, lawyers and judges. And the System needs to evolve constantly based on the actions of these 3.

No, that isn't even remotely similar to what I said. I was talking about rulemaking, and used an example from my field of expertise. And no, you don't need attorneys to reduce the abusiability of a system, only better crafted rules. And it's 'strives', not 'thrives'. I would continue at greater length, but you clearly didn't spend any time trying to understand my last post, so I see little point in doing so, when I could do something far more productive, like banging my head against a wall.

Funky

I did read your post carefully, i'm not sure about the understanding, i.e. interpretation of the analogy. i simply may have a different take on how rulemaking works. in my oppinion, making the rules, and trying to improve is a good start, but a legal system is usually in place preciselly to allow for such improvements to happen. i.e.: the basic premise behind jurisdiction is that the rules are not unabiguously, and one has to carefully judge how to apply them. unpreciseness in the rules in never in question here. and there is the option to use all instances in a legal system, and finally force a reevaluation of the rules.

so what i tried to get across was this: it's an iterative process, one that is flexible and allows for changes to the rules over time (decades and loger at times).
in a way it's a learning by doing process, something which will most likelly not see the light of the day in an online voting system such as the one in question.
i haven't seen such a flexible and selfimproving system on any website yet, and if it does it requires the input over many users. which itself is an abusable process. so the process has to be reevaluated itself.


i'm straying away again.
i cast my vote for a simple system with a little thumb up or down. and i raise the question whether a more complex system could be made less abusable with increased complexity; and how the process to find such a system should work. the real world doesn't look like it would be an easy process, than can be done by a handfull of people, even less so a priori.

an personally, i found your analogy quite nice, which is why i was interested in responding and offering a different view on the analogy; on how far the question in place here is comparable, and where the differences are. no harm intended, because such things are always a matter of personal interpretations and view.
and excuse any language mistakes i make, please; i'm not a native speaker.


edit: read your later edits, that were written after i started this post.
some good arguments about the visibility of "troll" votes in a point system.
the same could be implemented in a thumb system, by comparing users average voting habit
% of up and down votes of a user compared to % up down votes on a module.
compare standard deviations of module to userbase: average voting likelyhood vs voting likelyhood in a specific module.

the more i think about it, the more i think a 10 point system might give the same information.

yes, if i think this discussion has led me to a new additional conclusion, it would be this:
give other users more insight in the votes. add refined search functions like voting probability, let us see how the votes are spread
say two mods with an average 8. one very small deviations (6-10), one where we see lots of 10 and a few 1s.
this kind of information doesnt make the system better per se, but allows users to better form their own oppinion about a certain mod, based on the votes that have been given.

visibility is the key. anyone want to agree on that?
               
               

               

                     Modifié par Gruftlord, 30 avril 2013 - 06:36 .
                     
                  

            

[Legacy_FunkySwerve]

               

Zarathustra217 wrote...

The way I see it, a major part of the problem is people's differing voting habits. Some people consider carefully whether to vote a 7 or 8 (assuming a ten point scale), whereas others throw 10s at everything they like (and sometimes, 1s at things they don't like).

A way to overcome this is to weigh each person's score against the past scores the person has given to other things (their voting habit). There's many considerations you could use to shape out this. A vote of 9 from a person with an average score-giving of 5.0 could weigh more than a 10 from a person with an average score-giving that was much higher (e.g. 9.0). A vote of 9 from a person that rarely votes outside the 3-7 range could weigh more than a vote of 10 from a person that has voted exclusively 1s and 10s.

Totally agreed. and sort of in line with what was discussed earlier. Average votes above mean (I think median would be a little resource-intensive to continually recalculate) should mean a lower weighting on high votes, and a higher weighting on low votes - and vice versa.

I also think that fewer votes means they should receive lower overall rating, as a rough guage of experience with the game and the voting system. This will help to downplay the effects of votes from people who just hop over to vote for their favorite PW, for example, without much in the way of context (and yes, would likely bring my PW's net vote down). The complication with that, however, is that if you publicize it, you engender an incentive to spam-vote, requiring either a layer of secrecy (not good for transparency, and thus credibility) or an additional layer of complexity, to code against spam voting (and that is one very complex layer, akin to bot-detection on a PW).

Funky
               
               

               
            

[Legacy_FunkySwerve]

               Thanks, MM, you saved me a lot of wasted typing. '> You got one thing wrong, though, or at least not as clear as I would prefer.

   Gruftlord wrote...

   You can add further complexity to the sytem, but that only makes it more complex to abuse it. you never get rid of the abuse, unless you introduce some attorneys and judges, you handle the abusers.

You wrote...

You don't need to get rid of the abuse - you just want to try to create ways to limit it.  And sometimes opening it up further to abuse is worth it because, on the whole, the system is much better.

Gruftlord is asserting that added complexity is pointless. That's just incorrect. He also asserts that it 'just' raises the complexity required to abuse it, which is more accurate, but inaccurately downplays the importance of doing so. Murder laws 'just' make it harder to get away with killing someone, but few people seriously advocate getting rid of them. I'm reminded of the stupifying gun control debate in the US, where the gun lobby's argument was essentially 'if you can't craft a perfect system with zero gun crime, there's no point even trying to reduce gun crime. QUICK LOOK AT THE SHINY OBJECT!' More succinctly, they want the perfect to be the enemy of the good, which is essentially what you're arguing for. Sure, all systems have flaws, but they can reduce them by adding layers of complexity - something I actually talk about in my above reply to Zarathustra when dealing with the idea of weighing the votes of infrequent voters less. The added layers of complexity actually accomplish something - shocking, I know. '>

Funky
               
               

               
            

[Legacy_Gruftlord]

               since some thoughts are going in a common direction now, let me add something to it again:

what if the new vault would NOT add a final score, what if there is no vault selected list of top mods,
but just a good search function, where different parameters can be filtered.
like average vote, total number of votes, number of downloads, votes per download, votes per download multiplied by average vote,
add refined options by width of vote spread, or some comparisons of average vote vs users average vote.
(lets say a mod in a hard battled field with critical downloaders. all vote 4 on average, mod in question gets a 6. factor 1.5 over voters average. yes, that would be a nice search option: factor of average voting score compared to (modspecivic) voters average score.)

let the users decide for themselves, what they consider important in their mods. in the end, the system is supposed to raise awareness and let users find what they are looking for. don't make the system complex, make the search function complex (and i'm not going to try to find a gun control analogy for that. there is none :-P)

edit: i'm not actually questioning the usefullness of a complex system. i'm questioning whether such a system can be established with the resources and manpower at hand. since as i explained, in my oppinion finding such a system is a time-consuming iterative process.

edit 2: a complex search function is, too. but users are less likelly to protest when the search function s changed on the fly vs their votes are changed/weighted differently on the fly. flexibility...
               
               

               

                     Modifié par Gruftlord, 30 avril 2013 - 06:54 .
                     
                  

            

[Legacy_Rolo Kipp]

               <icing his feet surreptitiously...>

Just a statement here: I am still following this thread with interest and still "staying loose" on final design.

However, I am pretty solid now of a multi-metric voting/rating system that will (I hope) incorporate the best of the ideas I'm seeing. A hefty portion of that will be subrosa to reduce "gaming the system" but will include:

• Useage stats for content that is accessed (with a 24hr granularity to reduce spam-updating)
• Download stats
• Comment activity (Hot Projects)
• A karma system with a floating threshold that weights votes according to various inputs.(Edit: influenced by the slashdot moderator system)
• Seed values taken from the V1 (with minimum weight). I'll discuss that if needed, but one of my goals is that no project will be harmed during this migration.

@ Gruftlord: I already have some pretty good search options. I may expand them later with SOLR search, but give it a try :-) (the VPP in my sig)

<...before walking the coals>
               
               

               

                     Modifié par Rolo Kipp, 30 avril 2013 - 06:59 .
                     
                  

            

[Legacy_MagicalMaster]

               

FunkySwerve wrote...

Thanks, MM, you saved me a lot of wasted typing. '> You got one thing wrong, though, or at least not as clear as I would prefer.

Actually, looking at it, I did get it completely wrong.  I was in a rush and read it as being something like "You can add complexity to the voting system (beyond thumbs up) but that makes it more abusable" and was pointing out that even if it IS more abusable (and/or more easily abused), it can still be worth it if, on a whole, it provides better information.

But he was actually talking about increasing the complexity in terms of making it harder to abuse (though his point was still inaccurate regardless).
               
               

               
            

[Legacy_FunkySwerve]

               

Rolo Kipp wrote...
(Edit: influenced by the slashdot moderator system)

Could you elaborate on this? Not at all familiar with it.

Thanks,
Funky
               
               

               
            

[Legacy_henesua]

               Funky,
regarding Karma:
http://slashdot.org/faq/karma.shtml

This tends to work well on "Answers" websites like Stack Overflow. I'm not sure how that will translate to the Vault, but I have liked it on "Answers" sites.
               
               

               
            

[Legacy_FunkySwerve]

               Got it, thanks. I was actually thinking of something like their anti-troll filter solution for the cure to the spam-voting issue I mentioned here, though I had in mind a 24-hour block.

FunkySwerve wrote...
I also think that fewer votes means they should receive lower overall rating, as a rough guage of experience with the game and the voting system. This will help to downplay the effects of votes from people who just hop over to vote for their favorite PW, for example, without much in the way of context (and yes, would likely bring my PW's net vote down). The complication with that, however, is that if you publicize it, you engender an incentive to spam-vote, requiring either a layer of secrecy (not good for transparency, and thus credibility) or an additional layer of complexity, to code against spam voting (and that is one very complex layer, akin to bot-detection on a PW).

Basically, the greater the time bar you set, the higher the cost to the
voter (they have to care enough to remember to come back) in terms of
time and attention. This is handy for stopping actual spammers, but also for regulating less questionable behavior.

For example: suppose your favorite PW wants you to vote for their PW, their chat system, and their legendary level system. If one of the goals of the voting system is to assess how much the voter cares about the thing being voted for, you may want to parse out those votes in some way, because the person coming to vote may care quite a bit about the PW, but have little knowledge of, say, the chat system, and vote for it more out of loyalty to the PW than as an evenhanded appraisal of the system itself. Requiring a span of time between votes cast would raise the cost of voting, and weed out the less-intense feeling voters. Of course, it might also irritate people, who are fond of voting and tend to regard it as a god-given right. '> Either way, this is an effective gauge of intensity of feeling for the content being voted on, at least.

Funky