Author Topic: Voting, Karma and Vaults - Oh my!  (Read 4276 times)

Legacy_Rolo Kipp

  • Hero Member
  • *****
  • Posts: 4349
  • Karma: +0/-0
Voting, Karma and Vaults - Oh my!
« on: April 21, 2013, 11:41:44 pm »


               <on the eve of an age...>

There are a lot of things The Vault has done well and a couple it has done great.

By now, I think I've shown my affection for the old gal and so hope this thread will not be construed as criticism or the dreaded "vault-abuse" so many wizards are known for.

I want to talk about voting and karma and how we sort things on Vault v2.0 (V2).

The old way - a ten point comment vote by registered users - failed for several reasons. The user registration was a joke (your email is bite@me.com? Srsly?). The voting was unmoderated. The voting was unlimited. the voting was unfocused.

But that vote, and a download count, and a time-since-first-upload is what determined ranking, Hall of Fame status and some other stuff. Basically, it determined how much exposure a project got, in what lists, with what highlighting.

There was some discussion about this before on the NwN2 forums (persevere, there's a lot of other subjects in there, also), and recently I had a pretty good PM with MagicalMaster :

[ edited ]
MagicalMaster
Seems to be what looks like a spam comment on my module's page
But it's definitely not the usual spam. I tried adding the last part of his link there manually to the end of the Bioware boards and it doesn't lead to anything. I'm hesitant to actually click the link since it might somehow to redirecting elsewhere (though offhand I don't see any obvious signs of it).
Something still seems fishy for someone to vote on my module 18 minutes after registering and claim he has a module up or something as well.   

Rolo Kipp
That fellow's a regular troll. His latest email for registration, for example, is asfasfaf@duudu.com.
He thinks up a new name and registers and posts something infantile. I blacklist the new name.
On the new vault, I'll just blacklist the IP, but then you need a valid email to register anyway :-P
Anyway, tell me when he returns :-/ I'll hammer him again.

MagicalMaster
But...but...his vote of 10! Surely getting my module a higher score is more important than the integrity of the vault?

Rolo Kipp
Tell ya what I'm gonna do, ya get twenty four o' them votes -thet's jest one shy a the whole kit an' kaboodle -an' I'll spot ya the last vote fee free!How's them apples? ;-)


MagicalMaster
You, sir, are a [ unprintable compliment edited out by shy wizard ].

Rolo Kipp
Well, a [ edited ] at least :-)
Seriously, though, I'm hoping to work out a more effective and rewarding voting system on Vault 2.0.

MagicalMaster
Yeah, it is kind of annoying having over 200 downloads and two whole votes/comments. One of which was from someone else in the ABC, meaning only one person who found it on the vault has voted.
I get that it is technically only about half of a module, but it is still 2-3 hours of play and you'd think people would have SOME opinions on it - hate or love.

Rolo Kipp
One of my thoughts on voting (just toying with it) is a karma system. You start with a karma pool of a hundred. You don't lose karma by voting, but you can only vote your karma. If you get votes for something, it increases your karma pool, allowing you to vote more. If you never get votes, your limited to the votes you can cast.
This would reward frequent contributors and quality contributors and give a vested interest in receiving votes.Thinking about it.

MagicalMaster
More seriously, though, I don't expect everyone to be interested in an action-oriented level 40 module - but you'd think that people who bother to download it would be interested in it, if that makes sense. You could say I'm not annoyed by people who look at it and say "Nah, not really interested" but it is frustrating when people go "Yeah, that looks cool" and then don't say anything.
I'm confused about your proposed karma system. You say you start with 100 and do NOT lose karma by voting. In that case, you can always vote 100 on everything since you never lose anything by voting. What's the point?I also would be strongly against any system that forced players to submit something in order to get karma points - not everyone (indeed, a minority) will create stuff in the toolset.

Rolo Kipp
Re Karma: you don't "spend" karma... That's the finite number of votes you can give. Vote ten point on ten projects... Or five on twenty. The authors of those projects has their pool increased and can vote more. So how many total votes you have out there helping *other* people is a basic dole + received (merit) karma.
All this is a thought experiment at this stage, but the current voting scheme is rather pointless, IMO.

MagicalMaster
So if I vote 10 on 10 projects, I am down to 0 remaining karma (or maxed at 100/100 karma if you prefer) and cannot vote on anything else until someone votes for my stuff?
What's the point of that, *especially* for people who play modules and don't create them?

Rolo Kipp
The value is that the votes mean something. If you only have a hundred points to distribute, you're not gonna slap a ten on everything ever submitted. You're going to think about where to put them and maybe even adjust some over enthusiastic reviews down to give you the mojo to pump up something really good. But, just like gold and other valuables in a campaign, there has to be multiple faucets & sinks and limiting good stuff to CC Makers is pretty short-sighted.
So the karma points would have to be pertinent to any member of the community, whatever their role.The value of each point is increased both for the seller who has a limited supply and the buyer who always needs more. 
Perhaps calling them votes is contraindicated. It comes with far too many connotations and the implication of a ten point (or hundred point) scale.
Lots to consider and I'm always willing to listen :-)
Do you mind if I quote the vote-pertinent parts of this Convo?
It'll give give me a start to the thread I want :-)

MagicalMaster
Feel free to quote any of this, sure.
More things to think about:
1, some people play simply play mods that are better than average simply because they choose mods well - saying that they can't rate every amazing mod at a high score doesn't make much sense.
2, even if you did do something like that, it would cause people to game the system. Like I throw up a module with one monster and you say "10/10 would play again!" to give me points to vote on YOUR module and give it a 10.
3, in this system, how do people who simply play modules get more points to distribute?
I'm not saying that the inflated vote system on the vault that has mediocre mods getting scores of 9+ is a good thing. But let's make sure we don't make the situation WORSE than it already is, eh?

MagicalMaster
The good news!
I got a vote and comment on my module (http://nwvault.ign.c....Detail&id=6386) that started with "This module is incredible!"
The bad news!
It's the troll again.

Rolo Kipp
Heh. Want to keep the vote or take out the garbage?

MagicalMaster
KEEP IT! KEEP IT!
Not only do I get a vote of 10, I also got credit for making the final mod in the Prophet series!Damnit, you already deleted it.

As you can see, I'm far too funny to be in charge of this :-)

Er, I meant, I'm trying to think of alternatives.
Researching alternatives brings me to the Slashdot moderation method, which really looks good to me.

If you think about it, we, as a community, are judging and ranking two separate things. The value (to *us*) of a given project (mod, hak, whatever) and the value (to *us*) of an author (of a project, post, comment, tutorial, editorial, amusing screenshot...).

We have the potential with Neverlauncher of tracking how often (for those willing to participate) a project (not just mods, but any project looking for an update) is used (when it goes online to check for updates). This metric makes far more sense to me than arbitrary votes for ranking projects.

With a karma system similar to Slashdot's in place, this makes much more sense to me than a forum with no (or absent) moderators. And even newbs can see immediately how much weight a given poster has in *this* community.

As I told MM above, this is a thought experiment at the moment, but in the shift from IGN to V2, I want to do *something*  better than Kickbong Voting :-P

Give me ideas, questions, concerns, solutions, links... 

<...and the dawn of another>
               
               

               


                     Modifié par Rolo Kipp, 21 avril 2013 - 10:47 .
                     
                  


            

Legacy_Tarot Redhand

  • Hero Member
  • *****
  • Posts: 4165
  • Karma: +0/-0
Voting, Karma and Vaults - Oh my!
« Reply #1 on: April 22, 2013, 12:11:12 am »


               The problem with the karma system that I can see is that it will make some people even less inclined to vote. I'm talking about those people who don't actually make stuff. Register and get so many karma points to spend, now spend them wisely because you won't get any more... I know at least one person who almost never (no I won't tell who, but it was just the once) makes anything but is still active. They get to have so many karma points and then...

TR
               
               

               
            

Legacy_Rolo Kipp

  • Hero Member
  • *****
  • Posts: 4349
  • Karma: +0/-0
Voting, Karma and Vaults - Oh my!
« Reply #2 on: April 22, 2013, 12:18:54 am »


               <nods vigorously...>

Oh, I've dropped the "only 100 points" thing. I'm much more impressed with the Slashdot style vote up/down thing and allowing users to set their own thresholds.

So it would be only 1 vote per post, not per user.

Sorry I didn't make it clear... this is an evolving process.

<...and shakes his head>
               
               

               
            

Legacy_WhiZard

  • Hero Member
  • *****
  • Posts: 2149
  • Karma: +0/-0
Voting, Karma and Vaults - Oh my!
« Reply #3 on: April 22, 2013, 12:19:29 am »


               So if I had 100 karma, I could vote 1 on 100 items and really skew the overall reputation of contributors?  I'm not sure if the voted points (or lack thereof) should be tied to an average if there is a limit based on points allocated.

What might be better is to give out status rankings to good vault members so that their rating is weighed more than other raters.  For example let's say I am a newcomer.  I would have a tier 1 status.  My votes would only be influential on content not heavily voted on.  The more I vote and play would aid me in getting up to tier 2.  Tier 3 would be reserved for those who contribute and tier 4 for those whose submissions get good votes.  Tier 5 is hall of fame.  If there are a significant number of votes from a higher tier than the lower tier votes are ignored or have reduced weight.
               
               

               
            

Legacy_Rolo Kipp

  • Hero Member
  • *****
  • Posts: 4349
  • Karma: +0/-0
Voting, Karma and Vaults - Oh my!
« Reply #4 on: April 22, 2013, 12:30:03 am »


               <shifting his...>

Something like this?

This module only recalculates the voting results, wont change database records.
So it's quite safe to install and uninstall.

Features:
Set voting weight on roles

Usecase:
Votes of good authenticated users should have more weight.
Keep anonymous and new users from voting flooding, more or less.


Or perhaps a hybrid of vote up/down & weights? Or perhaps the Apple-like "useful - yes/no/maybe"?
I like that the weight is applied to the results and not a part of the data.

Edit: Also, am I correct that we'd pretty much all like *something* other than the ten point vault-style vote?

<...weight>
               
               

               


                     Modifié par Rolo Kipp, 21 avril 2013 - 11:32 .
                     
                  


            

Legacy_Tarot Redhand

  • Hero Member
  • *****
  • Posts: 4165
  • Karma: +0/-0
Voting, Karma and Vaults - Oh my!
« Reply #5 on: April 22, 2013, 12:31:38 am »


               Actually (I know it will be heresy to some) I think we need to drop points altogether. I'll clarify. On the users side instead of pick a number you have the following categories "Avoid, Poor, Satisfactory, Better, Good, Very Good, Excellent and Superb". This removes numbers from the users side, much to the relief of some (aka its good but is it a 7 or an 8?).  Anyway it's just a thought.

TR
               
               

               
            

Legacy_Pstemarie

  • Hero Member
  • *****
  • Posts: 4368
  • Karma: +0/-0
Voting, Karma and Vaults - Oh my!
« Reply #6 on: April 22, 2013, 12:35:32 am »


               Why not allow Vault 2.0 users to link their Vault 2.0 account to Facebook and use a simple like/dislike system that posts a link to the work to Facebook if the voter so desires?

Social media is such a prevalent part of life - why not exploit it?
               
               

               


                     Modifié par Pstemarie, 21 avril 2013 - 11:36 .
                     
                  


            

Legacy_Rolo Kipp

  • Hero Member
  • *****
  • Posts: 4349
  • Karma: +0/-0
Voting, Karma and Vaults - Oh my!
« Reply #7 on: April 22, 2013, 12:36:32 am »


               <thinking up...>

It looks like Slashdot does something like that Tarot. They specifically mention "Flame bait (-1)" and "Redundant (-1)" categories. I guess you rate it with a radio button...

<...some really descriptive titles>
               
               

               
            

Legacy_Pstemarie

  • Hero Member
  • *****
  • Posts: 4368
  • Karma: +0/-0
Voting, Karma and Vaults - Oh my!
« Reply #8 on: April 22, 2013, 12:37:47 am »


               While they're at it they could also "tweet" a link to the Vault 2.0 entry, etc.
               
               

               
            

Legacy_WhiZard

  • Hero Member
  • *****
  • Posts: 2149
  • Karma: +0/-0
Voting, Karma and Vaults - Oh my!
« Reply #9 on: April 22, 2013, 12:38:37 am »


               

Rolo Kipp wrote...

<nods vigorously...>

Oh, I've dropped the "only 100 points" thing. I'm much more impressed with the Slashdot style vote up/down thing and allowing users to set their own thresholds.

So it would be only 1 vote per post, not per user.

Sorry I didn't make it clear... this is an evolving process.

<...and shakes his head>


The +/- system while good at distinguishing good from bad, does not have enough range to compare good submissions.  If I had a submission with +56, and another person had one with +112, then is their's better than mine? Assuming that all those that voted gave a + since their was quality in both of our submissions, his 112 better reflects good marketing while my 56 shows only a decent level of advertising.  (Granted that 56 and 112 votes are quite a large sum for our current system).  I think the 10 point system works well if one was forced to vote (if they hadn't already the default setting at 5 should they try to ignore even this) when they initially posted with a comment or concern.
               
               

               
            

Legacy_Rolo Kipp

  • Hero Member
  • *****
  • Posts: 4349
  • Karma: +0/-0
Voting, Karma and Vaults - Oh my!
« Reply #10 on: April 22, 2013, 12:39:07 am »


               <face-...>

Easy to do, actually. Except I, personally, do not have a facebook account :-P Or twitter, myspace, uh, others...

BSN & the various vaults are enough of a time-sink... but I digress :-)

Facebook integration is on the table unless I hear some strenuous objections.

<...palming>
               
               

               
            

Legacy_Tarot Redhand

  • Hero Member
  • *****
  • Posts: 4165
  • Karma: +0/-0
Voting, Karma and Vaults - Oh my!
« Reply #11 on: April 22, 2013, 12:50:34 am »


               No objection to fb but beware. It seems that there are one or 2 innocuous looking apps on there that can multiply the spam you get in your email something dreadful. However I am against a nero-esque thumbs up/down like system for voting on submissions.

TR
               
               

               


                     Modifié par Tarot Redhand, 21 avril 2013 - 11:51 .
                     
                  


            

Legacy_Rolo Kipp

  • Hero Member
  • *****
  • Posts: 4349
  • Karma: +0/-0
Voting, Karma and Vaults - Oh my!
« Reply #12 on: April 22, 2013, 01:04:56 am »


               <making a...>

'Image

<...face>
               
               

               
            

Legacy_Rolo Kipp

  • Hero Member
  • *****
  • Posts: 4349
  • Karma: +0/-0
Voting, Karma and Vaults - Oh my!
« Reply #13 on: April 22, 2013, 01:15:44 am »


               <stepping back...>

Just to be clear: I am talking about a different metric for *projects* and *posts*.

Projects would be ranked in a relative standing based on actual use reported. User votes will be with their game client. :-P
Edit: Might not be a bad idea to give one vote per download/day/ip as well. The use metrics would still dominate, but that gives a voice to forest creatures who only come into town once a month to buy toilet paper and download stuff...

Initial weight will be given to the current votes on the vault (when migrated) which will result in the old stand-bys starting at the top (where they are fossilized now) but moving downward as others are used.

Posts would (I'm proposing) use a Slashdot style community moderation system. Probably based on label feedback ("Flaming" "Pointless" "Redundant" "Meh" "Interesting" "Relevant" "Great" "Wow!"). Users would use the "karma" rating this produces (with weight, I do like that idea) and set their own threshold of what they want visible and the visible stuff will be marked in some way to indicate ranking.

Or Posts might be unranked entirely (current system). And leave it entirely up to this old wizard to moderate everyone's posts. Fair warning!

Edit: Re-reading that, it sounds like my mind is made up... far from it! Talk, I'll listen :-)

<...and refocusing>
               
               

               


                     Modifié par Rolo Kipp, 22 avril 2013 - 12:22 .
                     
                  


            

Legacy_FunkySwerve

  • Hero Member
  • *****
  • Posts: 2325
  • Karma: +0/-0
Voting, Karma and Vaults - Oh my!
« Reply #14 on: April 22, 2013, 01:46:30 am »


               To be clear, do you actually have authority to change the voting system? Last time I spoke to Maximus about the problems of the current system, he said he didn't have authority to change it because it was done by IGN standards.

Has something changed? Is Vault 2.0 a new community project, rather than an update to the actual Vault?

Either way, what I told Max was, user verification issues aside, that things should be ranked by a combination of votes and values - that is, a new submission with 10 10.0 votes should not be higher than an older sumission with 20 10 votes and 5 9 votes, just because it had a higher average number - the lower sample set means a less accurate valuation, even before accounting for the way I've seen people go about getting those first 10 votes. Instead, rank by total value - that way the 25 vote submission above would have 245 points, and the new one only 100. This, of course, favors older submissions, but it's still quite a bit less distorted than current possible outcomes.

Of course, if you want to talk about methods of weighting votes, you could give additional weight to people who actually have submissions (and therefore are likely to have some idea what they're talking about). Of course, that would engender a spam-submission problem, so you might want to limit that to HoF status submissions in the given category.

Funky