Voting, Karma and Vaults - Oh my!

[Legacy_The Amethyst Dragon]

               

Rolo Kipp wrote...
...exactly zero donations (not that I'm really expecting them, understand)...

Creating and hosting this sort of stuff takes a lot folks, so let's push over a few dollars/euros/etc.
               
               

               
            

[Legacy_henesua]

               I agree with the need for Votes and Comments but in different measures than the "Vote" proponents. For me the comments are the most important thing. As someone who primarily works on scripts, I want feedback on my work. I want to know what didn't work for someone so that I can improve it. Criticism is more valuable than encouragement to me, but I'll take either. And its difficult to get that without comments.

Also helpful is having a discussion area associated with a project. I really like that "forum" link on the vault which we can use to point back here. I am disappointed that no one ever says much, but the potential is there and sometimes threads get lively. But the Neverwinter Nexus and the BSN projects enable you to create threads specifically tied to a project. I like this even more.

Perhaps one of the things about the voting on the vault that doesn't appeal to me is that the wide scale from 0 to 10 is ignored. I think most people are overwhlemed by the wide scale and simply vote at the ends. It is difficult for them to construct criteria or at least more effort than they are willing to apply when all they really want to do is let you know if they liked or disliked something. I think the Nexus gets around this entirely with a "like" feature - their "endorse" button. I think a 3 point scale could also work. Negative, Positive, In the Middle. But given the lack of thought most people put into voting (most do not vote), I think the simpler the better in the hopes that more people will participate rather than just downloading.

[edit]
In other words I am (more or less) with PHoD on this one.
For finding content, better means to organize them is the a more important thing in my view than user ranking. I think a TAG system as well as a CATEGORY system is essential. And as viewers gain status in the community their ability to create TAGs and CATEGORIES should come on line. I think that is where KARMA comes in.

I've seen KARMA work best on question and answer web sites. I'm still not sure how it would work at the vault, but I think its a good idea and like what I have heard with regards to implementing it so far.
               
               

               

                     Modifié par henesua, 22 avril 2013 - 06:35 .
                     
                  

            

[Legacy_NWN_baba yaga]

               I dont remember the last time where i not gave a 10 to be honest. But a 10 in this time (for me) means much more then just pure quality... as i almost only vote for modeled/ textured content.

It has more to do with how i judge someones work in terms of his enthusiasm and love for the game or the content he creates. Someone who just started to mod for nwn and you can clearly see he/ she spent some time (even hours) and thoughts behind his content shouldnt be judged like CODIs work. No one could just get a 10 for pure gfx or technical aspects of modeling except someone like BatintheHat who is the unbeatable champion of modeling/ texturing up to this day! his stuff is so good it´s timeless!

and a 0 is for guys like barry... LOL!

Well i was a bit offtopic again right? hmm. i like the vault system and i miss the days of the many comments and huge activities down there... yeah i really miss those days:crying:
               
               

               

                     Modifié par NWN_baba yaga, 22 avril 2013 - 06:43 .
                     
                  

            

[Legacy_painofdungeoneternal]

               Treat them as separate problems

1. First problem is content being there to begin with. This is simply preserving the Vault and what it is, and is of course the only current priority.

2. Voting is a completely different issue, and frankly there are merits to multiple systems. The criteria for any system should be based on some metric ( end user input, downloads, playtime, number of times playing, number of users at any one time ) which once understood becomes metrics ripe for abuse. But by having enough metrics, the abuser cannot manipulate every input well enough to not be noticed. The basis of any system should be usage metrics, which is the second problem.

3. The goal and purpose though is presenting the end user with the best content first of all. Higher quality stuff should be easier to find, a user should not have to dig thru ten pages to find project Q, CEP, or the other must have content. ( and this should not be subject to favoritism, SEO trick keyword spamming, or the like ) I'd call this scoring.

4. Secondary is feedback and support to the content authors, which is largely thru pat on the backs, and thru user feedback, not to mention discussion. I'd call this feedback. It should be judged on how many end users decide to make a comment of some sort, if there are 20,000 downloads and 5 comments, this is not working very well.

I personally don't think there is one solution for the last 2 problems, and any solution should support multiple solutions for 3 and 4. Problem 2, should support a lot of inputs, some of which hidden, and from multiple sources ( the VPP, the Nexus, etc ), it's more about tracking data and correlating it in a central location, and providing it to system who does scoring. Regardless we need to start at problem 1, then figure out how to get valid numbers on the usage metrics.

Problem 3 I'd like to allow it to be open enough to allow multiple scores. This seems more complex, but it's actual laziness on my part, i'd prefer to deal with part of the problem and make tools which enable others to deal with the much harder problem of how to rank content. I see manually tagging ranks on things, and also doing it based on metrics ( downloads, votes, etc ) as both being important ( and there are examples of each in the various search engines ).
               
               

               

                     Modifié par painofdungeoneternal, 22 avril 2013 - 07:19 .
                     
                  

            

[Legacy_Guest_Iveforgotmypassword_*]

               PLUSH HYENA of DOOM.. I couldn't agree with you more ! I really really hate the voting nonsense and both my modules have high scores and are in the hall of fame but I still find it horrible and it takes the fun out of releasing things for anybody in the world to enjoy. If it wasn't that in order to have comments you have to allow votes I would never ever have accepted them being any part of my modules.

Scrap votes ! Important is indeed a big archive of community content that continues to grow and headlining some things by giving them high scores or putting them in halls of fame or top tens can only damage that by putting off other people that don't get lucky to start with.
               
               

               
            

[Legacy_MagicalMaster]

               

Rolo Kipp wrote...

@ MM: that's something like what I was thinking

Er, I haven't posted in this thread until right now '>

I don't like the idea of a thumbs up/down either, for various reasons mentioned.

However, as Henusa somewhat just beat me to the punch on, I think part of the problem is that there are 41 possible rankings on the vault currently.  What deserves a 9.5 vesus a 9.25 or 9.75, for example?  And if we're trying to make distinctions between those, it's only natural to expect to see most people voting 9+ (because even at the 9+ mark there are five possible rankings).

I think either a four or five star ranking would well, something like...

1: Avoid
2: Average
3: Good
4: Excellent

and possibly

5: Amazing

I think people would feel better about voting a 3 or 4 if they're only 1 or 2 rankings "down" from the highest and are still portrayed as being good or better.  And even if you still see a lot of 5s, the main goal is to be able to indicate what modules are extremely well done, not try to arrange them in a precise order.

Hell, I'd probably argue that at most you should list modules alphabetically for each category of score.  So assuming we have 1-5 and module a has a 4.1, b has a 4.9, c has a 5.0, d has a 4.3, and e is a 4.0, it would still be...

a
b
c
d
e

Because the categories would be 4.0+, 3.0+, 2.0+, and 1.0+.  Make people go to the individual pages to see the actual average so they can read the description and comments.  Don't have the average available outside of the page itself.
               
               

               

                     Modifié par MagicalMaster, 22 avril 2013 - 07:20 .
                     
                  

            

[Legacy_Claudius33]

               

PJ156 wrote...

Hearty agreement from me in this respect. I don't create for vote nor comments but I do appreciate them for two reasons:

One - A vote tells me that one person, just one person per vote has actually played my work and thought about it. A download does not tell me that. My module took five years. I don't mind that but I want to confirm that someone has played it. A comment does that too but a vote is more informative to me, you can sort modules by votes you can't sort them by comments.

Two - A vote and a comment show that the person who downloaded my work has the decency to give up fifteen minutes of thier time, to pay back my many hours, by telling me what they think of it. I don't mind if they think it sucks or they love it, if they tell me why it can all go into the melting pot.

I second PJ for the exact same reasons. I am very happy when a player writes a mini review of my mod or writes he just registered to comment it. As a moder you are eager that other moders use your creations and that players play your modules. Above you can’t wait for getting feedbacks, of course positive '> , but also constructive criticisms.

No matters the type of vote, as the Vault today, A to E, stars split in graphics, fun, ... I probably would prefer A to E or stars, just a personal opinion. I am not a big fan of classification, 9.xx vs 9.yy (damn! I did it in my news ).

So I would prefer simple lists of modules/creations with a rate (4 stars, 3 stars, or whatever) rather then a ranked list. Of course moderation would be welcomed because I totally understand why Tsongo advises to scrap votes. A very few (immature or irrespectful :innocent:?) people can kill any willingness of submitting something. I could live with no voting all, however I still feel that's a way to reward quality submissions.

As other have written, categories/tags are interesting for modules eg. length (gameplay hours), setting (D&D, historical, SciFi, ...), story driven / sand box, hack & slash, riddles, thievery, ...

However, IMO, what drastically matters is the visibility. Therefore, whatever the voting/rating system, I am a huge fan of rotating lists  on the front page, so every module/creation gets a chance :

• New modules or creations (current year + last year)
• Highly rated modules or creations (curent year + last year)
• Highly rated modules or creations (old goodies)
• Any modules/creations whatever the rating and the submission date.
• List by categories.

In addition, a topic open to mention  â€˜underrated’ modules or creations should be opened.

Hope it helps.
               
               

               

                     Modifié par Claudius33, 22 avril 2013 - 08:03 .
                     
                  

            

[Legacy_FunkySwerve]

               

MagicalMaster wrote...
Hell, I'd probably argue that at most you should list modules alphabetically for each category of score.

Allow me to introduce the latest iteration of my PW,  ________AAAA Aaron's Higher Ground PoA Legends!

Phonebook syndrome, no thanks. '> Either a raw list by score, or ORDER BY RAND() the display order each display.

Funky
               
               

               

                     Modifié par FunkySwerve, 22 avril 2013 - 08:31 .
                     
                  

            

[Legacy_Gruftlord]

               a simple like/endorse system would be the hardest to abuse. you can only like a contribution and never dislike it.
then add search functions based on number of likes, number of downloads and lumber of likes per download.
also the less you activelly support/spotlight mods that gather high ratings (irrespective of the rating system) the less likely the system will be abused.

at this time and age i cast my vote for the system, that is least likely to be abused (or that is the least abusable) by random internet trolls and spam bots.

my two cents as a mod user
               
               

               
            

[Legacy_Tarot Redhand]

               I have seen a number of different proposals for scoring systems on here and they all have the same small but significant problem. They are all based on an odd-number. This is a something that many courses both academic and vocational teach you to avoid because people will tend to pick a middle number if you give them a choice. Better to use an even number be it 4/6/8 or whatever.

TR
               
               

               

                     Modifié par Tarot Redhand, 22 avril 2013 - 08:48 .
                     
                  

            

[Legacy_MagicalMaster]

               

FunkySwerve wrote...

Allow me to introduce the latest iteration of my PW,  ________AAAA Aaron's Higher Ground PoA Legends!

Point taken, I didn't consider people trying to get to the "top" of the list (since the "top" would have no inherent meaning).

I was trying to see if we could get away from the idea of a "top 10" list or something, come up with a system where people can generally see what modules are considered to be roughly on par with each other and then they look at the actual modules.  Which also reduces the impact of a module being rated 9.9 versus 9.8.

FunkySwerve wrote...

ORDER BY RAND() the display order each display.

Would be fine as long as the order changes each visit or something and it's made clear that it is random.

Tarot Redhand wrote...

I have seen a number of different proposals for scoring systems on here and they all have the same small but significant problem. They are all based on an odd-number. This is a something that many courses both academic and vocational teach you to avoid because people will tend to pick a middle number if you give them a choice. Better to use an even number be it 4/6/8 or whatever.

Hey now, I gave a system for four!*

*with the possibilty of a fifth if desired, so I'm still semi-guilty.

That said, I don't think what you're saying is entirely accurate - if they played a module and really liked it, they aren't going to vote for a 3 out of 5 because 3 is the middle.  If you asked them if they thought it was a 3, 4, or 5 they might pick 4 because it's the middle of those, though.  Framing is important.
               
               

               

                     Modifié par MagicalMaster, 22 avril 2013 - 09:16 .
                     
                  

            

[Legacy_Tarot Redhand]

               @MagicalMaster, I didn't put names to it and I didn't say all. All I am saying is that it is something to be aware of. One thing I think definitely needs to be given the order of the boot is fractional scoring, So no more 9.5, 9.6 etc.

TR
               
               

               
            

[Legacy_MagicalMaster]

               

Tarot Redhand wrote...

@MagicalMaster, I didn't put names to it and I didn't say all.

Tarot Redhand wrote...

I have seen a number of different proposals for scoring systems on here and they all  have the same small but significant problem.

>.>

Tarot Redhand wrote...

All I am saying is that it is something to be aware of. One thing I think definitely needs to be given the order of the boot is fractional scoring, So no more 9.5, 9.6 etc.

Indeed.
               
               

               

                     Modifié par MagicalMaster, 22 avril 2013 - 10:09 .
                     
                  

            

[Legacy_ColorsFade]

               

PLUSH HYENA of DOOM wrote...

IMPORTANT:- A whopping, dirty great Archive of Community Content (hopefully continuing to grow).

UNIMPORTANT:- Pretty much everything else.

I just want to second this before I go on to write about voting... Preserving the data seems paramount. 

Now, about voting... 

I can only tell my side of the story, as a user. 

When I decided to "reintroduce" myself to the NWN2 world, I played through the OC and about half of MotB before I started to get the itch to take a look at the Toolset and do my own thing. But I decided that before I embarked on learning the Toolset, I wanted to see what kinds of content other people had created with it. But I have limited time, so I didn't want to play just anything, or to play the modules with the most downloads. I wanted to play the stuff that the community, as a whole, generally considered the cream of the crop. 

To that end, the voting system was invaluable to me.

Seeing the Hall of Fame tags was great; It gave me a really concise idea of what was considered good, quality content. Seeing the high aggregate ratings also helped. Seeing the Top lists also helped. 

It was through all of those votes, awards and whatnot that I was able to narrow down my first downloads to a few select modules that seemed to best represent the "best quality" work of the community. And from there, I was able to learn, play, have fun, and generate my own ideas for campaign. 

A site like Vault for NWN2 content is a lot like shopping on Amazon. There is a reason rating systems exist - so users can make efficient use of their time in trying to find what they want. Yes, comments are ultimately the most valuable in this regard, but that's data that requires a serious time investment to parse. A simple five-star rating system allows a user to make efficient use of their time by consolidating the data. 

If it were me, I'd have a rating system in place. And if possible, I'd add additional tags to the content, for things like Hall of Fame, etc. I think it's still worthwhile and relevant to let someone know, "Hey, this content was moved to Vault 2.0, but back in the day, this module won a Hall of Fame award in 2007". To me, as a consumer, that's useful information. It lets me know that at one point in time, that content was considered the highest quality. 
               
               

               
            

[Legacy_Tarot Redhand]

               Oooops, Sorry. Well I thought I hadn't. Getting senile. Bad Kitty.

TR