How should the Karma system be designed?


#13

Repeatable, sure 20 and 60 another 500pt. I think it should be in addition to other bonuses that one may get for submission. There could probably other awards like 200pt for 30 pros which shouldn’t be a additive, so it likely isn’t going to be simple to add point senerios without consideration to other groups.


#14

Thanks for: Instead of just being able to say thanks, it might be useful to have a drop down menu so that you can specify why you are thanking them. For instance using images, video, or other not text media or maybe your answers are well written or well researched. These don’t necessarily need to be tied to points, but as I suggest in my next topic it can help sort users

Titles and Points: Karma points are good for a quick study of someone’s usefulness, but I would suggest (and perhaps this needs a topic all on its own) a system for Titles. For example if you provide Apple related answers, or add pros and cons to Apple related answers and you get a lot of votes for them you can get the title of ‘Apple Expert’. If you get a lot of ‘Thanks for: good writing’ you can get the ‘Well Spoken’ title. This way users can get a more in depth background of someone without having to delve too deep.

As for points I would suggest points for answering questions that have long gone unanswered. Otherwise those questions will just get pushed further and further back with little to no hope of getting answered.


#15

Another thought - have a karma budget per question.

  • Each question’s budget is determined by a value algorithm (e.g. number of views).
  • Instead of receiving quantified points for contributing, people take a percentage share of that question’s budget.
  • Each option, pro and con could equally be budgeted, a percentage of the question’s budget weighted by number of votes received.
  • Any contribution gets the same percentage within that area (e.g. someone who creates a pro gets the same percentage as someone who fills out the description for the pro, and the same as someone who corrects spelling errors in there). Whilst potentially unfair, it’s impossible to have a better system without intelligent intervention; since removing superfluous content or sparking an idea for content is potentially just as valuable as describing that idea in detail.
  • Contributions which don’t add value are rejected, meaning those contributors don’t take any karma percentage.
  • Given the budget is affected by the popularity / votes against that content, it’s in all contributors’ interests to make their answer popular / higher quality.

Still not a perfect system, but an alternate angle to approach the problem from which may yield interesting results.

Inspiration: http://www.slant.co/topics/1202/viewpoints/5/~what-is-the-best-karma-gamification-system-employed-on-a-large-community-site~hit-record-accord#3


#16

I don’t think we want to be tying points to popularity, fixing the spelling in a option that has no votes should get the same points as fixing one with 100.

Editing is a little tricking. I still don’t think it should be X points per edit. The main issue is identifying that the edits as valuable. The best I’m coming up with is for 10 edits Xpts, one of those edits will be placed in review and points are awarded if the edit is accepted.


#17

So here is what we think is a good & simple v1:

  • Level 1 Pro = TItle Only
  • Level 2 Pro = Title + Description
  • Level 3 Pro = Title + Description + Source
  • Added L1 Pro = 20
  • Added L2 Pro = 50
  • Added L3 Pro = 150
  • Added Option = 30
  • Added Source = 50
  • Added Description = 30
  • Edited anything = 10 (capped at 100 points/day)
  • Someone subscribed to your question = 20

We don’t think we can use votes as an input as there is pretty much no correlation to the quality of the user input. This system would highly encourage people submitting fully fleshed out Pros/Cons with sources. Thoughts?


#18

Seems ok to me. Not sure if starting at 10 is needed, but maybe it leaves some room for other scoring types.


#19

How about an ELO system supplemented with badges…

ELO would reward consistency and decrease separation between top and bottom users that is inherent in a completely vertical points system. A system like this would allow adjusting points distribution over time to change focus with less community friction.

Add to this a badge/flare/cosmetic system to reward things like consistency (add a “veteran” title beneath their name, or have their name in a different color), taking on new questions (give them a “desert warrior” badge), etc.

As a user it would go something like this. “Oh, one of the contributors name is just slightly orange instead of black like all the others, I wonder what that is all about.” Check the profile and see badges that say he’s helpful to newcommers, an expert in Python, thorough and explains things in a simple language. “Cool, so when this person talks about Python, I should listen”.

It gives contributors something to gain by being helpful and doesn’t create authority that’s infallible only because of insane amounts of karma points (see mrbabyman/karmanaut).

With this system you can get points relatively easy and understand what your strengths are as a contributor, but the real accomplishments come from being recognized by the community.


Btw, I’d avoid calling them badges. Better call them something like “known for”, “accomplishments”, etc. Less gamey sounding although that’s exactly what it is.


Also, this way contributors could be in charge of their domain, maintaining things that they are knowledgeable and updating them as they evolve instead of jumping around different subjects searching for points. It would reward deeper knowledge instead of wider overall knowledge.


#20

The issue with ELO (As far as I can gather) is the inputs need to be in the form of Wins/Losses as it’s a relative ranking system. Not sure we have the right inputs to do ELO?


#21

You are right, ELO doesn’t quite fit.

I guess what I was thinking was just a generally weighted system and ELO was the one that came to mind that used that concept.

How I see it:

You have a baseline of points, say 1k.

@ 1k you get a 5x multiplier to points you get (to incentivise new users)
@ 2k - 2x multiplier
@ 3k - 1x multiplier
@ 4k - 0.5x multiplier
@ 5k - 0.1x multiplier

And you lose a set number of, say, 100 points per day, though you can never drop below 1k.

Additionally, to get the first 1k points:
You register - 100p
You go through the tutorial (showing guidelines, teaching how Slant systems work, etc.) - 900p

Where points show how good you are currently, badges would add lasting accomplishments.
You can hold 3k+ points for a week? Badge
4k+ points for a week? Badge
For a month? Badge
Get most "thank you"s? Badge
Most contributions within a category? Badge
Etc.

Edit: I should add that the baseline should probably be a lot higher and the weights to kick in after certain number of criteria - so that an everyday contributor would start feeling the effects of weights after, say, a month and someone who only occasionally adds content would not feel them at all. So the weights basically start becoming a thing when reaching power-user amounts of points.


#22

hrmm, this is more complicated of a system than I was envisioning but I do see the benefits of this approach. One issue I have is we’re effectively penalising people not constantly using the site. As @johnlbevan said:

As a founder I obviously like the idea of having users compelled to constantly use the site…but I can’t help but think there might be some backlash? Thoughts?


#23

Also still thinking about:

In my v1 idea above I did give X points per edit, but maybe that is the wrong approach and we should do something along the lines of what @hethegreat suggested. Wikihow has a “patrol recent edits” feature where the mods can review and approve edits. Maybe we need something like that before we can include edits in the karma ranking.


#24

For an initial push I think a point cap for edits would be satisfactory. Leaving it off would probably be best to keep points mostly pure.


#25

Ok so we have a new concept. What if we used the point system I described above to calculate “Top contributors” for each option. So it’s less a race to acquire points, but more to go around and focus on a few options you care about.

The UI would be a little more badge based, and you could semi-compete foursquare style to be the main contributor for options/categories you care about. I think it’s a nice system as it gives people ownership over content they put effort into.

Profiles would highlight your best contributions etc and we might have leaderboards as well etc.

Thoughts on the concept?


#26

I agree. Using badges that you can work towards is a great way to keep people contributing. Encouraging them to own a topic and keep it updated will certainly help the site to stay current and give users a sense of accomplishment.


#27

Also, possibility to give out bounty (+100 points for example) for tackling certain questions.


#28

Ok we’re finally getting around to building a v1 for this. We eventually want to do a proper badge system, but in the interim we’re going to build out a more simple karma system. It’s going to work on the concept of “validated” contributions. You get Xpts for adding something, then X + Y when we get a signal that it was a good contribution. In general, X < Y. Here is what the initial system will look like:

  • Ask Question - 20 points - Another 10 points for every person who watches the question.
  • Add Option - 40 points - Another 40 points if at least one other user votes for it. Another 20 points if it’s the first option in the question.
  • Add Pro - 30 points - Another 20 points if at least one other user votes for it. Another 20 points if it’s the first pro for the option.
  • Add a source - 10 points - Another 10 points if it has a quote. Another 10 points if it’s the first source for the option.
  • Add the description to a Pro (that previously didn’t have one) - 10 points.

If the contribution you added is spam, when it’s removed the karma will also disappear. Thoughts?


#29

We’re also going to use this data to have a weekly leaderboard on the sidebar (replacing featured users).


Complied list of complaints about the current community experience
#30

Sounds good.

One question - with Pro (& Con), you get points for being the top Pro. That may change over time - do you lose the points when another pro bubbles up to first place / what happens if there’s a tie for first? Similarly for Sources.


#31

It’s not the top rated pro that gets more love, but the very first one that was added. We want to encourage/reward the people who get the ball rolling and add the initial content.


#32