How to change the order of the options over time


#1

A few of the posts on the “I dont recommend this option” thread brought up the issue of changing the order of the options as newer ones get released and older options become obsolete. Before we get into how to practically go about implementing such a feature, lets agree on what behavior we would like to see. To kick things off here are some characteristics I would like to see:

  1. Recently released or added options can climb the rankings without having to acquire more absolute votes than older options.
  2. More obscure but awesome options can compete with the popular ones.
  3. The rankings are resilient to groups attempting to astroturf the question.
  4. Experts can have their opinions heard.

What would you all like to see?


#2

Maybe content quality could also influence an option’s ability to rise to the top. An option with 9 votes, a promo video, a full summary, and full descriptions on pros and cons could be above an option with 10 votes but almost no content.

If users are able to influence ranking by putting down more information, maybe they will be motivated to contribute more. Developers could also be motivated by this to start adding higher quality content so their option is more likely to be at the top.


#3

Have you looked at reddit’s ranking system? I don’t remember the exact formula, but it’s something like:

submitTime + votes * scale
--------------------------
       currentTime

where times are POSIX (or something) and scale makes votes significant compared to those massive numbers.


#4

If we have upvotes and downvotes the Wilsons interval is probably relevant now.


#5

Can you elaborate on this? How do you imagine it?

Because if I’m coming in from Google looking for the best, I might not want to end up getting something that’s trending. Of course if I’m just browsing the site looking for cool new stuff then stuff that’s different might be more interesting to me.

There’s value in ordering items by absolute values. It represents inertia and stability. It’s clearly battle-tested if it has 10x the recommendations than something that’s awesome, but new (and might have holes that no-one has found yet). I know if I go with the battle-tested option, I might not be getting the latest hotness, but I’m definitely not making a bad decision. No one ever got fired for getting an IBM. Plus, the anti-recommend/dissuade button should work towards bumping obsolete options down.

Offering both sorting options could of course be a solution, but then I’m wondering which use-case comes first?


#7

tldr: most popular != best.

Sure can. I call it “the Samsung problem”. Based on sales, general polls etc the Samsung galaxy s4/5 is commonly the most popular Android smartphone. Thus if everyone was using Slant, it would probably be #1. However, I would say in the circles of people who actually do research and care, they would not have it #1.

So we might need to use other signals to rank an option that has lower total votes over one with more, if better people, sources etc back it up.


#8

I think there’s big cognitive dissonance between the idea that the popular vote is not always the best measure, and the idea that there should be absolutely no discrimination between users.

For the record, I come down solidly in favor of discrimination. We should have recognized experts, and their votes should count for more. This is a huge can of worms and I’m as reluctant as anyone to open it, but I don’t think we’ll ever fully shake the “internet poll” vibe without it.

Note that this too could be crowdsourced, by weighting someone’s votes based on how many people are following them. There’s the obvious danger of this simply compounding the problem, but on the other hand it’s a good way to avoid ill will and conspiracy theories – so I think, with reasonable weighting, it’s a good counterpart to a separate “expert” class of vetted category experts.


#9

I think when we do decide to tackle this issue, it won’t be with a single algorithmic input such as weighting votes by various methods but a lot of different inputs such as voters, source quality, age, vote velocity etc. I do expect “expert” votes to have a higher influence at some point though.


#10

So I was just thinking a bit about this, maybe it would be good to have a per question algorithm that can be tweaked depending on the question, instead of something site wide like Reddit. As some questions move very fast making it more pertinent to change the weight of votes with a lot of content coming in, for example Chromebooks. Unlike most other laptops that get a yearly refresh, Chromebooks come out at a pretty quick pace, making for a question that gets a lot of options added to it that would normally get buried under the higher voted older models. So date would be more of an issue on that question compared to something like "best programming laptops, which mainly only has yearly new models.

I would imagine it would be a lot more work, but we could have an algorithm that is site wide, as to cover everything at once but then have it to where it can be tweaked on a per question basis when needed.