Silk Road forums

Discussion => Silk Road discussion => Topic started by: Dread Pirate Roberts on July 23, 2011, 06:19 pm

Title: Seller ranking and feedback overhaul
Post by: Dread Pirate Roberts on July 23, 2011, 06:19 pm
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hey gang,

Our old feedback system, as great as it was, was overly simple and had started reaching it's limits lately.  Inspired by many of your suggestions for improvement, we've done a complete overhaul that I hope you will find more informative and help you determine the best sellers to work with.

Instead of using a simple average of a sellers feedback scores, now a weighted average is done.  The weights are determined by several factors.  They are, in order of importance:
1) How active the buyer is in rating other sellers
2) How old the rating is
3) How much was spent on the transaction

also, multiple reviews from the same buyer count for less and less.  In this way, a seller's feedback score is dominated by recent, unique feedback from active, experienced buyers.

In addition to this, we've developed a ranking system that takes into account not only the average feedback for the seller, but also how experienced they are so you can quickly find the best sellers who have both good feedback and lots of it.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.14 (GNU/Linux)

iQEcBAEBAgAGBQJOKxCgAAoJEAIiQjtnt/oljWgH/iCP5ecS442qbWYEWhQ66m74
y+JKIo4UXYg9XQsSDEUUzsRXAzDtCdlKiDQdJ2tzxPOl3AAcQw7eTScAmB+ANWEv
Lvt0NHe/mAHnD1zKgk1g3eRadSFq+m8N44/D0sOkySf4naSHzAhaiu8dttXvSxvZ
ljzBBsQW+B47ZsKKagSAg5zcxQAXM+XzNKfujVw7ckNKXh7ZKzIlieQK3Zf2Q6hM
ALLBmyHTY2AbTQeZGXTm/N+mgJ9ujczq6vyT96PO2CwkP4oxzkaPFYtWoqq0jDk6
X+rySRIjFT6BfYzZvrlBbKhCvcRldgdtfF3P3EcmSW/j4Nxja7VMFiugQYlo6Rw=
=c7pz
-----END PGP SIGNATURE-----
Title: Re: Seller ranking and feedback overhaul
Post by: listentothemusic on July 23, 2011, 07:20 pm
I love you.

PS, I dropped from a 99 to a 98 in the new system :(
Title: Re: Seller ranking and feedback overhaul
Post by: phubaiblues on July 23, 2011, 07:40 pm
I appreciate the improvements, but I do wish there was some way to have a note if a seller is in bad repute, or you've been notified of scams.  There is a seller on there that seems to have conned more than one newbie into selling outside of escrow, and then rips them off. 

But in any case, thanks again for constantly improving around here.  As always, I do wish there was a way to get newbies to forums so they could find out more about the sellers--and buyers--before they purchase...
Title: Re: Seller ranking and feedback overhaul
Post by: v86 on July 23, 2011, 11:10 pm
Check how many sales you have listentothemusic.  I noticed they have me down for an extra sale that doesnt exist.  I have 19 sales and they have it down as 20 which really fucked up my ratio.  I have only 1 4 rating the rest are 5.  The math is off.  Check yours out and let us know as well as silk road admin.
Title: Re: Seller ranking and feedback overhaul
Post by: nomad bloodbath on July 23, 2011, 11:17 pm
Oooo i hope this really improves the feedback system, cant wait to do my first new feedback. :D
Title: Re: Seller ranking and feedback overhaul
Post by: Rook on July 23, 2011, 11:25 pm
Hurray!  I for one LOVE the new feedback system!  It makes way more sense
Title: Re: Seller ranking and feedback overhaul
Post by: AlvinPowers on July 23, 2011, 11:28 pm
Very nice and needed update, but I also think we need some way of featuring newer and small time sellers. Otherwise we might end up with an oligopoly where its near impossible for smaller sellers to sell anything and reach the front page. Maybe we can have a category for this?
Title: Re: Seller ranking and feedback overhaul
Post by: Dread Pirate Roberts on July 24, 2011, 12:11 am
Check how many sales you have listentothemusic.  I noticed they have me down for an extra sale that doesnt exist.  I have 19 sales and they have it down as 20 which really fucked up my ratio.  I have only 1 4 rating the rest are 5.  The math is off.  Check yours out and let us know as well as silk road admin.

This is due to transactions that have finalized but the buyer has yet to leave a review.
Title: Re: Seller ranking and feedback overhaul
Post by: OpticalInfinity on July 24, 2011, 12:14 am
Oh, this is tits xD
Thanks, SR!
Title: Re: Seller ranking and feedback overhaul
Post by: joeblow2 on July 24, 2011, 06:00 am
MOST EXCELLENT!

For the time that SR has been running "live" online, it has already evolved in an amazing fashion and at a breakneck pace.  I actually find that very exciting. :D  How cool to participate in the first sophisticated form factor for any future underground enterprise.  Nifty.   8)

Possibly the math needs to be checked/tweaked, but this type of weighted averaging system will produce, over time, the best possible portrait of a "lifetime" positive feedback percentage for the sellers.  And, it will doom someone like Barry Sneaky, whether he has 12 ratings or 500.

Only one situation remains: Barry Sneaky and his ilk have to somehow be removed before they scam any more newbies. 

@HappyTree-no need to learn more now ma'am.  This just took care of any issues you were worried about.  YAY!
Title: Re: Seller ranking and feedback overhaul
Post by: smokeourbowls on July 24, 2011, 07:12 am
I would love to see feedback be editable from the buyers side, in-case a seller receives it later and needs to fix their mistake. I have a customer who requested that this week, he jumped the ball and apologized.
Title: Re: Seller ranking and feedback overhaul
Post by: v86 on July 24, 2011, 04:06 pm
Check how many sales you have listentothemusic.  I noticed they have me down for an extra sale that doesnt exist.  I have 19 sales and they have it down as 20 which really fucked up my ratio.  I have only 1 4 rating the rest are 5.  The math is off.  Check yours out and let us know as well as silk road admin.

This is due to transactions that have finalized but the buyer has yet to leave a review.

I dont understand how this happens if feedback is posted when escrow is released  . Also If thats the case then why is it that my sales numbers match the number of comments I have?  And both those numbers are different than yours by 1.  I know you can change your comment but how do you bypasst commenting at all when escrow is released?  How long does a customer have until they have to post a comment?  Thanks for listening.
Title: Re: Seller ranking and feedback overhaul
Post by: v86 on July 24, 2011, 04:22 pm
If you say a customer released escrow but didn't comment wouldn't then the number of sales and comments differ?  Mine are the same.  The only difference is in your count by +1 .  How does a customer go about releasing escrow without commenting?  How long does a customer have before they need to comment?  Thanks for your time SILKROAD  I just dont like my rep suffering for no reason at all.
Title: Re: Seller ranking and feedback overhaul
Post by: hybridmike on July 24, 2011, 07:16 pm
I would love to see feedback be editable from the buyers side, in-case a seller receives it later and needs to fix their mistake. I have a customer who requested that this week, he jumped the ball and apologized.

you can do this already... the buyer goes to his account tab, then click feedback... he can edit it there
Title: Re: Seller ranking and feedback overhaul
Post by: smokeourbowls on July 24, 2011, 07:33 pm
SICKKK thanks mike!
Title: Re: Seller ranking and feedback overhaul
Post by: GreenCo on July 25, 2011, 03:56 pm
Thank You Silk Road.
Title: Re: Seller ranking and feedback overhaul
Post by: theonetheonlyandy on July 25, 2011, 05:49 pm
I wonder whos seller number #1??
Title: Re: Seller ranking and feedback overhaul
Post by: OpticalInfinity on July 25, 2011, 08:42 pm
I wonder whos seller number #1??
Surprise! It's 3jane! Who else would be #1? x)
Title: Re: Seller ranking and feedback overhaul
Post by: Kind Bud on July 25, 2011, 10:54 pm
Yay 3Jane!
GreenCo (currently #2) is catching up and getting close.
http://ianxz6zefk72ulzz.onion/index.php/silkroad/user/1647

what is 3Jane's link?
Title: Re: Seller ranking and feedback overhaul
Post by: skunkworks on July 26, 2011, 12:04 am
This is a great idea. When our shop opens, we are gona be up there hopefully no1 !
Title: Re: Seller ranking and feedback overhaul
Post by: swann on July 26, 2011, 04:33 am
Rating's gone down from 98 to 97.4 to 96.7 to 96.1 without any of my orders' statuses changing...  :-\

I take it the new algorithm is giving increasing weight to my only non-5-star review because that guy is reviewing a lot of things or something?
Title: Re: Seller ranking and feedback overhaul
Post by: novocaine on July 26, 2011, 06:08 am
lol I have 14  5rating and 1 4outa5 and I dropped from 100 to 99:( I wont be able to sleep tonight.

God help us all if you post the very next day express, after you hit confirm sent....but you add some extra and there is a 1 day delay from when the buyer assumes he gets his goods... You watch those points get taken from you.

I have been reading some feedback from some sellers on SR. One of the comments were 'added a bit extra' and the prick gave a 4 . 5 >:(   some people you just cant please so I suppose this new rating system will work better in the long run.

SR is awesome imo. Its great to see it evolving:)
Title: Re: Seller ranking and feedback overhaul
Post by: joeblow2 on July 26, 2011, 01:16 pm
Wow.  That guy did you dirty, dirty, dirty @ novocaine.  He should be ashamed of himself.

I think though that the sellers are maybe a bit *too* worried about trying to maintain a 100% perfect rating.

In fact, as time goes on (say in 6 months) it would seem to me that anything over 90% for a seller of normal items and anything over 80% for a seller of hard to get/prone to buyer abuse items (like H, crack, coke, meth) would be an oustanding feedback number and get you plenty of business.

Maybe it would be good to start putting in the minds of the buyers a direct comparison of feedback scores to scores on a test?
Using the more liberal scale:  90+ is an A, 80+ is a B, etc.  That's a concept of grading that everyone knows ever since grade school.
And no amount of being wonderful is going to cause every person out there to give you a perfect score.  Don't get me wrong, you should be all about customer service.  Prices here reflect the buyer protections they receive and part of that (even though it's not strictly a "protection") is the great customer service.

If I was a seller I just wouldn't want to be checking my feedback score, nervously, like a guy does when he day trades stocks.  It's just another way that doing this biz can eat up all the other good things that are in your life. 

Have fun, enjoy your life and selling on SR as part of that life and don't let the people who treat you badly without cause get you down.
Just think: because it's online, you never have to meet them.  That's a big plus; don'tcha think?  :D
Title: Re: Seller ranking and feedback overhaul
Post by: RedRocket on July 26, 2011, 01:54 pm
the nice thing about this system,it doesnt just say 95%,80%,100% etc etc...it gives the number of transactions,that is very important,i would lean way more towards someone with 95% with 200 sales that someone with 100% and 10 sales....i reckon though,that with this new system,when a buyer sees less than 100% they believe that the seller received a 1/5 at some time...when in fact it was more like a few 4/5,which is fantastic.i think it should emphasize the lowest score and highest score in that new area and then the user will be able to click the numbers to view the feedback..i dunno...
Title: Re: Seller ranking and feedback overhaul
Post by: Kind Bud on July 26, 2011, 03:46 pm
@swan the % rating doesn't change : it is still (total points received) / (total possible points) This only counts actual ratings, you can have transactions without any rating but this will not change your % number.

I like the rating system AND the percentage number. The numbers help new sellers (although they get faked too much) and rating shows constancy.

4/5 is not fantastic, ever. 4/5 = 80% = acceptable but not awesome. 80% should be reserved for some minor seller screw up that was still acceptable. This business has a very low tolerance for mistakes and any 4/5 should be taken pretty seriously. A serious seller will make some 4/5 mistakes but they should be very rare.

Any mistake that involves safety or privacy should be judged much harder. 

I am wary of rating trolls.
Title: Re: Seller ranking and feedback overhaul
Post by: Dread Pirate Roberts on July 26, 2011, 05:25 pm
If you say a customer released escrow but didn't comment wouldn't then the number of sales and comments differ?  Mine are the same.  The only difference is in your count by +1 .  How does a customer go about releasing escrow without commenting?  How long does a customer have before they need to comment?  Thanks for your time SILKROAD  I just dont like my rep suffering for no reason at all.

Sorry I'm not being clear.  When a buyers finalizes an order, the funds are released, etc, and she is given the OPTION to leave feedback.  If she leaves the site and never comes back, for example, then your transaction count will always be higher than the number of ratings left for you by 1.  If your transaction count and number of ratings are the same, then it means all of your buyers left feedback for you.  All that being said, it doesn't affect your average rating one bit when someone completes a transaction, but doesn't leave feedback.  It's as if the transaction never happened as far as feedback is concerned.
Title: Re: Seller ranking and feedback overhaul
Post by: swann on July 26, 2011, 06:22 pm
@Kind Bud

Well, that doesn't seem to be true in my case! I have 17 reviews, one of which is a 3/5 review. ((17*5)-2)/(17*5)=97.6%. But I'm sitting at 96.2% (which is kind of a drag because (98) looks a whole lot better than (96)).
Title: Re: Seller ranking and feedback overhaul
Post by: Kind Bud on July 26, 2011, 07:35 pm
17 transactions does not equal 17 reviews. Reread my post or Silk Road's right above you. If you link to your profile I can tell you more precisely.
Title: Re: Seller ranking and feedback overhaul
Post by: swann on July 26, 2011, 08:10 pm
I don't have 17 transactions; I have 20 transactions.

http://ianxz6zefk72ulzz.onion/index.php/silkroad/user/1298

Hence my post!
Title: Re: Seller ranking and feedback overhaul
Post by: mseller on July 26, 2011, 08:26 pm
RedRocket, nice fly walking around. I left my fingerprints on my monitor ;D
Title: Re: Seller ranking and feedback overhaul
Post by: Kind Bud on July 26, 2011, 09:21 pm
@swann My bad. I worked on this for a while and could not come up with any way to make it work. I have seen wrong numbers before but always just considered it lag. I suggest you either take it to Silk Road directly or start a new thread. I am super interested but don't want to hijack this thread.
Title: Re: Seller ranking and feedback overhaul
Post by: smokeourbowls on July 26, 2011, 10:47 pm
Im not a huge fan of this new feedback system. My shit is dropping. Repeat customers should have more weight because their opinion is backed by double the experience or more.

@Silk Road - Can you explain the theory behind this? Im interested, but it seems like less experienced buyers have the most power over my rating which scares me.
Title: Re: Seller ranking and feedback overhaul
Post by: Kind Bud on July 27, 2011, 12:11 am
@smokeourbowls this system is awesome. Look at all the top rated sellers. 100s of feedback and only a couple of 4/5s  at the most. This is exactly who buyers WANT to buy from.

Experienced buyers DO get more pull, the theory being that they actually know what they are talking about. That is the #1 criteria. It is only later, if they rate the same person over and over that the effect starts to go down. Getting 5 reviews from one friend is less important than 5 reviews from 5 different buyers.

Title: Re: Seller ranking and feedback overhaul
Post by: joeblow2 on July 27, 2011, 04:32 am
I agree that the new feedback system is awesome and a great improvement over the old one.

But I would agree with @smokeourbowls that it would be nice to see how the system works.

Or is this something that needs to stay private so some of the more skilled of our sellers work the system to geek it for their own benefit?

I just like transparency, when it is appropriate.

So I would love for SR to weigh in and either tell us it needs to stay private OR show how the system weights the feedback.

This has been a regularly scheduled polite request for more information.  bye!  :)
Title: Re: Seller ranking and feedback overhaul
Post by: RedRocket on July 27, 2011, 05:45 am
sellers should advertise there products in the appropriate section with an attractive wright-up if they are feeling intimidated by the new system
Title: Re: Seller ranking and feedback overhaul
Post by: Dread Pirate Roberts on July 27, 2011, 06:07 am
we can definitely share the detail, though it gets a bit technical, and we are tuning some the parameters, so the whole thing isn't set in stone.

basically, each feedback score is weighted by three factors: buyer activity, age, and purchase price.

The buyer activity weight is determined by finding the average and standard deviation of the log of the number of vendors reviewed by buyers who've ever left a review.  if the log of the number of vendors reviewed by the buyer who left the feedback is greater than the average, the review has a "buyer activity" weight of the number of standard deviations above the average plus one.  If it is below average, then the weight is the inverse of the number of standard deviations below the average minus one.

The age weigh is found by raising 1.02337 to the power of the review's age in days and then inverting it.  We got 1.02337 by making a review half as valuable after one month as one that's just been posted.

The price weight comes from the following formula: 0.2*log(0.02*($price)+1)+1 where $price is in dollars.

You then multiply the weights together to get the total weight for that review.  Add up all of the weighted reviews and divide by the total weight to get the weighted average, then do 25*(1-avg) to get "percent positive reviews" metric.

To get the seller rank, we find the averages and standard deviations of the "percent positive feedback" and the log of the total weight of the reviews for all active sellers. We then average the number of standard deviations a seller has above or below the average with a 4 to 1 weight on their average feedback over their total weight.  This is a seller's rating, and we rank everyone according to this rating.

There are a few little details left out for brevity, but that's basically it.

Title: Re: Seller ranking and feedback overhaul
Post by: swann on July 27, 2011, 07:05 am
(A shameless plug for my self-serving interests:)

I'd like to see a variable number tempered by a coefficient of determination of a sample of recent reviews matched against a the forecast of a trendline of past reviews used as the base of the power function determining the aging weight; a general change in quality could increase the base and make past reviews have less weight and minimal change in quality could allow past reviews to hold higher weight. It just seems like a nice stability buffer for generally-consistent vendors (plus, I don't see why past reviews should lose half of their value every month if the vendor's staying consistent).
Title: Re: Seller ranking and feedback overhaul
Post by: Dread Pirate Roberts on July 27, 2011, 07:23 am
(A shameless plug for my self-serving interests:)

I'd like to see a variable number tempered by a standard deviation of total reviews used as the base of the power function determining the aging weight; a general change in quality could increase the base and make past reviews have less weight and minimal change in quality could allow past reviews to hold higher weight. It just seems like a nice stability buffer for generally-consistent vendors (plus, I don't see why past reviews should lose half of their value every month if the vendor's staying consistent).

our thinking here is to keep established vendors from resting on their laurels.  If the weight of ones feedback doesn't fall off over time in some way, then new feedback becomes less and less important, and new vendors will have little chance of catching up with the senior sellers.
Title: Re: Seller ranking and feedback overhaul
Post by: swann on July 27, 2011, 07:46 am
our thinking here is to keep established vendors from resting on their laurels.  If the weight of ones feedback doesn't fall off over time in some way, then new feedback becomes less and less important, and new vendors will have little chance of catching up with the senior sellers.

Doesn't this system encourage resting on their laurels under the presumption that most established vendors have their lowest ratings near the beginning of their vending career? I don't suppose you're expecting new vendors to catch up with senior sellers if the senior sellers retain their quality over time?

Tempering the weight on consistency should give a more accurate representation of a vendor's quality at any given time and doesn't explicitly favour older or newer vendors (under the presupposition that the weight is attributed equally for positive and negative trend changes (K*abs(R²))).

Not to mention that resting on one's laurels doesn't do a vendor any good if they aren't actively vending anything...
Title: Re: Seller ranking and feedback overhaul
Post by: Kind Bud on July 27, 2011, 04:47 pm

our thinking here is to keep established vendors from resting on their laurels.  If the weight of ones feedback doesn't fall off over time in some way, then new feedback becomes less and less important, and new vendors will have little chance of catching up with the senior sellers.

Awesome! There is plenty of room for excellent vendors. Established vendors should have to work at it and new vendors should get opportunity. I am sure the actual numbers may change slightly as Silk Road observes the system in action and that ratings match intuition.

@swann There should be some balance so that old vendors don't dominate but also being in business consistently over the long haul should count. There are almost no vendors left that started up right before the MtGox Crash and those that stuck with it are inherently more reliable than a new seller who hasn't had to stay in business even when honoring the deals on Silk Road lost money.

One other thing, I think you are underestimating the turnover. Silk Road gets lots of new sellers who burn bright and then burn out. But most sellers, even the honest ones, don't actually have any business experience. The barriers to entry on Silk Road are pretty low, but consistency is hard work. In a month you will see less than 25% of the current sellers will still be around,  if you are serious and know what you are doing you can be one of the veterans.
Title: Re: Seller ranking and feedback overhaul
Post by: swann on July 27, 2011, 07:04 pm

our thinking here is to keep established vendors from resting on their laurels.  If the weight of ones feedback doesn't fall off over time in some way, then new feedback becomes less and less important, and new vendors will have little chance of catching up with the senior sellers.

Awesome! There is plenty of room for excellent vendors. Established vendors should have to work at it and new vendors should get opportunity. I am sure the actual numbers may change slightly as Silk Road observes the system in action and that ratings match intuition.

@swann There should be some balance so that old vendors don't dominate but also being in business consistently over the long haul should count. There are almost no vendors left that started up right before the MtGox Crash and those that stuck with it are inherently more reliable than a new seller who hasn't had to stay in business even when honoring the deals on Silk Road lost money.

One other thing, I think you are underestimating the turnover. Silk Road gets lots of new sellers who burn bright and then burn out. But most sellers, even the honest ones, don't actually have any business experience. The barriers to entry on Silk Road are pretty low, but consistency is hard work. In a month you will see less than 25% of the current sellers will still be around,  if you are serious and know what you are doing you can be one of the veterans.

I think that's exactly what my suggestion addresses (though without having to make the assumption that older vendors are inherently more reliable). Vendors who come and go quickly aren't penalised for doing so (but their transactions hold--appropriately--high weight compared to transactions of a vendor who has made more transactions in the past and has stayed consistent into the present) and vendors who have been around a long time aren't penalised for that either (which I am admittedly extra-vehement about, sort of falling in that boat--though I don't suppose I'd have a different opinion in any other case) because notable change (positive or negative) in a vendor's reviews would place more weight on recent reviews (preventing senior vendors from resting on their laurels but keeping the significance of past reviews in cases where there isn't any reason to exclude it). It's not a penalty for vendors who aren't long-term, consistent investors; it's a mechanism to avoid placing undue weight on aberrant data points due to their recency (for everybody).

As it is, I could start a new vendor account, make 3 transactions, and be ranked like fifty places higher than I am right now... but that seems dishonest.
Title: Re: Seller ranking and feedback overhaul
Post by: SR_Seller_Accounts on July 27, 2011, 08:39 pm
I think it is a bad idea to decrease the importance of past feedback.

My first customer is just as important as my last customer and they should hold equal weight.

I believe that this new system will skew results and decrease the amount of good feedback left by good buyers because they will realize it will eventually be meaningless or otherwise a waste of their time.
Title: Re: Seller ranking and feedback overhaul
Post by: Dread Pirate Roberts on July 27, 2011, 09:07 pm
I think it is a bad idea to decrease the importance of past feedback.

My first customer is just as important as my last customer and they should hold equal weight.

I believe that this new system will skew results and decrease the amount of good feedback left by good buyers because they will realize it will eventually be meaningless or otherwise a waste of their time.

I think the opposite is true.  If a seller has a huge amount of feedback from a year of selling, then new feedback is a drop in the barrel and will barely change their rating.  So the vendor won't care about getting a good rating, and the buyer won't feel like their voice is heard.  However, if new ratings are more important than old, then the vendor will always have to work for good ratings from current customers.
Title: Re: Seller ranking and feedback overhaul
Post by: Variety Jones on July 27, 2011, 10:55 pm
I think the opposite is true.  If a seller has a huge amount of feedback from a year of selling, then new feedback is a drop in the barrel and will barely change their rating.  So the vendor won't care about getting a good rating, and the buyer won't feel like their voice is heard.  However, if new ratings are more important than old, then the vendor will always have to work for good ratings from current customers.

So this is currently on the front page at SR.

Not picking on Lexi, but still at 100 with this latest rating?

http://xfq5l5p4g3eyrct7.onion/view.php?image=ad2a64e5c82bec3b92df3c4e3402b188.jpg
Title: Re: Seller ranking and feedback overhaul
Post by: BitShuffle on July 28, 2011, 01:08 am
Where the fug is your GPG sig on all those posts, dammit?

- Bit
Title: Re: Seller ranking and feedback overhaul
Post by: Rook on July 28, 2011, 03:56 am
I think the opposite is true.  If a seller has a huge amount of feedback from a year of selling, then new feedback is a drop in the barrel and will barely change their rating.  So the vendor won't care about getting a good rating, and the buyer won't feel like their voice is heard.  However, if new ratings are more important than old, then the vendor will always have to work for good ratings from current customers.

So this is currently on the front page at SR.

Not picking on Lexi, but still at 100 with this latest rating?

http://xfq5l5p4g3eyrct7.onion/view.php?image=ad2a64e5c82bec3b92df3c4e3402b188.jpg

I noticed this too.  It certainly seems like there are still a few bugs in the system..
Title: Re: Seller ranking and feedback overhaul
Post by: Colonel Sanders on July 28, 2011, 07:23 am
So...

Does this mean a small-scale seller with a previously stellar track record can get one bad review from a flakey buyer and have their rating totally fucked because the new feedbacks carry more weight and they don't have as many feedbacks as a large seller?

Or am I misunderstanding something here.  Because I'm not really digging the sound of that.
Title: Re: Seller ranking and feedback overhaul
Post by: novocaine on July 28, 2011, 08:21 am
to put it into perspective I have 16 transactions with only 1 4/5 rating in my second last feedback. I dropped from 100% to 98.7%. 4/ 5 is still great...but to me, I failed. 

So really the rating is only there to make the sellers not sleep at night.... worrying about their scores ::)

Its the feedback that will get you buyers not the rating. If you suck as a seller it will show in your feedback. A decent buyers will judge you by your feedback not your rating.

Even if you get the occasional feedback troll you will still be miles ahead if your a decent seller because peeps want their shit again and again and they are going to kiss your ass if the product and service is good :D
Title: Re: Seller ranking and feedback overhaul
Post by: DigitalAlch on July 28, 2011, 06:33 pm
Kinda odd the Lexi has a 2/5 but is at 100%.

Weird,
DigitalAlch
Title: Re: Seller ranking and feedback overhaul
Post by: SR_Seller_Accounts on July 28, 2011, 09:06 pm
no seller that ever got a 2/5 should ever be at 100 unless they sold enough transactions to enable the old algorithm to allow it mathematically.

what this new system does is forgive sellers for past errors without notifying the buyer, at a glance, with accurate information the buyer would never know about unless they spent alot of time reading each feedback left.

being a buyer would be horrible in this scenario because, as has been proven, its totally unreliable and misinforms the buyer, telling them that a person that got a 2/5 is just as good as a person who did not.

IMO this is to a costly a price for the buyer to pay to give sellers unearned visibility and fraudulent credibility.
Title: Re: Seller ranking and feedback overhaul
Post by: Colonel Sanders on July 29, 2011, 05:39 am
no seller that ever got a 2/5 should ever be at 100 unless they sold enough transactions to enable the old algorithm to allow it mathematically.

what this new system does is forgive sellers for past errors without notifying the buyer, at a glance, with accurate information the buyer would never know about unless they spent alot of time reading each feedback left.

being a buyer would be horrible in this scenario because, as has been proven, its totally unreliable and misinforms the buyer, telling them that a person that got a 2/5 is just as good as a person who did not.

IMO this is to a costly a price for the buyer to pay to give sellers unearned visibility and fraudulent credibility.

+1

I don't really have strong opinions about this, but I think the old feedback system was the lesser evil here.
Title: Re: Seller ranking and feedback overhaul
Post by: MagicKillerMan on July 29, 2011, 07:16 am
I don't know how I feel about this new system. The small time vendors are at a disadvantage. It really puts the pressure on us, and for people like me, Most sales come from 3 or less repeating customers, I need them to be able to vogue for me. I just don't know how accurately this new scoring system rates trustworthiness and vendor reliability of smaller less frequent sellers.

-Magic
Title: Re: Seller ranking and feedback overhaul
Post by: anarcho47 on July 29, 2011, 03:30 pm
I'm allright with it.  I only have 40-something transactions and I am in the top thirty.  I wouldn't call myself a "big-time" seller on here, just an average volume one.  So the feedback system makes sense to me. 

I think it exposes more as far as a vendor is concerned.  Although a breakdown of the criteria used to determine rank would be good, in case customers want to be more informed on what those numbers mean in real terms.
Title: Re: Seller ranking and feedback overhaul
Post by: Dread Pirate Roberts on July 29, 2011, 09:23 pm

So this is currently on the front page at SR.

Not picking on Lexi, but still at 100 with this latest rating?

http://xfq5l5p4g3eyrct7.onion/view.php?image=ad2a64e5c82bec3b92df3c4e3402b188.jpg

ranking and feedback is updated twice a day.  We don't do it in real-time to avoid using up alot of system resources.
Title: Re: Seller ranking and feedback overhaul
Post by: Dread Pirate Roberts on July 29, 2011, 09:31 pm
So...

Does this mean a small-scale seller with a previously stellar track record can get one bad review from a flakey buyer and have their rating totally fucked because the new feedbacks carry more weight and they don't have as many feedbacks as a large seller?

Or am I misunderstanding something here.  Because I'm not really digging the sound of that.

if you have a bunch of good feedback for a month, no feedback for a couple of months, and then start getting bad feedback, then yes, your avg rating will drop more quickly than if there wasn't more weight given to newer reviews, but this is what we want.  Basically you have to stay consistently good to keep a high score and can't rely on your performance from months and months ago.
Title: Re: Seller ranking and feedback overhaul
Post by: Kind Bud on July 29, 2011, 10:19 pm
you have to stay consistently good to keep a high score
Who else wants to have babies with Silk Road ?
Title: Re: Seller ranking and feedback overhaul
Post by: Variety Jones on July 29, 2011, 11:19 pm

So this is currently on the front page at SR.

Not picking on Lexi, but still at 100 with this latest rating?

http://xfq5l5p4g3eyrct7.onion/view.php?image=ad2a64e5c82bec3b92df3c4e3402b188.jpg

ranking and feedback is updated twice a day.  We don't do it in real-time to avoid using up alot of system resources.

Fair enough, but it's been over 2 days now, and still no change in Lexis' rating.

http://xfq5l5p4g3eyrct7.onion/view.php?image=e2ca1ea8e9d4552b92b659e9668c24c3.jpg

Image shows that as of a few minutes ago she's still showing 100%, with a 2 day old 2/5 feedback...
Title: Re: Seller ranking and feedback overhaul
Post by: swann on July 30, 2011, 04:48 am
So...

Does this mean a small-scale seller with a previously stellar track record can get one bad review from a flakey buyer and have their rating totally fucked because the new feedbacks carry more weight and they don't have as many feedbacks as a large seller?

Or am I misunderstanding something here.  Because I'm not really digging the sound of that.

if you have a bunch of good feedback for a month, no feedback for a couple of months, and then start getting bad feedback, then yes, your avg rating will drop more quickly than if there wasn't more weight given to newer reviews, but this is what we want.  Basically you have to stay consistently good to keep a high score and can't rely on your performance from months and months ago.

Or if you get one bad review and aren't a large-scale drug trafficking operation...

My feedback percentage is continuing to drop as I get more and newer positive reviews because my past reviews are aging into obscurity (which won't be resolved until my single non-5 star review fades into obscurity over the course of several weeks--something which is rather ridiculous in such a volatile business). Having suggested a quick and intuitive fix for misrepresenting people's feedback in terms of statistical anomalies, I'm rather appalled that my suggestion has been waved off in favour of preserving some integrity of the system whose existence balances precariously on a logical faux pas concerning the possible volatility of the quality of vendors (which is addressed and amended in my suggestion, no less) and I am further concerned that you've acknowledged my grievance--and are hence aware that the system does not always adequately represent a vendor's feedback--and that you are actively campaigning against implementing changes to the new system in spite of that. While I wholeheartedly appreciate the service that Silk Road provides in the world, I am slightly disappointed at the moment.
Title: Re: Seller ranking and feedback overhaul
Post by: watmm on July 30, 2011, 10:30 am
Here's another account worthy of interest:
http://ianxz6zefk72ulzz.onion/index.php/silkroad/user/752

ranked #169 out of 169 sellers with 100% positive feedback from 11 transactions

Though the feedback looks suspect to me.
Title: Re: Seller ranking and feedback overhaul
Post by: podperson on July 30, 2011, 10:38 pm
in the last 24hrs I just got a 1 out of 5 dated 16 days ago for a transaction that never took place. what the fuck is going on?

there was never a sale 16 days ago and that feedback was not there yesterday.  what the hell?
Title: Re: Seller ranking and feedback overhaul
Post by: Colonel Sanders on August 02, 2011, 06:11 am
How the hell does Goldismoney have an 86 rating with all these new 1 reports that are supposedly weighed heavier?

http://ianxz6zefk72ulzz.onion/index.php/silkroad/user/55
Title: Re: Seller ranking and feedback overhaul
Post by: mycontact on August 12, 2011, 06:01 pm
There will be always fake feedbacks just to make better rating. the same person make 2 different account and make rating for himself. that's so all over Internet. the best is that people try product and decide is it good or not, and there should be only rating in the sense of: scam or not. in other case, there will be "feedback war" between competitions. I think website should be neutral but it is not if they gave possibility for war. where is money, there is war.
I remember when I sold weed before 6 years (in that time the price was just 1 eur per gram, I brought 4 grams), 2 people pretended they want to buy it, just to get chance in front of other people to say how it is shit small package, etc, but they did so just because they sell weed too. For me it was comic because it was city with 1,5 million people and there is enough place for everybody. but some people are small in their mind and that's so.
and in this site are not 1,5 m people, so, war possibility should be disabled, as I said, it is enough to know if it is scam or not, the rest people must check personally, they can order 1gram, before they order more.
Title: Re: Seller ranking and feedback overhaul
Post by: Rook on August 12, 2011, 07:33 pm

So this is currently on the front page at SR.

Not picking on Lexi, but still at 100 with this latest rating?

http://xfq5l5p4g3eyrct7.onion/view.php?image=ad2a64e5c82bec3b92df3c4e3402b188.jpg

ranking and feedback is updated twice a day.  We don't do it in real-time to avoid using up alot of system resources.

Fair enough, but it's been over 2 days now, and still no change in Lexis' rating.

http://xfq5l5p4g3eyrct7.onion/view.php?image=e2ca1ea8e9d4552b92b659e9668c24c3.jpg

Image shows that as of a few minutes ago she's still showing 100%, with a 2 day old 2/5 feedback...

Make that 12 days..
Title: Re: Seller ranking and feedback overhaul
Post by: CaptainSensible on August 18, 2011, 10:24 pm
Sorry, but until there's a way to deal with fake feedback the new rating system can't be much better than the old system.  There are some sellers who have exactly the same wonderful comment in two different feedback entries.  What a strange coincidence!  >:(   And if you look over the very similar writing style and choice of words in some comments, you get a strong feeling that the same person left two (or sometimes more) feedback entries.

This wouldn't bother me so much except that that all feedback, truthful or not, apparently will be taken into account if a transaction goes to arbitration.  When I see that a buyer only got some of his Bitcoins back from an order that never arrived, and the seller has a lot of obviously fake feedback, that just makes me question the escrow/arbitration process of Silk Road.
Title: Re: Seller ranking and feedback overhaul
Post by: Jhdrgdzw on September 03, 2011, 12:31 am
Is there a way to see what user leaves the feedback?
Title: Re: Seller ranking and feedback overhaul
Post by: anarcho47 on September 03, 2011, 02:04 am
I have that sometimes, two similar feedbacks, but that's because for my US customers I charge a shipping addon - it's a lot easier for me to manage my inventory than creating two entire sets of listings at two different price levels and having to adjust the damned inventory every depending on who happens to order to Canada and who happens to the US.

The feedback ratings are only half the story - they are quite limited.  If you want the real juice go to the rumor mill.

And no there is no way to see what users leave feedback, since you can link through to the item being reviewed.  This was one of the big no-no's pointed out when modifications to the feedback system were being discussed - we need as much transparency as possible without divulging too much and threatening buyer's privacy.  The solution right now I think is best (and also happens to be the one I suggested :)  )
Title: Re: Seller ranking and feedback overhaul
Post by: SkyB86 on September 04, 2011, 10:31 am
It's probably a lot more accurate !
Title: Re: Seller ranking and feedback overhaul
Post by: 4sale on September 04, 2011, 03:35 pm
I think that the feedback system has some flaws:


1) It's bad statistically-wise: 90% of the seller are in the (90-100) range, while (0-90) is almost empty.

2) Due to point one, it is very difficult to distinguish between sellers rank-wise

3) Moreover, a (98) seller can be much more trustworthy than a (100) seller.


I would change it following the ebay way:

1) A score based on # of sales, eventually weighted over time (Old transactions eventually fade out
- or -
A score based on internal SilkRoad metric, with no upper limit.

2) A score based on  customer's satisfaction.


People inside Ebay worked a lot on finding a system which is scam-proof (as much as a feedback system can be) and intuitive (rich in informations).

No need to reinvent the wheel IMHO...
Title: Re: Seller ranking and feedback overhaul
Post by: anarcho47 on September 04, 2011, 03:46 pm
SR's feedback system is already front-weighted, albeit on a longer time-scale.

Just looking at the number beside a seller's name isn't enough.  Checking their profile page (get a feel for them by what they say), reading through feedback, even the fact that each product has its own feedback is handy to a buyer.

The real value is in the forums though.  That is where you get a lot closer to real time, with praise, issues, resolves, etc. being posted up.

I think for the next feeback edit out, breaking feedback into three separate categories would be more ideal:  customer service, shipiping/packaging, product quality.  Each one would have a score out of 5, and be weighted equally (i.e. 33%) when considering their contribution to a seller's rank, and would give buyers a clearer picture in an instant.  But feedback commentary should also be thorough on the buyer side, as opposed to just "perfect" or "great" or "Decent".  Take ten seconds and explain what made you come to that conclusion.

We made a step in the right direction, though, and SR's improvement will always be a series of small steps in the right direction.
Title: Re: Seller ranking and feedback overhaul
Post by: 4sale on September 04, 2011, 03:50 pm
  But feedback commentary should also be thorough on the buyer side, as opposed to just "perfect" or "great" or "Decent".  Take ten seconds and explain what made you come to that conclusion

Lol indeed.

I hate when I take every step possible to satisfy a customer, and I get a 5 of 5 Leave FeedBack here. Still it's utopic, human beings are lazy by nature :P
Title: Re: Seller ranking and feedback overhaul
Post by: anarcho47 on September 04, 2011, 04:25 pm
I prefer to label it as "human beings trend towards efficiency" ;)

hence the fact you can pick up a phone instead of hiring a runner with a horse.....
Title: Re: Seller ranking and feedback overhaul
Post by: cactusboy on October 24, 2011, 06:53 am
The real problem as stated earlier is the possibility of sellers setting up buyer accounts just to boost their ratings if a seller is rated 86% and they have left half the feedback themselves. They look like they are average when they may have scammed 3/4 of their customer.
The other thing is that if feedback looses half it's value every month and the buyer has finalized early and left positive feedback, waits 2-3 weeks for package to arrive before changing feedback 1-2 later that feedback is half as effective. Not to mention ratings can take 7 days to adjust. Meaning a smart scammer will ask people to wait a bit longer for their package while adding there own positive feedback.

Any seller under 95% is a possible scammer/ selective scammer.

We should not put up with scammers.

I would suggest more buyer fees to help protect up from buyers been able to do this. Maybe this extra revenue could fund more buyer / seller protection I think this thing would become more popular if it were designed to protect people even better.