Silk Road forums
Support => Feature requests => Topic started by: ChronicallyMedicated on June 22, 2011, 06:25 am
-
It appears that there are certain vendors (ex. 'vortextmilkman') who may be 'ordering' their own product through accounts they create in order to leave bogus feedback to boost their seller ratings. I am proposing a two-pronged method to hopefully help mitigate this issue:
1: Give each feedback an estimated 'authenticity' rating. For bogus accounts that have been created to shill a single vendor, these accounts will have only left feedback for a single vendor, so tend to be more skeptical of accounts that have rated only a single vendor. Also put less trust into accounts that have left less feedback, or have less time from (date of last feedback- date of first feedback). Trust feedback from accounts that have been recently registered less. Trust feedback more from accounts that are older than the vendor being left feedback for.
2: Differentiate feedback left from virtual goods or legal goods, as this can be an easy loophole for bogus accounts to look more legitimate, or for LE to establish legitimate positive feedback.
In terms of implementation (this will obviously have to be tweaked), something like this might be a good place to start off:
Feedback authenticity (0 - 100) =
20 * clamp_to_1.0(number_of_distinct_vendor_feedbacks / 5)
+ 20 * clamp_to_1.0(number_of_total_feedback_left / 5)
+ 20 * clamp_to_1.0((day_of_last_feedback - day_of_first_feedback) / 15)
+ 20 * clamp_to_1.0(age_of_buyer_account_in_days / 15)
+ 20 * (if(buyer_id > seller_id) 1.0 else 0)
So with these particular details, feedback authenticity would be high from buyers who:
+have given feedback to at least 5 vendors
+have left at least 5 total feedback ratings
+have been leaving feedback for at least 2 weeks
+have been registered for at least 2 weeks
+have been registered before the seller
Feedback authenticity would be lower if any of these criteria are not met. Obviously this isn't perfect, but I think it would make it much more difficult for scammers to create new fake feedback from shill accounts.
-
This would definitely help, I'd be down for it :] Although I might lower the distinct buys / total feedback to 3 personally. I'm sure there are going to be a lot of one-time buyers on SR and we don't want to TOTALLY discredit their ratings, just be able to see if a vendor only has ratings from low-ranked buyers
-
I see the need for such a system, but I don't think it's wise for SR to create a database that stores the history of how many transactions buyers have made.
-
Or just add another column for the buyers user ID, of course it wouldn't be the full ID but something like 162XX
Then people would see if someones feedback came from only people higher user IDs than the seller. Of course they could have dummy accounts with lower user IDs but I doubt it. I would trust the person with a mix of old and new user IDs on their feedback page before someone with all feedback user IDs in the 17000s
-
I like these ideas guys. :D
-
yea, great ideas. I don't think the simple rating system we've been using will hold up much longer.
-
Thanks Silk Road!
As was suggested in another thread, everyone should remain skeptical of any Sellers with userIDs higher than 2500. (I realize I'm included in this group and sketchy newcomers, but my products aren't competing with any established sellers, so you'd have to take a risk either way :P )
-
Great ideas. You should also be able to leave feedback for a seller that cancels your order. Maybe not a 1 to 5 rating, but something that lets the buyers know how often they do it.
Right now you can't rate sellers that cancel.
-
bump this super old thread - great idea :)
I still see vendors padding their own feedback today
(edit - anyone else see OPs avatar picture link trying to connect to bing clearnet site?)
mods may want to move this thread to feature requests
-
yea, great ideas. I don't think the simple rating system we've been using will hold up much longer.
The lord has spoken.
Agreed - but now the issue is correctly weighting it, can be a right mathematical tangle. Let me play with some numbers and I'll post up with my results.
Edit: Jesus christ just noticed the age of this thing. Necrophied (< Is that a word?) much?
-
Necrosity or no, this is a really interesting thread... Feedback authenticity is sorely needed. What's in place gets the job done as it is now, but the possibilities could promise better things! ;)
-
yea, great ideas. I don't think the simple rating system we've been using will hold up much longer.
Glad to hear you are on it. While you are at it I would just LOVE to be able to leave feedback for buyers.
Great ideas. You should also be able to leave feedback for a seller that cancels your order. Maybe not a 1 to 5 rating, but something that lets the buyers know how often they do it.
Right now you can't rate sellers that cancel.
Why on Earth do you think you should be able to leave feedback to vendors who don't want to do business with you?
We don't have a block buyer option(Hey DPR, how about a block buyer option?) so if your suggestion was implemented a buyer could repeatedly buy from a seller and leave negative feedback and the seller would be helpless.
-
Changes to the feedback system have been implemented since this thread was started back in JUNE, 2011. Yes, that is a hint to please not bump outrageously old threads. :P
You can read about how the feedback system currently works here:
http://dkn255hz262ypmii.onion/wiki/index.php?title=Feedback_Score
Feedback Score
What is Feedback Scoring?
You may have noticed the (100) next some vendors names, this is their Feedback Score. This is a quick way of finding out how reputable a vendor is, a step to take before reading some of their actual feedback. This score is a weighted average, based on many aspect of their transactions. The weights are determined by a few factors, and here they are, in order of importance:
How active a buyer is in rating other vendors
The age of the rating
The price of the transaction
In order for a vendor's rating to appear next to their name, they must be reviewed enough for it to be a meaningful average. This requires roughly 10 reviews per month, but can be more or less depending of the weights of the reviews.
The Formula
The buyer activity weight is determined by finding the average and standard deviation of the log of the number of vendors reviewed by buyers who've ever left a review. If the log of the number of vendors reviewed by the buyer who left the feedback is greater than the average, the review has a "buyer activity" weight of the number of standard deviations above the average plus one. If it is below average, then the weight is the inverse of the number of standard deviations below the average minus one.
The age weight is found by raising 1.02337 to the power of the review's age in days and then inverting it. We got 1.02337 by making a review half as valuable after one month as one that's just been posted.
The price weight comes from the following formula: 0.2*log(0.02*($price)+1)+1 where $price is in dollars.
You then multiply the weights together to get the total weight for that review. Add up all of the weighted reviews and divide by the total weight to get the weighted average, then do 25*(1-avg) to get "percent positive reviews" metric.
To get the seller rank, we find the averages and standard deviations of the "percent positive feedback" and the log of the total weight of the reviews for all active sellers. We then average the number of standard deviations a seller has above or below the average with a 4 to 1 weight on their average feedback over their total weight. This is a seller's rating, and we rank everyone according to this rating.
There are a few little details left out for brevity, but that's basically it.
Libertas
-
That did not give an answer to the fact that there is no authenticity to feedback left for vendors.
ANY vendor can create a shill account and leave FAKE feedback for themselves.
This has nothing to do with the vendors overall score, this is about verifying feedback as authentic.
The bump was justified.
-
bump