116
« on: August 14, 2013, 05:16 pm »
great ideas everyone. I'm really glad I opened up the discussion. The delivery success rate and transit time metrics are ones I've wanted to have for a long time. The problem with transit time is when to count an item as delivered (when it's finalized? What about FE? Depend on the user to submit the data? What about different postage options like express vs first-class vs. international and variations in the postal systems of different countries?). Then the problem with delivery success rates is the same problem we have with feedback in general, which is fraudulent data, whether it's vendors padding their own stats, or knocking their competitor's stats, or buyers using their review as leverage over the vendor's stats.
So as you can see, it's a tough nut to crack in its own right. For now, I'd like to focus on improving just vendor feedback and reviews. I've been playing around with different systems and parameters and seeing how it affects the current and historical stats. One thing I keep coming back to is something that was said earlier in the thread: "money talks". It's looking more and more like the most important parameter for weighting buyers and sellers is simply how much money they've spent/earned. Basically, it's really hard for a scammer to fake feedback from buyer accounts that have spent lots of money with vendors who have lots of sales. I think one of the main reasons for this is that, to fake a transaction, you still have to pay the commission, so pretending to be a top vendor through dummy transactions with high volume buyers simply costs a lot.
I've gone ahead and implemented one behind the scenes change that I think has improved things a little bit already. The vendor weight, which determines their percent ranking as displayed on the vendor pages is now determined with just two variables: sales volume and age of sales. So, the vendor weight is the sum of all of their transactions, but each transaction is weighted from 100% today down to 0% once it is 8 months old. There is a whole thread in the vendor forum of vendors who are really pleased with the effect this change has produced.
I'm still liking the idea of decoupling the 1 - 5 review and the written feedback.
Another idea I am liking is displaying not just the average review expressed as a percent, but the actual distribution of 1, 2, 3, 4, and 5 point reviews. I think this will make it much more acceptable to leave a 4 because instead to damaging the average, it just puts a little more weight in the 4 column. This is something Amazon does and I think we should adopt it.
This, I think, is enough to chew on for now and get pushed out into production so we can see it in action. We don't want to change too much at once because it is hard to foresee all of the effects a change will have until it is actually made.