All About Trust and Reputation in the Digital World

Tag: Rating Services

Peeple, A Cautionary Tale About Rating People

Peeple A Cautionary Tale About Rating People

The announcement, in late September, of an unreleased app (called Peeple) that aims to allow individuals to rate each other triggered a huge media frenzy (both in traditional media and social media) about the toxic nature of this app.

Peeple, as described by its founders, will give the possibility to rate individuals, with a 1-to-5 star rating system, and provide review from three different angles: personal, professional, or romantic. The traditional media and social media reaction was extremely negative, they deem the app a total disaster that will take cyberbullying and harassment to a whole new level. Critics from all over the digital world pointed out how the app will do more harm than good. Peeple’s founders got death threats; their social media accounts were hacked; their private photos were leaked. At some point, even cops and the cybercrimes unit were involved.

Peeple’s founders for sure made some glaring mistakes both from a design and communication standpoint. Talking about an app that deals with sensitive issues, such as privacy and reputation, before getting any feedback from potential users and iterating to build a product people would love is a misstep of Everest proportions. Discussing in major media about features that, if they are not put in context, would alarm any sensible user (like the impossibility to opt-out from the service and the absence of any form of consent from those subjected to rating and evaluation) is another blunder.

But at the same time, we were surprised by the fact that this app was unanimously vilified, the main argument being we cannot rate people or judge their personality and behavior. But the fact of the matter is that the digital world is full of services that allow to rate people and judge their traits and behaviors. Actually all the services related to the so-called sharing economy (you know Uber, Airbnb and the likes) include comprehensive rating systems. More than that, it is an essential feature to foster trust and reputation among their communities and exclude dishonest and ill-mannered individuals.

Rafik Hanibeche & Adel Amri (Trustiser Founders)

Fake Reviews, the Plague of Rating Services

Fake Reviews The Plague of Rating Services

Once again, Yelp is  in the news for fake reviews.  This time around, it is a mattresses and furniture  store in La Mesa, California.  “This business created a dozen or so accounts on Yelp from the same IP address (their IP address) which they used to create fake reviews of their own business,” said Vince Sollitto, Vice President of Corporate Communications at Yelp.  “The business then used those accounts to message people on Yelp offering to pay $25 by PayPal or mail for a five star vote,” Sollitto said.

Yelp Consumer Alert

As outlined in a previous post in this blog, fake ratings, along with the lack of models to manage trust placed in raters, haunt existing rating services and are  major issues undermining their credibility.  These issues should be addressed urgently, as more and more people rely on rating services to make increasingly important decisions.  We are of the opinion that these issues have to be addressed by combining a computational approach with a human controlling effort in order to substantially improve the overall trustworthiness of rating services, and this is what Trustiser is all about.

Rafik Hanibeche & Adel Amri (Trustiser Founders)

Can We Trust the Crowd Miners?

Can We Trust the Crowd Miners

The digital world is caught in a data deluge, caused to a large extent by the huge collection of actions, ratings, recommendations, opinions, and mere information (in the form of text, audio, or video) generated every day by the citizens of the digital world.  This phenomenon has not gone unnoticed by the research and commercial communities.  As a result, many companies and universities have invested heavily in developing various data mining techniques to harness the exaflood of data generated by the data deluge and discover valuable knowledge and relevant patterns.  

Of particular interest is crowd mining, where gigantic databases of social information are mined to extract useful knowledge.  One example is dishtip, a service offered by TipSense.  TipSense devised a data mining algorithm which is able to reveal best dishes at restaurants by crunching millions of reviews, mentions, and photos of food.

Crowd mining looks very promising but the data extracted from social databases convey malicious content, such as fake ratings and recommendations, that can corrupt the results of crowd mining tools.  In this context, several approaches have been developed to fight malicious content by cleaning the data.  In the realm of rating services, several universities (e.g. Cornell University) and companies (e.g. Google) are working hard to detect fake ratings. 
However, we do believe that fake rating detection algorithms are necessary but not sufficient to deliver high quality data to crowd mining tools.  Indeed, all ratings are not equal, that is the reason why each rating has to be weighted by the trust placed in the user who performed the rating.  In this context, Trustiser will push the envelope by providing crowd mining engines with reliable ratings generated by a community of members arranged hierarchically; the basis of the hierarchy is the trust placed in raters in relation to each topic.

Rafik Hanibeche & Adel Amri (Trustiser Founders)

Humans, Trust and Reputation

Humans trust and reputation

In order to manage efficiently trust and reputation among humans in the digital world, two important dimensions should be taken into account.

The first dimension is related to the organization of human societies.  Human societies are organized hierarchically, and the basis of the hierarchy is trust.  Indeed, the trust placed in an individual with regard to a given topic determines his position in the hierarchy related to the topic.  The level of trust, and the reputation it creates, depends on the sincerity and the experience or expertise exhibited by the individual in relation to the topic.  Therefore, trust and reputation management in the digital world should also be hierarchically organized, topic-related and experience or expertise-linked.

The second dimension revolves around the importance of ratings for humans.  Indeed, for the human brain, ratings, recommendations, and opinions are much more useful than mere information because ratings and opinions, especially ratings and opinions from reliable sources such as renowned experts, knowledgeable people, and close friends, have a big impact on trust inference.  Trust inference is of primary importance because it helps the human brain in making decisions very quickly.  In this respect, we would like to quote the columnist Charles McCabe: “Any clod can have the facts, but having opinions is an art”. Therefore, trust and reputation management in the digital world has to adopt, whenever possible, rating-centric and opinion-driven approaches.

Rafik Hanibeche & Adel Amri (Trustiser Founders)

Rating the Raters

Rating the Raters

Rating services are playing an increasingly important role in the digital world as more and more individuals rely on these services to plan for the future, that is make critical or life-style-driven decisions, solve problems or discover opportunities.  In this context, the success of rating services will depend to a large extent on their ability to establish a trusted relationship with their users.

As a matter of fact, existing rating services generate a great deal of distrust among their users because they are fed by humans who all too often are able to discover and use the flaws in the algorithms that underpin current rating services.  In addition, current rating services place the same amount of trust in all users, regardless of their sincerity and the level of their experience or expertise-linked knowledge.  On top of that, businesses are devising strategies of all sorts to exert an influence on their customers’ ratings and reviews. Some businesses have gone so far as to pay targeted customers to get positive ratings and reviews.

Basically, there are two major issues that need to be addressed:
– Existing rating services are not able to defend themselves against individual and collaborative fake ratings and reviews, and thus are subject to manipulation.
– There is a lack of models to manage efficiently the trust placed in each rater.

Let’s take a closer look at each of these issues.

The inability to counter fake ratings

According to Bing Liu, a computer science professor at the University of Illinois at Chicago and a leading researcher in the area of fake online ratings, “for some products, up to 30 percent of ratings can be fake”.  Moreover, according to Gartner, by 2014, 10 to 15% of online ratings will be fake and paid for by companies. Fake ratings and reviews, a very cheap way of marketing, can be either excessively negative, bashing competitors’ products, services or brands, or overly positive,  raving about the products, services or brands of specific businesses.
Fake ratings and reviews can be published directly by businesses or posted by customers who receive perks for their contributions. Fake ratings and reviews can also be obtained by relying on professional favorable ratings providers.

A lot of effort and money are spent in research and development to rise to the challenge and devise mathematical models that detect and combat fake ratings and reviews.  For example, a team of researchers from the University of Rhode Island is underway with a project to develop algorithms that can serve as a defense against collaborative, profit-driven manipulations of online rating services.

The lack of models to manage trust placed in raters

The raters space is completely “flat” in existing rating services, and there is no hierarchy in terms of competence, that is experience or expertise-linked knowledge.  Moreover, current rating services are not intrinsically organized to attract competent raters.  As a result, everyone can rate anything in any domain.  Although some rating services include features aimed at tracking raters’ activities and giving a competence-based weight to each rating, we are far from addressing this issue adequately and effectively.

At this point, it is interesting to draw a parallel with Google’s search engine.  Considering that the web is a graph of documents, the power of Google’s search engine algorithms revolves around assigning a reputation index (a Page Rank) to each document and classifying the documents by their reputation inside the graph.  These algorithms not only take into account  the number of documents linking to a given document, but they also consider that a document partially inherits the reputation assigned to the documents that point to it.

If we consider the analogy with the graph of documents, rating services should include a graph of raters and implement algorithms similar to Google’s Page Rank algorithms.  But assigning a reputation index to each rater in the graph of raters through incoming reference count would not be sufficient to estimate the real reputation of a rater because:
– The graph of raters is more complex.  Reference counting alone is not sufficient to express this complexity.  The type of relationship a rater has with other users and the level of influence he exerts within the community of users should also be taken into account.
– The reputation index shouldn’t be considered as a single scalar number because when we look at the graph of raters from the angle of knowledge, the graph lives in multi-dimensional space, each domain of knowledge should be considered as a dimension, and reputation index should take the form of a vector.

In summary, although there is a great deal of effort to make rating services more reliable, the accomplishment of this goal is a long way off.
We do believe that a pure algorithmic approach is not sufficient to fix all these problems.  There is a need to combine a computational approach with a human controlling effort in order to substantially improve the overall trustworthiness of rating services.

Rafik Hanibeche & Adel Amri (Trustiser Founders)

%d bloggers like this: