Whether it is browsing news, forums, or postings, even under the product reviews of Taobao and JD.com, and app appraisals from the Apple App Store, we can often see some advertisements and worthless spam messages. The traditional approach is to screen manually and delete it in the background. It is not only time-consuming and error-prone but it may change in the future.
Recently, foreign media reported that Yahoo will use artificial intelligence technology to delete spam messages under news reports. In the test, this algorithm has a 90% probability of identifying inappropriate language, and this accuracy is superior to manual troubleshooting and other powerful deep learning techniques.
Yahoo's algorithm is different from the traditional keyword recognition, but will focus on a series of additional factors, such as the length of the content and the use of punctuation. Subsequently, the trained staff also evaluates the results of these intelligent algorithms and feeds them back to the algorithm to train the AI ​​to feel the nuances of the language. In addition, Yahoo also used the additional evaluation results of crowdsourcing in Amazon Mechanical Turk to provide these evaluations as non-professional comment managers to comprehensively train the accuracy of this artificial intelligence algorithm.
At present, this technology is not publicly available. However, Yahoo believes that this will be an important step in the natural language processing of computers. The technology is well used inside Yahoo, but it will also need to be tested by other databases in the future.