MIT Algorithm More Quickly Predicts Trending Topics On Twitter
Michael Harper for redOrbit.com – Your Universe Online
Businesses and organizations covet the “Trending Topic” status on Twitter. Just as it sounds, a topic “trends” whenever a mass of people begin talking it up on the micro blogging site. Twitter posts these topics on their front page and, as one might expect, this kind of free publicity can be great for a company or group.
Of course, finding the top trending topics isn’t as easy as searching for the most used words and, as such, Twitter employs an algorithm to discover which topics have seen a recent uptick on the site. Now, a professor and one of his students at MIT say they’ve come up with an even better algorithm which can accurately predict what will be trending up to 5 hours in advance.
Associate Professor Devavrat Shah and his student, Stanislav Nikolov will present this time-traveling algorithm at the Interdisciplinary Workshop on Information and Decision in Social Networks this month.
While Shah and Nikolov have high hopes for their algorithm, they also say it needs to be trained in order to be its most effective. For example, in the testing phase, this algorithm was given data about different topics, and then picked up on different patterns between those topics which did trend and those which didn’t.
Shah explains that the general assumption about these trending topics is that there is a sudden shift, a “step” in the way these topics suddenly become popular.
“The problem with this is, I don’t know that things that trend have a step function,” says Shah.
“There are a thousand things that could happen.”
Shah and Nikolov explain that their algorithm simply lets the data decide which topic is going to trend and which isn’t.
For example, this algorithm watches the number of tweets about a certain topic in an hour and compares it to the changes in other tweets in the sample set. In essence each topic can eventually trend, but those topics which have already built up steam are more likely to reach Twitter’s homepage.
In their research, Shah and Nikolov trained the algorithm with 200 topics which made it to the trending list and 200 which didn’t.
Then, the pair let this algorithm loose in the wild to predict which topics would trend next, in real time. According to these tests, their algorithm was able to predict the next trending topic with 95% accuracy. While these results are already quite impressive, Shah predicts the algorithm will only get better when they increase the sample set.
Twitter has been making moves in recent months to become a more singular and independent company, placing severe restrictions on companies they once partnered with.
Shah and Nikolov’s new algorithm can accurately predict the next trending topic an hour and half on average before Twitter’s algorithm. This kind of accuracy and advanced notice could be just the thing for a company who is looking for ways to bring in advertisers to their service. If Twitter knew what topics would be trending as many as 5 hours before it happened, they’d be able to adjust their ad pricing for that time. For the hours leading up to trending topic status, Twitter would be able to offer premium ad space at a premium price.
Shah and Nikolov say this algorithm could be used in other areas outside of Twitter. For instance, with this algorithm, movie studios could predict when tickets were set to peak, commuters could predict the length of a bus ride, and investors could be better informed about market trends.
redOrbit.com
offers Science, Space, Technology, Health news, videos, images and
reference information. For the latest science news, space news,
technology news, health news visit redOrbit.com frequently. Learn
something new every day.\”
2012-11-02 07:20:23
Source: http://www.redorbit.com/news/technology/1112724599/twitter-trending-topic-algorithm-110212/
Source:
