1 Good Reason – Social Marketing

BLUF (Bottom Line Up Front)- Twitter is a great service, but their search results are simply garbage.  They are unreliable after 3 to 5 days, and spotty at best after 24 hours.  You cannot do a date based search to find out if people were talking about a topic or product 6 months or 1 year ago with any accuracy.  The only way to get reliable historical Twitter data is collect it your self in real time and store it.

Twitter’s search results can best be described as bizarre.  I know this because I am working on creating a tool that allows people to manage contests on Twitter in real time.  It uses the Twitter Search API to gather every tweet mentioning a hashtag or other phrase.  Using this software I’ve done thousands of searches on Twitter and can say with absolute certainty that there is nothing certain about Twitter’s searches.

My guidelines for reliable Twitter Searches:

• You must perform the search within 24 hours to ensure that you get 100% of tweets that mention any given search term.
• Searches going back up to 3 days will be 95% accurate.
• Searches going back 2 weeks or more will fall to 90%.
• If your data needs require that you not miss a single tweet (because you never know which one you’ll miss) search every day.
• Stored searches don’t perform any better.  They are executed when you view them, not continuously.
• Searches become less accurate depending upon the load on Twitter for live traffic.  See this blog post about how Twitter dies to understand the interplay between search and live loads.
• Searching on a term for 1+ year ago is a crap shoot at best.  Only use this if your looking for big news with 10,000+ messages or more.

As an example I searched on tweets that mentioned my birthday for the last 3 years.  My searches returned zero results for 2008, 2009, and 2010.  However each year as my follower count grew and my Facebook friends grew I got more and more birthday wishes via Twitter.  As a matter of fact, I even got a happy birthday wish from @GaryVee this year.  So it doesn’t matter if you have 800,000 followers or 200, you tweets will be forgotten by Twitter’s search.

How can you determine if  Twitter search is a problem for you?

1. Look up an old tweet you sent or received (easy if you use Favorites)  Anything over 3 months old will do.
2. Pick something with a unique phrase that’s unusual so you won’t find too many others like it.  You can also pick time in your life when you had an event that was mentioned by others on Twitter- like a birthday.
3. Go to search.twitter.com/advanced and then enter some information to run the search.  For my birthday search I entered the following: (This is the most broad search which should find anything close to what I wanted.)
   1. Any of these words= birthday
   2. Referencing this person= ckieff (leave off the “@”)
   3. Since this date= June 14 2010
   4. Until this date= June 16 2010
4. Run the search.  Get zero results.  Lament the fact that everyone forgot your birthday.

What about Google and Bing?

“So what?”  you say, “I don’t care because I’ll just use Google.  Google’s got great searches and they can store everything!”  Great idea, except Google and Bing use the Twitter Search API to return results.   Run the same searches on those search engines to see if you find anything different.

The Bottom Line for Twitter Search:

Twitter Search data is only reliable for 24 hours to 3 days at a maximum, in my experience.

Don’t attempt any market research, historical data collection or other research with Twitter data unless you have collected and archived it yourself.  You can collect Twitter data using any of the analytics tools, Radian 6, Tap 11, Keen Kong, etc. and then look at this data later.  You can’t perform what-if’s if you didn’t collect that data before.

You can perform very broad popular topic searches on historical data.  However, note that your data is not accurate or complete if you depend upon Twitter Search.  Qualitative analysis is most likely fairly reliable with large samples, but quantitative analysis should be avoided.   That means you can say that most people feel this way, but you should avoid saying 5786 people said this…  Because you’ll be wrong.

Now why is Twitter search so terrible?  Beats the hell out of me!  Seems Twitter could have hired someone from Google or Yahoo, etc. to manage their data at some point along the way.

Can you give me 1 Good Reason why Twitter’s Search is so terrible?

Update

My Twitter buddy @mdy directed me to this page on Twitter’s help site which states among other things that:

“Due to resource constraints, not every tweet can be indexed in Twitter Search. Our engineers are always working to index more updates, but we can’t force individual, missing Tweets into search. Rest assured that your followers will still see all your updates and @replies, and we’ll update on this page as we improve this issue.”

So now that we know Twitter is fully aware of the issues, we can all rest easily…

Share this: