FEATURE: Search Engines Get Personal

by: Eileen Kowalski, October 2004

Toted as the "next big wave" in search at Search Engine Strategies (SES) in San Jose, personalized search looks like it may be the beginning of a tidal wave. The motivations for creating a search engine that yields more personal results seem simple enough:

• the search engine business is becoming more competitive;
• the online information pool is expanding daily;
• and the online population is growing and becoming increasingly diverse.

Especially after a ChoiceStream study found that 81% of users say they want personalized content and 64% were willing to provide personal data to get it, the time for personalized search has come. However, the big question is: what exactly is "personalized" search?

Search Filtered by a Personal Profile

Google rolled out the beta version of their personalized search in late March this year. The service allows you to select interests to search the Google index through a personalized filter. It is a great concept. However, the taxonomy of interests is limited and does not allow you to add custom categories. For example, the closest you can get to "SEO" or "SEM" as an interest is "Internet". Hopefully, this limitation is just the result of an early fear that people would not want to provide search engines with specific personal information to get results. After all, once you own Orkut, doesn’t it seem redundant to make people form their personal profile inside a rigid taxonomy?

Searching Inside a Personalized Index

A9, MyJeeves, My Yahoo! Search and Furl all allow you to build an index of your own, enhancing the search engine’s knowledge of user behavior at the same time. A9 saves and searches your history, bookmarks and web diary entries. Plus, the A9 toolbar tracks your every move online to get a full picture of your browsing habits. This information fuels A9’s Discover function which provides you with "personalized" results once a day, suggesting "Related Sites" and human–created "Categories" to check out much like Amazon suggests books. Likewise, MyJeeves allows you to save, notate, categorize, print, e-mail and search within your saved search results.

Although it is not being marketed as a personalized search engine, Furl provides a richer personal index than A9 or MyJeeves. The beauty of Furl is that you simply add "Furl It" to your bookmarks toolbar and click to add any website to your personal index with comments, keywords and a quality rating. Furl gives you more control over your personal index than A9 whose toolbar automatically adds every site you view to your history. And it is more versatile than MyJeeves, which only allows you to add MyJeeves search engine results to your personal index. Plus, Furl archives every website that you add to your personal index, so you can find information inside your archive even if a website is taken down or moved without a proper redirect.

Still, Yahoo seems to have taken a suggestion made on Google’s beta testing board and the personalized indexing ideas manifested in A9, MyJeeves and Furl to build potentially the best personalized search to date for users and the scariest for site owners and SEOs. The personalized engine that Yahoo released this month allows you to create your own categorized index by saving search results (like A9 and MyJeeves) and add pages that you encounter while surfing through a "Save Y! My Web" button that you can add to your bookmarks bar (like Furl). However, Yahoo’s clincher is the ability to block certain sites from ever appearing in your web–wide search results. From an SEO standpoint, it is at once the best thing ever and a complete nightmare. If someone is spamming Yahoo and ranking with an irrelevant webpage, it takes seconds to block that result and all other results from the entire site from appearing in your search results. No more search engine spam. However, the key here is that if you block any page from a website, results from the entire domain or subdomain will also be blocked from all search results for any keyword from that point forward. This begs the question: How can a site be redeemed once it is blocked? What if an unscrupulous SEO firm has added spammed–out doorway pages to a business owner’s domain without their knowledge? By the time that the business owner calls in another SEO to clean up their site, is it too late? There is an "Unblock" option inside Yahoo’s personalized search, but how do you alert people that the site is "under new management". Even more frightening is the possibility of purchasing a new domain name, only to find that the practices of the former website owner have caused that site to be blocked out by thousands of users without your knowledge (and possibly be penalized in the main search index because Yahoo knows that thousands of users find it irrelevant). Will webmasters soon need to do "credit checks" via archive.org before purchasing a domain name? Hopefully, there will be some kind of answer to this question before Yahoo’s personalized search comes out of beta and is fully integrated into My Yahoo!.

NOTE: In taking each of these search engines for a test run, I noticed that A9, MyJeeves and My Yahoo! Search appear to be searching personal indexes with a watered–down version of their full search engine algorithms. With http://www.searchenginewatch.com as a "Saved Result" in MyJeeves and My Yahoo! and as a "Bookmark" in A9, all of the engines failed to return any "personalized" results when I searched for "Danny Sullivan". Meanwhile, doing an full web search on each engine for the same keyword returned the correct website: http://www.searchenginewatch.com. Surprisingly, the only search engine that yielded the right result was Furl, a personal web service owned by LookSmart that does not profess to do personalized search. Obviously, some serious beta testing needs to bring the others up to code.

Personalized Search Gets Social

One of the most interesting developments in personalized search though, has been its integration with social software models. Furl incorporates a few social software elements, creating a community where members share indexes via RSS feeds and the website recommends "furlmates" based on what you have chosen to index. However, Eurekster is the recognized innovator in this arena. Inside Eurekster, like–minded individuals form "information nations" where the search habits of the population influence the search results inside that nation, which is supposed to be bona fide online community with a message board and a blog. What is really interesting is that each information nation can choose how it is governed:

• by anarchy where everyone’s behavior influences the results and everyone has moderator rights.
• by governed democracy where everyone’s behavior influences results and only the Founder has moderator rights
• by autocracy where only the Founder's behavior influences results and only the Founder has moderator rights.

So far the most popular information nations on Eurekster are Rugby, Fishing and EMI Music nations, which only speaks to how much this service is still in beta. However, I would keep an eye on Eurekster. The service will either attract quality members interested in providing each other with relevant search results and stimulating conversation or just be the latest online toy flooded with irrelevant results and link spam. The gamble with any social software system is that the website can only be as good as the people who use it.

Search Engine Optimizing for Personal Search

So how do you optimize a website to do well in personalized search engines?

1. Write an attractive title. Achieving a high click–through rate is especially important to success in engines like A9 where visited sites are automatically added to a user’s history and it will encourage people in MyJeeves to save, print or e–mail your search result.


2. Always 301 redirect moved webpages or at least a create custom 404 page for your site. The probability of people bookmarking and saving old search results with your old URL has gone way up with the popularity of personalized index creation. Be sure to 301 redirect your old URL to your new URL or at least create a custom 404 page for your site, so people can follow old links to the new page.


3. Check the history on new domains. Until My Yahoo! Search comes out some kind of assurance that all blocks will be removed from a specific domain once it has transferred ownership, check the history of your domain and only use domain names with no history or the best history possible.


4. Be relevant. Being irrelevant or having a poor quality page may result in a person blocking your search engine result in My Yahoo! Search, which blocks all of your website’s results for that person. Keep it relevant and remember that with personalized search it takes seconds to lose a potential customer and possibly forever to get one back.


5. Maintain good customer relationships. Personalized indexes and social ranking algorithms re–emphasize the importance of building lasting relationships with your customers. If you treat your customers badly, not only will you lose your place in personalized indexes, but you may find your website blocked from personal indexes or being e–mailed and streamed to friends and family with a note warning others not to buy from you.


6. Get involved. Join a Eurekster information nation that is relevant to your site and search for your own site. Once it is on the recently searched list, see how the community reacts to it. Open a Furl account and highlight important articles from your website on your public Furl index page. Then find people who have similar interests. Personalized search has opened up a whole new sandbox to play in.