Joel Liu's List: Search Application research

• Me.dium offers a service in which users can see what Web sites their friends are visiting. That allows the company to collect information about what sites have “buzz” at any given time, said Kimbal Musk, the company’s chief executive. Me.dium will use that information to rearrange and supplement Yahoo’s search results, creating a service that captures the “social zeitgeist” of the Web, Mr. Musk said.

• “We think that for a good percentage of searches, we’ll get people to where they want to go a lot faster than regular search engines,” Mr. Musk said. He said it would have been impossible for Me.dium to create a search service on its own.

  Yahoo estimates that it would cost $300 million to build a search service from scratch.

• Yahoo executives acknowledged that the new strategy, if successful, could cannibalize Yahoo’s own search business. But they said that if a search start-up became popular, it would probably take more users away from Google than from Yahoo, as Google has a far larger share of the market.

  “We did a lot of analysis,” said Bill Michels, senior director for Yahoo’s open search platform. “We are comfortable with the risks associated with it.”

1 more annotation...

• Three levels, but only BOSS Custom has real potential for a highly differentiated service offering.

   

  There are three levels to the BOSS program, according to SearchEngineWatch:

   
   
  • self-service API
   
  • BOSS University for academics
   
  • BOSS Custom, designed for companies with their own ranking and/or presentation methodologies. Or alternative, companies with proprietary data that can help as an additional signal that factors into relevancy.
   
   

  I’ll go over all the aspects of the BOSS program below, and then come back to BOSS Custom as evidence that Yahoo! just might Use The Force. But the basic features looks like a free version of Google Custom Search Engine.
• 
• Crawling and Indexing: Not the real barrier to vertical search
• Building a repository of documents is only the beginning. The key is extracting meaning from the documents (and the relationships between the documents) to power your ranking algorithm.

• Me.dium is the example highlighted by Yahoo! Silicon Alley Insider points out that Me.dium is adding social signals to Yahoo! ranking and calling it “Social Search” which ironically is “using a name Yahoo! has already attached to a failed product.” VentureBeat has a more positive spin on the Me.dium demo application, although Dan Kaplan concludes:

   

   

  The question that hangs over Yahoo, BOSS, and Me.dium is whether or not any search player will really be able to change user behavior and get people to consistently use something other than Google; the results would probably have to be a noticeable leap forward, and even then, it would be hard to break Googling habits.

   

   

  Dan is absolutely right, and BOSS will have to go a lot further to deliver a true “leap forward.”
• As an added plus, Yahoo! is providing the BOSS Mashup Framework. According to Yahoo! Search Blog: “We’re releasing a Python library and UI templates that allow developers to easily mashup BOSS search results with other public data source”
• Real-time indexing is appropriate for time-sensitive information like news and blogs. This could be an advantage over other wholesale crawling and indexing methods that other private label providers are providing.

• Integrating query suggestions (Search Assist technology)

   

  Query parsing services is one of the most interesting aspects of BOSS Custom. Search Assist is very well done already, and if it could be integrated with our custom, industry-specific ontology.
• There are many ways of making search results more relevant. One is to understand the user intention better and refine the search, which structured search addresses. Another way is to build the ontology/taxonomy/directory to classify queries and documents, which can benefit from the categorizer.
• Crawling is not a barrier to vertical search engine. It is true. However, first, it still makes an under-funded startup to get start easily; second, Yahoo data helps on time-sensitive data; third, some startups may want to try semi-horizontal market, such as content for kids and shopping, which is more costly to crawl.

• Huanjin: In general, I think this is a very important event for all search startups. It opens new opportunities. Google has pushed search relevance to the limit that the current approach can achieve. To significantly improve the search relevance, we have to take drastically different approaches, i.e., not solely by keyword matching, not solely by statistical signals, and not treating every user the same. My take on the direction of search technology is

   

  (1) Meaning based (semantic and/or syntactic).

   

  (2) Context aware. The same word means different things in different contexts.

   

  (3) Must be able to treat different users differently

   

  (4) Opinion based.

   

  (5) Leverage human knowledge base
• The BOSS Custom roadmap needs to envision supporting these innovative approaches, because that is the only way that new startups can create a reason for people to leave their horizontal search engine like Google and adopt an alternative.
• Andrew:  “The extreme approach - well not even that extreme these days, given Facebook - would be to let developers build extensions to the search engine that actually run on top of the *.yahoo.com domain. They can provide an API, do app approvals, and direct only small bits of traffic to each app to test them out - then ramp up the ones that perform better than anything else. There are difficult pieces necessary to make this work, but if done well, it has the potential to change the search game by letting developers target small groups of queries the way that advertisers have been able to.”

• Most niche search engines lack a traffic strategy that envisions coexisting with Google and Yahoo! Getting search traffic from existing engines needs to be a component of their business vision, as it is with Uptake.

   

  Beyond that, discoverability has been the biggest issue for the alternative search engines. ASE’s underestimate the brand loyalty that Google has earned that goes beyond the real advantage in search quality. If Yahoo! could play a role in that discoverability, through Search Monkey and other programs, that could be an advantage to the alts AND also to Yahoo!

   

  This is a two step process. Open up search through BOSS Custom to allow innovation to bloom. Then harness that innovation back into differentiating Yahoo! as a search and discovery platform that leverages this innovation to be different from Google. Yahoo! funnels traffic to BOSS alts, but then consumers use Yahoo! and build brand preference to Yahoo! and its rag-tag Rebel Alliance fleet!

12 more annotations...

• There are three levels to the BOSS program. The first is a self-service API, which will be available to partners who want to build their own search engine using Yahoo results as a base. Examples could include social, vertical, or visual search engines. The second is an API program for academics, dubbed BOSS University. Yahoo is partnering with the computer science departments of several top universities to allow them to use the BOSS program in their research.

• The third tier is BOSS Custom, a program where Yahoo will work with a very limited set of partners to customize their integration of Yahoo's results into their own search engines. These partners will generally fit into two main categories, Raghavan said. The first includes companies with their own ranking and/or presentation methodologies, such as semantic search engine Hakia. Hakia is using Yahoo's results, to which it applies its own "secret sauce," he said. Hakia is not replacing its own indexing process completely, but rather using BOSS to accelerate the process.

    

  The other category of partner includes companies with proprietary data, such as user profiles or behavior, that can be used to affect search results. One such company is Me.dium, a browser toolbar that lets users connect with each other and find related sites recommended based on other users' surfing habits. Me.dium will use Yahoo's BOSS data to create a social search engine that will rank results based on what its users say is important.

• Related Suggestion 

    The Related Suggestion service returns suggested queries to extend the power of a submitted query, providing variations on a theme to help you dig deeper.