Takuya Homma's Bookmarks tagged Search →  View Popular

<!--NO VIEW-->

"We test almost everything, even things that you would think are so small that we could not possibly care (nor could they possibly matter). In fact, small changes do matter, and we do care," said Ben Gomes, a Google distinguished engineer, in a blog post.

"I believe this notion of understanding user intent--being able to analyze (search queries) and come up with search patterns and use them to shape the search experience--is one of the most important areas for us," Nadella said.

"If they wanted to triple size of their index, they'd have to triple the size of every
server and cluster. It's not easy or fast," said Patterson.

In a follow-up, Cuil added that Google being as large as they estimated
it to be now was largely down to Patterson's work at Google, and since she's
no longer there, increasing the index size will be a "non-trivial" exercise.

Unfortunately, Google started reacting to Cuil's claims even before Cuil
made them. In a post
on Friday, Google just so happened to decide it was time to mention they
"knew" of 1 trillion items on the web. That will confuse some people into
thinking Google has indexed 1 trillion documents, even though they don't say
this. What Google did say clearly was:

We don't index every one of those trillion pages -- many of them are
similar to each other, or represent auto-generated content similar to the
calendar example that isn't very useful to searchers. But we're proud to
have the most comprehensive index of any search engine, and our goal always
has been to index all the world's data.

On Monday, their company, Cuil, is unveiling a search engine that they promise will be more comprehensive than Google’s and that they hope will give its users more relevant results.

“This is the most promising thing I’ve seen in a while,” said Danny Sullivan, who has followed the online search business for more than a decade and is the editor of Search Engine Land. “Whether they are going to threaten Microsoft, much less Google, that’s another story.”

Google, with its deep reservoir of data about online behavior gathered by tracking hundreds of millions of computers, is for the first time testing ways to use some of that data to aim ads at Web users.

　今後、広告主側がこの変化に対応していくためには、「ユーザーの行動予測に基づく誘導シナリオ」を構築するという考え方を持つことが重要になってくるのではないでしょうか

　すなわち、これまでの「キーワード」と「広告」の「点」における適合性に加えて、今後は「検索行動」と「誘導シナリオ」の「線」における適合性も踏まえたリスティング広告戦略を持つことが重要になってくると考えています。

To Microsoft, Facebook is a quick way to expand the audience for its search engine. More than 29 million people actively use Facebook in the United States. They will soon see prominent displays of Microsoft’s Live Search box on their friends’ and their own Facebook pages.

The deal fills a significant gap in Facebook’s service. When people are using the social network and have questions, they have to leave Facebook to go to a search engine. With the new search toolbar on the page, users can stay on Facebook while making their queries.

The heart of the matter is this: how do you measure the quality of search results? One of the essential requirements to train any machine learning model is a a set of observations (in this case, queries and results) that are tagged with "scores" that measure the goodness of the results. (Technically this requirement applies only to so-called "supervised learning" approaches, but those are the ones we are discussing here.) Where to get this data?

It came as a great surprise to me that Google relies on a small panel of raters rather than harness their massive usage data. But in retrospect, perhaps it is not so surprising. Two forces appear to be at work. The first is that we have all been trained to trust Google and click on the first result no matter what. So ranking models that make slight changes in ranking may not produce significant swings in the measured usage data. The second, more interesting, factor is that users don't know what they're missing.

• Navigational queries, where the user is looking for a specific uber-authoritative website. e.g., "stanford university". In such cases, the user can very quickly tell the best result from the others -- and it's usually the first result on major search engines.

• Informational queries, where the user has a broader topic. e.g., "diabetes pregnancy". In this case, there is no single right answer. Suppose there's a really fantastic result on page 4, that provides better information any of the results on the first three pages. Most users will not even know this result exists! Therefore, their usage behavior does not actually provide the best feedback on the rankings.

I'll close with an interesting vignette. A couple of years ago, Yahoo was making great strides in search relevance, while Google apparently was not improving as fast. Recall then that Yahoo trumpeted data showing their results were better than Google's. Well, the Google team was quite amazed, because their data showed just the opposite: their results were better than Yahoo's. They couldn't both be right -- or could they? It turns out that Yahoo's benchmark contained queries drawn from Yahoo search logs, and Google's benchmark likewise contained queries drawn from Google search logs. The Yahoo ranking algorithm performed better on the Yahoo benchmark and the Google algorithm performed better on the Google benchmark.

Peter suggests two reasons for this. The first is hubris: the human experts who created the algorithm believe they can do better than a machine-learned model. The second reason is more interesting. Google's search team worries that machine-learned models may be susceptible to catastrophic errors on searches that look very different from the training data. They believe the manually crafted model is less susceptible to such catastrophic errors on unforeseen query types.