Search Wars: 2023

Bing, Bard, and the race for AI-powered search supremacy

Thanks to our 565 subscribers, +90 since our last post (+20%)

Are your friends/family/colleagues interested in intelligent apps? Please share!

Last week we witnessed perhaps the most interesting five-day period in the search industry in ~20 years. On February 7th, Microsoft held a surprise event at their headquarters in Redmond to announce AI-powered Microsoft Bing and Edge, a “copilot for the web”, the culmination (to date) of Microsoft’s early and prescient partnership with OpenAI. Microsoft CEO Satya Nadella compared the product releases akin to “2007-2008, when the cloud was just first coming out”, and Sam Altman (CEO of OpenAI) remarked he had “been waiting for this for 20 years, so I’m very happy it’s here.”

Meanwhile, the longtime reigning heavyweight champion in search, Google, had a slightly more haphazard counterpunch. Anticipating Microsoft’s event, CEO Sundar Pichai published a blog post a day earlier on Feb 6th introducing Bard, Google’s experimental conversational AI service. They then followed that up with a livestreamed event in Paris led by Google’s head of search Prabhakar Raghavan revealing more details about Bard with live demos (one of which went not so well).

All in all, it was an intense week with two of the world’s largest companies reigniting their positions in one of the world’s biggest and most lucrative markets: search. While a range of players have battled for search primacy going back to the 1990s (remember AltaVista?), the key difference today is how artificial intelligence and large language models are rewriting what is possible in the category.

Why are we seeing a resurgent interest in search? Where can AI actually move the needle? Why does all of this even matter? Let’s dig in.

Ethan Caballero @ethanCaballero

"This new Bing will make Google come out and dance, and I want people to know that we made them dance." - @satyanadella

How Did We Get Here?

“Search wars” is not a new concept. Ever since the birth of the modern Internet in the 1990s, a host of various companies have fought to be the primary search engine users flock to. Take, for example, the below quote from an industry observer:

Today, we’re witnessing a clash of the search titans or search superpowers. Call them what you like — Google, Yahoo and Microsoft — are major players squaring off to control web search technology and hopefully, secure their own destinies.

What year do you think the observer is referring to? Without the dead giveaway of Yahoo being called a “superpower”, one wouldn’t be mistaken to think this was a reference to the current clash between Google and Microsoft. In fact, it’s from an April 2004 article by Danny Sullivan distinguishing the “new” (at the time) search engines from the web portals of the mid-90s. The article was written just six years after Google was founded and months before the IPO, but already the battle for search supremacy was brewing.

So how did we get here? While there were search engines dating back to 1990 like Archie, it was really on in the mid-90s when the “modern” search engine began to take form, with companies like Excite, Lycos, AltaVista, and AskJeeves popularizing the form factor of inputting a query and generating a list of results that most closely matched the input. These early engines were generally bot-based and used a measure of how many times the exact word or phrase being queried would appear on a company’s website, often returning good but not great answers.

The beloved Jeeves

Then everything changed with the founding of Google in 1998. Sergey Brin and Larry Page famously introduced a completely revolutionary approach to search which they called PageRank. Instead of simply counting the number of times a search item appeared on a webpage, PageRank measured how many relevant websites linked to a site, scored the website from 0-10, and used that scoring to prioritize which websites to return from a query. Essentially overnight, Google became the fastest, easiest, and most comprehensive way to search the Internet.

Fast forward to today, and Google continues to dominate the nearly $200B search engine category, with 92% market share (trailed by Microsoft which at #2 only accounts for 3%). And it’s an incredibly lucrative business. Google Search alone had revenues of $162B in 2022 and accounts for the vast majority of Alphabet’s (Google’s parent company) $60B in free cash flow. It is such an incredibly powerful cash machine that it has been called a monopoly many times.

But the desirability of search cannot be measured by financials alone. The dominant search engine also becomes the central hub where users organize their Internet activity around. It is where vast quantities of new data and knowledge freely flow to. It benefits from vast network effects and has a global reach. And it has become the next destination artificial intelligence will attempt to crack wide open…

AI Enters the Chat

As users, we have all experienced the magic of search. Open a new web page, type in a question, and within milliseconds, you have your answer. Search appears so commonplace given how integrated it is into our everyday lives, but how does search actually work? While search engines have become more complex over the last decade, the majority of search engines today still follow four main elements – crawling, indexing, querying, and ranking:

1. Crawling – Locating new content online (websites, links, images, etc.) and determining the worthiness of the content to be indexed.

2. Indexing – Storing information from the crawler in a database, and analyzing the content (images, text, video files, etc.).

3. Querying – When querying a search engine, the search engine needs to translate the query into terms that relate to its index to understand what it’s looking for. The search engine looks through its index to find web pages that match the written query.

4. Ranking & Ad Engine – The search engine presents a list of the most relevant results and prioritizes the results in what the engine perceives as the best answer. Query processor interacts with the ad processor in real time to rank the top and most relevant responses and offers personalized recommendations based on specific search queries.

Over the last five years, new technological advancements in deep learning and natural language processing have changed how search works. Chief among these are:

• Text Representation & Embedding: Text representation is the process of transforming text into a vector form. Through this process, simple text is converted into an array of numbers known as "text embeddings". Embeddings are useful as now text can be more easily compared to other text for similarity (numbers matching numbers vs. words matching words). This leads to more effective searches as matches are no longer solely based on keywords, but rather on semantics (i.e. context, themes, and concepts). User behavior and intent become more closely recognized. We anticipate text representation and embeddings will increasingly become a core part of AI-powered search, enabling everything from better product recommendations to travel itineraries and document retrieval.

Turning text into embeddings (per Cohere)

• Text Generation and Summarization: Large Language Models and Foundation Models can summarize data from hundreds of sources to give users a comprehensive and accurate view of their search results. For instance, a search engine can be asked to "find 10 different options of road bikes". LLMs/FMs can summarize results from multiple sources (e.g. REI top reviews, sport enthusiast magazines, YouTube reviews) and rank the top 10 road bikes according to the user's budget, riding ability, riding style, and other preferences, without having to read through hundreds of reviews. Additionally, LLMs/FMs can also change the way results are displayed. Instead of a string of text or a list of bikes with HTTP links, the user may be able to view the image of each bike, along with a chart comparing and contrasting all the different specs and prices.

• Co-Pilot / AI assistant: For more complex searches or multi-step searches, an AI assistant or a “ChatGPT” like search experience is now possible. This co-pilot assistant can answer queries for you, but may also be able to complete actions through its generative capabilities (help you write code, create new images, or create text content). Compared with needing to go into a new search browser, this AI assistant may be available on the edge or embedded directly into your workflow (Slack, Email, PowerPoint, etc.). Imagine an AI assistant that connects across all your data sources and learns in the background as you continue interacting with it.

New Microsoft Edge & Bing Search Experience

A New Arms Race

If the first era of search wars in the early to mid-1990s was characterized by the shift from web portals to search engines, and the second era defined by the displacement of bot-based classifications by PageRank, we believe the current epoch of search is being heralded by the arrival of AI and advancements in large language models.

While Microsoft threw down the gauntlet with AI-powered Bing and Edge, we are seeing a crop of new “Davids” challenging the “Goliaths”, ranging from You.com, Neeva, Perplexity AI and Quora (which announced Quora Poe). From our vantage point, there are several areas worth thinking through in the battle for search dominance:

• Distribution still matters → Part of Google’s dominance in search comes from its status as the default search engine in Safari (thanks to a $15B annual deal with Apple). For Microsoft, or any other challengers, cracking the distribution nut is pertinent. It is unlikely users will open a separate app for search, so “owning” the browser experience will become increasingly important.

• New monetization streams → Search has long relied on advertising as the key source of monetization. We believe that with the entrance of LLMs and new players in the market, we will see new monetization strategies emerge less tied to private data and scorned tactics like cookie tracing. For example, You.com has talked about potentially opening up new revenue streams like subscriptions, app platforms for developers, and private advertising.

• Vertical or Use Case Specific → We have long been accustomed to all search being done through Google given its dominant position, but perhaps as the market becomes more diffuse, we will see several search engines depending on the vertical or use case being queried. If you’re looking for travel ideas, you may run that search in a different platform that if you’re running a stock screener. Hard to imagine now, but this is where LLMs and multi-modality could make a real difference in the user search experience.

• Handling the Edge Cases → As with any other system leveraging LLMs, we are quickly discovering that AI chat can get weird, and scary, fast. NYT columnist Kevin Roose spent two hours conversing with Bing’s AI, which at one point tried to convince him he was unhappy in his marriage and that he should leave his wife. While these types of responses are often in the tiny minority of interactions, they grab headlines and can dominate the conversation. How the search engines handle and fix these “spooky” edge cases will become important in establishing credibility.

A gaslighting AI?

It feels like we are now about to enter a new era of the Search Wars, and for the first time in two decades, Google’s near-monopoly is finally being challenged.

Buckle up!!

Funding News

Below we highlight select private funding announcements across the Intelligent Applications sector. These deals include private Intelligent Application companies who have raised in the last two weeks, are HQ’d in the U.S. or Canada, and have raised a Seed - Series E round.

New Deal Announcements - 02/03/2023 - 02/16/2023:

We hope you enjoyed this edition of Aspiring for Intelligence, and we will see you again in two weeks! This is a quickly evolving category, and we welcome any and all feedback around the viewpoints and theses expressed in this newsletter (as well as what you would like us to cover in future writeups). And it goes without saying but if you are building the next great intelligent application and want to chat, drop us a line!