Opinion: web search companies won't take a cut in revenue to offer us a better search experience, especially since we don't pay for it

In over two and a half decades, internet search has not changed at all. It’s grown bigger in every sense, but it’s still basically the same: a short query, a list of web pages with embedded advertising, next query please. We are conditioned to this model and, while we all benefit from the engineering behind it, our search experience should be much improved by now.

We currently run over 3.5 billion searches on the internet every day, each answered in less than a quarter of a second. That’s a remarkable feat of engineering, considering the number of web pages on the internet – a figure so large it can no longer be tallied. The last count was in August 2005 with Google claiming an index of 8,168,684,336 pages


A search engine like Google or Bing doesn’t provide answers, but gives a list of web pages for us to visit. In turn, those pages should contain the information we’re looking for or, at worst, have links to take us to other pages which do. It’s a strange business model - providing pointers as a result of a search so we can go elsewhere - but it works. It’s hard to argue with Google’s 2017 advertising revenue of $32 billion.

The current process goes as follows. Positive factors boost a web page’s place in the ranking. These include links, frequency of updates, images, and its Panda ranking, an enormous machine learning application which has "learned" the aspects of a website that give user satisfaction. Negative factors push web pages down the ranking including website downtime, spelling errors in the page text, absence of images and a low number of visitors.

Half the time, we’re searching for stuff we’ve already seen

Those of us who care about the quality of searching know that there are better ways to search the internet. Currently, there is no carry-over from one search to the next, even from the same person at the same time and on the same device. If you are struggling to find something that you know exists, maybe because you saw it previously, internet search engines won’t connect your repeated searches as you struggle.That is because the business model is to serve fast, advertising-laden results, not to make intelligent connections from search to search. 

Half the time, we’re searching for stuff we’ve already seen. Future search should be personalised and universal, searching my email, documents, the parts of the web I’ve already browsed or seen, the news I’ve read and the videos I’ve seen.  

Another likely development for internet search is the generation of actual answers to questions rather than pointers to web pages. Google introduced the Knowledge Graph in 2013 which provides answers to likely questions about a searched topic. You can see this on a Google search results page for some topics. Search for "Leo Varadkar" and you’ll see his date of birth, partner, parents, siblings, education. This is great technology but its evolution into the search experience is disappointing. The arrival of voice-activated answering systems such as Amazon Echo may push further development.

Here's what you get when you Google "a picture of Leo Varadkar"

Almost by stealth, recommender systems have crept into the search landscape. We don’t always know what we’re looking for until we find it, hence the growth of systems that push content to us based on similarity to others.This was started by Amazon books (you bought books X and Y so we recommend book Z) and now it is the most common way for us to find our next Netflix movie to watch. The most sophisticated of the recommender systems is from Spotify who compose a playlist weekly using melody, lyrics, genre and artist. 

Finally, the future for web browsing is searching other media, like images and videos. Using machine learning and artificial intelligence techniques, as we’ve been doing for more than a decade at the Insight Centre for Data Analytics, we now have automatic tagging and captions for images and videos. Now your computer can recognise images without an accompanying text description and describe it automatically. This a major boon for internet search that will only get more sophisticated.

Lots of web page returns and clicks means lots of ad spend

The only thing stopping these search features from going mainstream is advertising. If I ask a single question – "give me a picture of Leo Varadkar" – and the web gives me exactly what I want and nothing more, how does a search company shoehorn advertising into this narrow communication? Not easily, and certainly not without a great deal of disruption.

Lots of web page returns and clicks means lots of ad spend. Web search companies won't take a cut in revenue to offer us a more satisfying search experience, especially since we don't pay for it. Until search companies can crack this challenge, we may be stuck with this generation web searching for another two and a half decades.

The views expressed here are those of the author and do not represent or reflect the views of RTÉ