Just getting back in the groove after SES San Jose. You may have caught some of my sessions or heard we have released a white paper looking at the future of search and with some eye tracking on personalized and universal search results. We don’t have the final version up yet, but it should be available later this week. The sneak preview got rave reviews in SJ.

Anyway, I interviewed a number of influencers in the space, and I’ll be posting the full transcripts here on my blog over the next week. I already posted Jakob Nielsen’s interview. Today I’ll be posting Marissa Mayer’s, who did a keynote at SES SJ. It makes for interesting reading. Also, I’ll be running excerpts and additional commentary on Just Behave on Search Engine Land. The first half ran a couple weeks ago. Look for more (and a more regular blog schedule) coming out over the next few weeks. Summer’s over and it’s back to work.

Here’s my chat with Marissa:

Gord: I guess I have one big question that will probably break out into a few smaller questions.  What I wanted to do for Search Engine Land is speculate on what the search engine results page might look like to the user in three years time.  With some of the emerging things like personalization and universal search results and some the things that are happening with the other engines: Ask with their 3D Search, which is their flavor of Universal, it seems to me that we might be at a point for the first time in a long time the results that we’re seeing may have a significant amount of flux over the next 3 years.  I wanted to talk to a few people in the industry about their thoughts of what we might be seeing 3 years down the road.  So that’s the big over-arching question I’m posing.

Marissa: Sure, Minority Report on search result pages…Well, I’d like to say it’s going to be like that but I think that’s a little further out.  There are some really fascinating technologies that I don’t know if you’ve seen..some work being done by a guy named Jeff Han?

Gord: No.

Marissa: So I ran into Jeff Han both of the past years at TED. Basically he was doing multi-touch before they did it on the iPhone on a giant wall sized screen, so it actually does look a lot like Minority Report. It was this big space where you could interact, you could annotate, you could do all those things.  But let me talk first about what I see happening as some trends that are going to drive change.

One is that we are seeing more and more broadband usage and I think in three years everyone will be on very fast connections, so a lot more to choose from and  a lot more data without taking a large latency hit.  The other thing we’re seeing is different mediums, audio, video.  They used to not work.  If you remember getting back a year ago, everytime you clicked on an audio file or a movie file, it would be, like, ‘thunk’?  It needs a plug in, or “thunk”, it doesn’t work.  Now we’re coming into some standardized formats and players that are either browser or technology independent enough, or are integrated enough that they are actually going to work.  And also we’re seeing users having more and more storage on their end.  And those are the sort of 3 computer science trends that are things that are going to change things.  I also think that people are becoming more and more inclined to annotate and interact with the web. It started with bloggers, and then it moved to mash ups, and now people are really starting to take a lot more ownership over their participation on the web and they want to annotate things, they want to mark it up.

So I think when you add these things together it means there’s a couple of things.  One, we will be able to have much more rich interaction with the search results pages. There might be layers of search results pages: take my results and show them on a map, take my results and show them to me on a timeline.  It’s basically the ability to interact in a really fast way, and take the results you have and see them in a new light.  So I think that that kind of interaction will be possible pretty easily and pretty likely.  I think it will be, hopefully, a layout that’s a little bit less linear and text based, even than our search results today and ultimately uses what I call the ‘sea of whiteness’ more in the middle of the page, and lays out in a more information dense way all the information from videos to audio reels to text, and so on and so forth.  So if you imagine the results page, instead of being long and linear, and having ten results on the page that you can scroll through to having ten very heterogeneous results, where we show each of those results in a form that really suits their medium, and in a more condensed format.  A couple of years ago we did a very interesting experiment here on the UI team where we took three or 4 different designs where the problem was artificially constrained.  It was above the fold Google.  If you needed to say everything that Google needed to say above the fold, how would you lay it out?  And some came in with two columns, but I think two columns is really hard when it was linear and text based.  When you started seeing some diagrams, some video, some news, some charts, you might actually have a page that looks and feels more like an interactive encyclopedia.

Gord: So, we’re almost going from a more linear presentation of results, very text based, to almost more of a portal presentation, but a personalized portal presentation.

Marissa: Right and I think as people, one, are getting more bandwidth and two, as they’re more savvy with how they look at more information, think of it this way, as more of serial access versus random access.  One of my pet peeves is broadcast news, where I really don’t like televised news anymore.  I like newspapers, and I like reading online because when I’m online or with newspapers, I have random access.  I can jump to whatever I’m most interested in.  And when you’re sitting there watching broadcast news you have to take it in the order, at the pace and at the speed that they are feeding it to you.  And yes, they try to make it better by having the little tickers at the bottom, but you can’t just jump in to what you’re interested in.  You can only read one piece of text at a time, and it’s hard to survey and scan and hone in on one type of medium or another when it’s all one medium.  So certainly there is some random access happening with the search results today.  I think as the results formats becomes much more heterogeneous, we’re going to have a more condensed presentation that allows for better random access.  Above the fold being really full of content, some text, some audio, some video, maybe even playing in place, and you see what grabs your attention, and pulls you in.   But it’s almost like random access on the front page of the New York Times, where am I more drawn to the picture, or the chart, or this piece of content down here?  What am I drawn to?

Gord: Right.  If you’re looking at different types of stimuli across the page, I guess what you’re saying is, as long as all that content is relevant to the query you can scan it more efficiently than you could with the standardized text based scanning, linear scanning, that we’re seeing now

Marissa: That’s right.

Gord: Ok.

Marissa: So the eyes follow and they just read and scan in a linear order, where when you start interweaving charts and pictures and text, people’s eyes can jump around more, and they can gravitate towards the medium that they understand best.

Gord: So, this is where Ask is going right now with their 3D search, where it’s broken it into 3 columns and they’re mixing images and text and different things.  So I guess what we’re looking at is taking it to the next extreme, making it a richer, more interactive experience, right?

Marissa: Rather than having three rote columns, it would actually be more organic.

Gord: So more dynamic.  And it mixes and matches the format based on the types of material it’s bringing back.

Marissa: Well, to keep hounding on the analogy of the front page of the New York Times.  It’s not like the New York Times…I mean they have basically the same layout each time, but it’s not like they have a column that only has this kind of content, and if it doesn’t fill the column, too bad.  They have a basic format that they change as it suits the information.

Gord: So in that kind of format, how much control does the user have? How much functionality do you put in the hands of the user?

Marissa: I think that, back to my third point, I think that people will be annotating search results pages and web pages a lot.  They’re going to be rating them, they’re going to be reviewing them.  They’re going to be marking them up, saying  “I want to come back to this one later”.  So we have some remedial forms of this in terms of Notebook now, but I imagine that we’re going to make notes right on the pages later.  People are going to be able to say I want to add a note here; I want to scribble something there, and you’ll be able to do that.  So I think the presentation is going to be largely based on our perceived notion of relevance, which of course leverages the user, in the ways they interact with the page, and look at what they do and that helps inform us as to what we should do.  So there is some UI user interaction, but the majority of user interaction will be on keeping that information and making it consumable in the best possible way.

Gord: Ok, and then if, like you said, if you go one step further, and provide multiple layers, so you could say, ok, plot my search results, if it’s a local search, plot my search results on a map.  There’s different ways to, at the user’s request, present that information, and they can have different layers that they can superimpose them on.

Marissa: So what I’m sort of imagining is that in the first basic search, you’re presented with a really rich general overview page, that interweaves all these different mediums, and on that page you have a few basic controls, so you could say, look, what really matters to me is the time dimension, or what really matters to me is the location dimension.  So do you want to see it on a timeline, do you want to see it on a map?

Gord: Ok, so taking a step further than what you do with your news results, or your blog search results, so you can sort them a couple of different ways, but then taking that and increasing the functionality so it’s a richer experience.

Marissa: It’s a richer experience. What’s nice about timeline and date as we’re currently experimenting with them on Google Experimental is not only do they allow you to sort differently, they allow you to visualize your results differently.  So if you see your results on a map, you can see the loci, so you can see this location is important to this query, and this location is really important to that query.  And when you look at it in time line you can see, “wow, this is a really hot topic for that decade”.  They just help you visualize the nut of information across all the results in these fundamentally different ways that ‘sorts’ kind of get at. But it’s really allowing that richer presentation and that overview of results on the meta level that helps you see it.

Gord: Ok.  I had a chance to talk to Jakob Nielsen about this on Friday, and he doesn’t believe that we’re going to be able to see much of a difference in the search results in 3 years.  He just doesn’t think that that can be accomplished in that time period.  What you’re talking about is a pretty drastic change from what we’re seeing today, and the search results that we’re seeing today haven’t changed that much in the last 10 years, as far as what the user is seeing.  You’re really feeling that this is possible?

Marissa: It’s interesting, you know, I pay am lot of attention to how the results look.  And I do think that change happens slowly over time and that there are little spurts of acceleration.  We at Google certainly saw a little accelerated push during May when we launched Universal Search.  I’m of the view that maybe its 3 years out, maybe it’s 5 years out, maybe it’s 10 years out.  I’m a big subscriber to the slogan that people tend to overestimate the short term and underestimate the long term.  My analogy to this is that when I was 5, I remember watching the Jetson’s and being, like, this rocks!  When I’m thirty there are flying cars!  Right?  And here I am, I’m 32 and we don’t even have a good flying car prototype, and yet the world has totally changed in ways that nobody expected because of the internet and computing.  In ways that in the 1980s no one even saw it coming.  Because personal computers were barely out, let alone the internet.  It’s interesting.  We do our off site in August. I do an offsite with my team where we do Google two years out. There it’s really interesting to see how people think about it.  I take all the prime members on my team, so they’re the senior engineers, and everybody has homework.  They have to do a homepage and a results page of Google, and this year it’ll be Google 2009.

Gord: Oh Cool!

Marissa: Six months out, it’s really easy because if we’re working on it, because if it’s going to launch in 6 months and it’s big enough that you would notice, we’re working on it right now and we know it’s coming.  And five years or ten years out we start getting into the bigger picture things like what I’m talking to you about.  When the little precursors that get us ready for those advances happen between now and then that’s what’s shifting.   So I’m giving you the big picture so you can start understanding what some of the mini steps that might happen in the next 3 years, to get us ready for that, would be.  The two to three year timeframe is painful. Everybody at my offsite said, “this timeframe sucks!” So it’s just far enough out that we don’t have great visibility, will mobile devices be something that’s a really big new factor in three years?  Maybe, maybe not.  Some of the things are making fast progress now may even take a big leap, right, like it was from 1994 to 97 on the internet.  Or if you think about G-mail and Maps, like AJAX applications..you wouldn’t have foreseen those in 2002 or 2003.  So, two or three years is a really painful time frame because some things are radically different, but probably in different ways than you would expect.  You have very low visibility in our industry to that time frame.  So I actually find it easier to talk about the six month timeframe, or the ten year timeframe.  So I’m giving you the ten year picture knowing that it’s not like the unveiling of a statue, where you can just take the sheet, snatch it off and go, “Voila there it is”.  If you look at the changes we’ve made over time at Google search they’ve always been “getting this ready, getting this ready”.  So the changes are very slow and feel like they’re very incremental.  But then you look at them in summation over 18 months or two years, you’re like, “you know, nothing felt really big along the way, but they are fundamentally different today”.

Gord: One last question.  So we’re looking at this much richer search experience where it’s more dynamic and fluid and there are different types of content being presented on the page.  Does advertising or the marketing message get mixed into that overall bucket, and does this open the door to significantly different types of presentation of the advertising message on the search results page?

Marissa: I think that there will be different types of advertising on the search results page.  As you know, my theory is always that the ads should match the search results.  So if you have text results, you have text ads, and if you have image results, you have image ads.  So as the page becomes richer, the ads also need to become richer, just so that they look alive and match the page.  That said, trust is a fundamental premise of search.  Search is a learning activity.  You think of Google and Ask and these other search engines as teachers.  As an end user the only reason learning and teaching works, the only way it works, is when you trust your teacher.  You know you’re getting the best information because it’s the best information, not because they have an agenda to mislead you or to make more money or to push you somewhere because of their own agenda.  So while I do think the ads will look different, they will look different in format, or they may look different in placement, I think our commitment to calling out very strongly where we have a monetary incentive and we may be biased will remain.  Our one promise on our search results page, and I think that will stand, is that we clearly mark the ads.  It’s very important to us that the users know what the ads are because it’s the disclosure of that bias, that ultimately builds the trust which is paramount to search

Gord: Ok.  Great to see you’re a keynote at San Jose in August.

Marissa: Should be fun.  This whole topic has me kind of jazzed up so maybe I’ll talk about that.