Say goodbye to the Information Age. It’s already over.

Ok, not over, completely. These things take time. You don’t just end an “Age” in an instant. But take a look around you and you’ll start to see the beginnings of the end, just about everywhere.

The last Great Age, the Industrial Age, started sometime around the invention of the steam engine, rallied with the internal combustion engine, and soared into its zenith with the jet engines.

We briefly flirted with an Atomic Age, which lasted about ten years before perhaps morphing into the Nuclear Age. You might have thought we got post nuclear when we stopped worrying about the Bomb and won the cold war, but there appears to be a nuclear renaissance ahead, thanks to both proliferation and the resurgence of nuclear energy (no green house gases, just glowing green waste…). Somewhere overlapping the Nuclear Age was about a decade we call the Space Age–thanks to Kennedy’s push to put a man on the moon by the end of the 60’s.

But the Atomic Age, the Nuclear Age, even the Space Age, they don’t mark profound changes in the average human life.

I won’t get into the uneven distribution of “progress” in today’s world, but let me make a brief signpost to point out that when I say “we”, I mean those of us blessed enough to participate generally in the first world.

When we figured out how to create factories, how to industrialize production, we profoundly changed the way we live and work. The Atomic age didn’t do that. It mostly affected the political and international landscape leaving the rest of us to continue our modern industrial lifestyle. So, not a Great Age. Just an age. Similarly, the Space Age may have inspired TV shows and science fiction, complete with new global heroes both real and imagined, but other than Tang and Teflon, it had minimal impact on our daily lives.

In contrast, the Information Age has radically changed the modern lifestyle. Computers and networks and telecommunications have profoundly disrupted and re-invented some of the most important elements of our society. How we buy and sell. How we communicate. How we socialize. How we commit crimes and how we enforce our laws. Even how we sin and how we fall in love. Without doubt the Information Age is one of the Great Ages, affecting the lives of almost everyone fortunate enough to be living in the first world.

So, how can I say it is over? What about the iPhone? Google? Web 2.0? Second Life and World of Warcraft? Or even one of my own favorite initiatives, Vendor Relationship Management (VRM)? Aren’t we still in the thick of it?

It is true that the machinery of the Information Age is here to stay, both in terms of infrastructure and social structure. After all, we didn’t give up factories or unions at the end of the Industrial Age. Nor did we abandon farming when we moved from the Agrarian Age to the Industrial–although we have lost “barn raising” as a social activity. Farms and factories became less important in the modern context as more people found ways to live without them as a major part of their life. Instead, we reinvented both in the image of the new age. We industrialized our farms and re-engineered our factories around information.

We move from one great age to another when, as a Society, we let go of the trappings of the previous age and begin to define ourselves in new terms, absent the defining elements of yesteryear. We no longer think of ourselves as farmers or factory workers… the Information Age has knowledge workers, and we largely define ourselves by the information accessories in our lifestyle: our iPod, our MySpace page, our blog, when in previous ages it may have been our car, our company, or our home town, livestock or crop.

But aren’t these defining accessories Information Age artifacts? Yep. And while they may seem hot today, their days are already numbered. I don’t know how to see the future to what the next Great Age is going to be… any of a number of developments could come to define it: nanotechnology, genetic engineering, even a return to global religious militarism, God forbid (pun intended, folks).

What I can see is that many of the most engaging and compelling conversations in our society today are decidedly post-Information Age.

Consider this. The Industrial Age was an age of STUFF. The amazing material wealth generated by modern industrial engines was staggering, and as Sears Robuck and Henry Ford brought that material wealth to the people, we came to define ourselves by how much stuff we had. “Keeping up with the Joneses,” we rallied. The accumulation of stuff became the calling card of America. “He who dies with the most toys wins.” We luxuriated in our newfound wealth as a huge middle class was born from workers who began to earn enough to actually buy the things they were making. When it comes down to it, the Industrial Age was all about MORE STUFF. Making it. Selling it. Buying it. Consuming it.

So it is fitting that the end of the Industrial Age began not so much with the invention of computers (in the 1950s) or the Internet (1969)–these were merely the instruments of what came to replace it. Rather, the end of the Industrial Age began with the realization by many people that perhaps we don’t need all that STUFF. It started to end when we started to question, in significant numbers, the defining foundation of the age: do we really need more stuff? The Great Gatsby (1925) by F. Scott Fitzgerald was a post-industrial tirade against the American Dream, but it wasn’t until mainstream America caught up with the concept after World War II that we began to think about and define ourselves en masse as something more than the sum of the bounty of our industrial might.

The late Anita Roddick, who passed away this month, tapped into this with The Body Shop’s line of products that weren’t about more stuff, but rather about a different kind of better stuff, about products that were both good for you and good for the world. In fact, the entire movement towards sustainability has, in its roots, the necessary realization that we must move to a post-industrial world, a world that isn’t defined by our stuff. As cable television and the Internet invaded our homes, we began to find that we could satisfy many of our wants and desires through Information rather than physical goods. It was liberating, intoxicating, and led to one of the most outrageous economic bubbles since the heyday of the Industrial Age triggered the Great Depression.

Similarly, the Information Age is, (surpise!), defined by MORE information. More channels. More telephones. More email. More websites. More advertising. More media.

And in a (perhaps) surprisingly short period, we now find ourselves echoing a new version of the mantra that ended the Industrial Age: “Enough! We don’t need so much Information!”

Consider the evidence:

TMI

Perhaps the most light-hearted evidence is the oft-used phrase “Too much information” or the blithely concise “TMI” when someone divulges more details than the listener really wants to hear… typically about things you don’t even want me to mention in this article.

Google

We also see it in embodied in the most powerful brand of the Age. Google has always stood for making the overwhelming complexity of the Internet, simple. Hence, the clean look, the great results, the fast response. Google knew that you didn’t want everything that’s out there on the Internet–if you wanted that you might try the cacophonous Yahoo! What you want, is just what you want, nothing more, nothing less. Give me what I’m looking for, and leave the rest of that stuff out there. (And kudos, btw, to Google for pulling back recently from productmania to refocus on their core value proposition.)

Blink

How about Blink by Malcom Gladwell? Gladwell outlines just how powerful it can be to think less, evaluate less information, and “thinslice” complex problems, in an instant. Less information: better results. Here’s a quote that is crystal clear about the frustration of the Information Age:

We live in a world saturated with information. We have virtually unlimited amounts of data at our fingertips at all times, and we’re well versed in the arguments about the dangers of not knowing enough and not doing our homework. But what I have sensed is an enormous frustration with the unexpected costs of knowing too much, of being inundated with information. We have come to confuse information with understanding. (p264 in the April 2007 paperback edition)

Gladwell goes on to illustrate through uncanny stories, both anecdotal and scientific, of how we can and do make split-second decisions on the most minute sets of data. Sometimes our bias in those moments can steer us astray, but when managed correctly, those “instant” decisions are not just as good, they can be better than those made in full conscious analysis of all available data. The message is clear. If you can distill your decision making to just the right subset of the data and you prime yourself correctly, you can make better decisions with less information.

The Zero Inbox

Have you heard of the 43 Folders Inbox Zero tutorial and video? From the introduction:

Clearly, the problem of email overload is taking a toll on all our time, productivity, and sanity, mainly because most of us lack a cohesive system for processing our messages and converting them into appropriate actions as quickly as possible.

Too much email! Nicely written and well worth the plunge. You’ll appreciate it. And as a post-email junky, you’ll find yourself liberated from one of the Information Age’s most consuming behemoths.

Burning Man

Have you heard of Burning Man? Arguably the most happening annual ritual qua week-long party on the planet. Not the largest, but certainly an indulgence of the hippest order, where the many of the most adventurous, most “out there”, and most creative head to the Nevada desert to unplug and reinvent themselves in a microcosmic society, disjoint from the “real world.” It is a ritual in unplugging from our information drenched reality to celebrate the physicality of life in the brutal extremes of desert heat, sandstorms, dust, and frigid cold nights, with only ice, coffee, and porta-potties provided as basic civic services (the first two for a fee). The rest you provide for yourself in a form of “Radical Self Reliance” that is a core principle of the event. Why would anyone endure such extremes? In large part because it is incredibly, primally, satisfying to extract yourself from the world of information overload and engage with real people in an real–albeit redefined–world. Whether you like it or not, Burning Man is a undeniably relevant celebration of the post-Information Age human experience.

VRM

In its small way, VRM is also contributing to this trend to reduce information overload. VRM redefines customer relationships with vendors by focusing on what individuals have and need, rather than what vendors have to offer. Rather than trying to index and analyze everything, just capture what’s near the user, and give people tools to leverage what they do know to have smarter, more rewarding engagements with vendors. The result will be a system focused on the individual and his or her relationship to vendors, rather than an aggregated, centralized knowledge base, index, or repository of all the world’s information.

These are but a few examples of how we, as a society, are starting to realize that perhaps we have too much information at hand. That instead, what we need is better, more meaningful information. As more and more of us define ourselves as something more than the information we consume and create, we are accelerating the end of the Information Age and clearing the ground for something new.

Of course, the end of the Information Age isn’t a particularly new or original concept. Just as Fitzgerald pre-dated, yet ultimately helped catalyze, society’s transition from the Industrial Age, so too have others observed and written about the post-Information Age.

Here are a few of note:

From Being Digital (chapter 13) by Nicholas Negroponte, quoted at http://archives.obs-us.com/obs/english/books/nn/ch13c01.htm

“The transition from an industrial age to a post-industrial or information age has been discussed so much and for so long that we may not have noticed that we are passing into a post- information age…

In the post-information age, we often have an audience the size of one. Everything is made to order, and information is extremely personalized. A widely held assumption is that individualization is the extrapolation of narrowcasting–you go from a large to a small to a smaller group, ultimately to the individual. By the time you have my address, my marital status, my age, my income, my car brand, my purchases, my drinking habits, and my taxes, you have me–a demographic unit of one.

And from Andy Orem “What comes after the information age” published just this last week at O’Reilly Radar:

But the Information Age was surprisingly short. In an age of Wikipedia, powerful search engines, and forums loaded with insights from volunteers, information is truly becoming free (economically), and thus worth even less than agriculture or manufacturing. So what has replaced information as the source of value?

The answer is expertise. Because most activities offering a good return on investment require some rule-breaking–some challenge to assumptions, some paradigm shift–everyone looks for experts who can manipulate current practice nimbly and see beyond current practice. We are all seeking guides and mentors.

What’s fun is actually seeing the transition with your own eyes, rather than hearing or reading about it as an intellectual exercise.

Take a look around you, in your own life and work. You might be surprised how often you find yourself craving less information and culling the dead bits from your data (can you say “Spam Filter?” I knew you could.). As you do, savor the end of the Information Age and the beginning of something new…

Phillip Lenssen at Blogoscoped points to this excellent interview of Eric Schmidt, Google CEO, by John Battelle.

Of particular note, at the tail-end of the interview, is the unequivocal assurance by Schmidt that Google will “never trap user data.”

Score one for user-centric silo busting. And another point for Google, when they actually deliver on this promise.  I already have a use for analyzing my Google search history, which currently only Google has access to. Schmidt mentions this obliquely, promising future access.  I look forward to seeing the API and using the service.  If it actually works, it would be a huge step forward in opening up the Google silo for third party applications.

Robert X. Cringely paints an intriguing picture of our future… well, perhaps we should call it Google’s future, since they own it:

Google will become our phone company, our cable company, our stereo system and our digital video recorder. Soon we won’t be able to live without Google, which will have marginalized the ISPs and assumed most of the market capitalization of all the service providers it has undermined — about $1 trillion in all — which places today’s $500 Google share price about eight times too low.

Possible? Definitely. Likely? Hmmm… hard to say. Scary? Yep.

This post from Google Blogoscoped is just too funny to ignore.

Of course, SwitchBook‘s technology really is going to change everything. It is, after all, about “Complex Search,” which is so much more than just “Search.”

Seriously, I think Google is a great target for the passionate entrepreneur. They’ve created a whole new industry and despite their mass assimilation of great talent, can’t possibly source all the great search innovations of the next decade. Game on, for all of us out trying to out-Google Google.

Nancy Gohring at ComputerWorld reports:

Managers waste hours every day looking for information that often turns out to be useless, according to a report from consulting firm Accenture Ltd.In a survey of 1,000 middle managers at large companies in the U.S. and U.K., Accenture found that managers spend as much as two hours a day searching for information — and more than half of the data they find has no value to them.

That’s one hour, every day, wasted by middle managers because their searches are ineffective. What a mess.

Information is often more difficult to find because it’s scattered, respondents said. Fifty-seven percent of those polled said that having to go to numerous sources to collect information makes managing data difficult. On average, the managers said they go to three different sources to find certain types of information….

Managers often face additional challenges because they don’t save important data in a collaborative place. The majority of the managers surveyed said they store their most valuable information on their computers or in individual e-mail accounts, where others can’t access it, Accenture said. Only 16% of managers said they store valuable data in a collaborative workspace, like an intranet portal.

Just under half — 42% — of those surveyed said they accidentally use the wrong information at least once a week.

Of all the managers surveyed, IT workers are the least likely to say the information they find is valuable, and they spend the most time trying to find it. They dedicate nearly 30% of their time trying to find information.

Until companies streamline the way that workers store data, information will continue to be a burden to knowledge workers, Accenture said.

I agree. I’ll add that we also need to streamline the way knowledge workers discover and track information when searching.

The paradigm for search today is built around stimulus-response queries at search engines, but most users engage in search as part of a larger inquiry with an end goal that is more complex and subtle than the intermediate search queries.

Think about online search for competive analysis, market research, or due diligence. Or it could be more consumer-centric, like buying a home, planning a vacation or looking for a job. For each of these searches the user will visit multiple, different search providers, and they will–more or less successfully–keep track of results across many many different websites. Some people keep track in their heads. Others use bookmarks or just keep tabs open or even print web pages out and put them in manilla folders.

There are a few tools out there helping with this: Onfolio, Google Notebook, ScrapBook, and Kaboodle. Of these, none seem to recognize the fact that the user is actively searching.
If you know of any others in this space, please chime in. Or if you’ve tried any of those listed, let me know how you like them. I’d love to hear about what’s out there and what people think.

This is the problem we are working on at SwitchBook, but I’m sure there are more players out there than I’ve discovered so far.

What do you get when you turn proprietary data silos inside out?

Users in control.

Doc Searls has been advocating VRM for a while (here too). What’s nice about his thinking — in addition to the open source/open standards approach we’d expect from a senior editor at the Linux Journal — is that he’s working the problem through the entire technological spectrum:

I don’t think VRM should be confined to a browser, either. I think this is something that should work through a cell phone, a card, or any other device or representation that works for the individual.

Not only are the vendor’s silos being turned inside out, so are the technology and network providers’.

My mindset has been stuck in the browser, perhaps with an accompanying helper application that does nice things for users, but still basically software on a personal computer. At the core, SwitchBook’s innovation is useful in larger contexts, but it won’t start out that way. Our strategy in simple:

1. Make it work with current search habits.
2. Augment IE and Firefox.
3. Expand to other OSes and browsers as quickly as possible.
4. Push the underlying API and data format as an open standard.
5. Open the tool for customization as widely as possible.
6. Open source the code for “built-in” customization

But are we going to take the time now to make sure it works in cell phones or datacards or iPods or anything other than a computer? There just isn’t enough bandwidth for that in a bootstrapping startup.

Fortunately, Doc’s VRM work as a Fellow at the Berkman Center gives him the freedom to invest in a solution of that breadth. A VRM solution that is bigger than any one company, technology, platform, or medium. Say goodbye to the silos.

It has also given me a fresh way to think about Complex Search. Much of VRM — as I understand it — is designed to be automagic. Specify your needs, receive bids from selected/qualified vendors using a tool that makes it easy to manage those relationships. But before one can specify needs, most people need to spend time discovering their needs. For all but the simplest purchases, that’s a Complex Search.

For example, take Doc’s latest VRM “Gesture”

I want a phone that is GSM-based (so it works overseas as well as in the U.S.), works across as much of the U.S. and Canada as possible (Verizon has been a disappointment in this respect), has a GPS, and has an easy-to-use UI. I don’t care about PDA functions, ringtones (I like the old Western Electric bell ring, though), or camera functions. I like keys that are easy to read and use, and an address book that’s easy to synchronize with a computer. It would be nice, for personal reasons (I work for Linux Journal), if it ran on Linux. I’d rather it not (for the same reason) run on Windows. Mostly I just want it to be a good GSM phone with a GPS. And I’m willing to let the GPS function slide, just to get a good phone.

That’s a mouthful. Doc is famous enough in the blogoverse to get feedback without the VRM infrastructure. He may not have a vendor make an offer directly (although a smart vendor would seriously consider sponsoring Doc), but he’ll probably get enough direction from peers to narrow down his vendor choices. With a fully operating VRM, the fulfilment side of that gesture will be streamlined and automated so that any vendor who wants to can cost-effectively make Doc a competitive offer, perhaps even a bundled package that leverages their unique value-add. That will take a lot of work, but the potential value to everyone in the transaction is clear.

Compare that to the broad, politicized, unfocused brush strokes of the AttentionTrust and you can see why I think the AttentionTrust goals are still too blurry and ambiguous to generate much success. VRM is working with Intention. It is highly focused. Its output is clear. The benefit to users and vendors is evident. AttentionTrust is stuck thinking about everything, all the time, and only online, then mashing that into some anonymized goulash from which magic is supposed to emerge. Bah humbug. I’ll believe it when I see it.

I think Doc is on to something, though. The Internet so radically drops the costs of so many different modes of communication, it will continue to restructure our society for another couple of decades, at least. Most of the success to date has been based on one-to-many marketplaces, such as Amazon or many-<aggregated-as-one>-to-many marketplaces such as eBay. VRM lets us create inverted “many-to-one” markets. Markets of one. Make your gesture, create a market. That’s powerful.

And yet, Doc’s gesture — as every request for bids must — also contains a treasure trove in the form of Doc’s requirements, a wealth of needs Doc learned the hard way. He’s a power user with heavy demands and he pushes technology to its limits. He is fed up with his current options and, having experimented enough, he knows just what wants. But he’s lucky to have that experience. Most people have no idea what the deciding factors could or should be for the products they want to buy. (Can you say megapixel?) Doc is anything but a typical consumer.

Consider what it was like when the web started taking off in 1994/5/6. At that time, I was out selling Internet marketing services and helping companies figure out what to do online. Overwhelmingly, time and again, smart, capable, professional people asked “How much does a website cost?” Well, what kind of website do you want? Their question was inherently non-sensical, but people didn’t understand that yet.

First, you have to figure out what you want, then, and only then, can you send out an RFP to get bids on it. Sure, you scale your RFP based on what your budget is — and unless you have deep pockets, it pays to be prudent in what you include in your request — but at the end of the day, only a detailed specification provides enough direction for vendors to submit a bid. The result of these conversations was often a small strategy and/or requirements engineering contract to distill their needs into just such an RFP.

So how does that work with VRM? How do people develop enough expertise and understanding of their needs so they can present a request like Docs? How does VRM work for regular folk?

In short, they search. They explore. They learn.

From friends. By reading reviews. Going to various manufacturer’s and vendor’s websites. By learning from people like Doc, either through blogs, reviews at CNET or ThisNext, pricing at PriceGrabber, Google, or through direct conversations. By trying out products. Even from advertising and retail stores. I happened to learn about Verizon’s data services in the Verizon store. Imagine that.

This is Complex Search. People aren’t going to rely on any one vendor or reference point, unless they have an absolutely trusted guide like a brother or daughter or college roommate to point them in the right direction. They are going to check out different sources, browse multiple websites, collate and corollate a lot of information from a lot of different places. Then, after they have searched and narrowed their needs down to the details, they can put it in the form of a digital RFP and see the power of VRM kick in. Zing! A Market of One.

VRM is still evolving. Questions and answers of many varieties must work their way through the community, from people’s and companies’ needs to draft technological frameworks, APIs, protocols, and working code. Good stuff.

Somewhere in there, I’m confident Complex Search will meet VRM and lots of real value will be created for people, vendors, and innovators alike.

Doc will be at the Identity Workshop in early December to discuss VRM and Identity with all comers. It should be a great opportunity to figure out where VRM is headed and how we can contribute. I hope you can make it.

In a conversation about the potential market lock-in of Google at Abe Burmeister’s blog, Dave Chiu introduced me to a great presentation Seth Godin made to Google early in 2006, explaining that it was marketing, and not technology, that made Google the market leader. I couldn’t agree more, even though most of my technology friends will swear it was all about the quality of PageRank.

Tests show that when Yahoo! & Google results are formatted identically, users can’t tell the difference. And yet, Google matters to people. They matter in a deeply personal way. They have created a powerhouse brand because better technology gave them an opportunity to market to the masses and that marketing worked.

Seth puts this in his framework of remarkable stories. His bestselling books The Purple Cow and All Marketers are Liars discuss this in much greater detail, with lots of anecdotes, examples, and advice on how to improve marketing through remarkable stories. Good stuff.

What he doesn’t talk about is how you create the right stories to tell. He misses that point in his books as well. But that’s ok. Stories are powerful marketing tools. That’s an important enough message by itself. In the presentation he also does a great job pointing out the anticipatory and experiential value of a brand–that value people get just because they buy the brand, independent of the actual value of the product. The driver for this type of value is story, especially when the brand connects with people’s identity in profound ways.

He then goes on to outline his view of Fashion/Permission marketing that is uniquely enabled by the Internet as a one-to-one disintermediated medium. He exhorts Google to create a permission tool that gets users to invite Google into a deeper relationship, one that gives Google more context and more details about what users are really looking for. In other words, leveraging the brand to enhance the technology by meeting users needs in a more meaningful way, which of course will only enhance the brand further. Great stuff. Note to Google: possible areas for development: VRM and Complex Search.

It is worth watching, if only to see the advice one of the hottest minds in marketing gives to the most influention Internet company on the planet.

Curiously, Seth missed the opportunity to explain to Google that their haphazard development strategy is steering them, inexorably, away from the branding that made them the market leader: the promise of making the Internet simple.

Last week, Seth Godin wrote about the critical role trust plays in our market relationships.

And the upside? The upside is that individuals (and organizations) that don’t stoop, that manage to figure out how to have influence without trying to profit from it, those brands are the ones that will last, that will thrive and that will bring the rarest commodity–trust–to the table.

Trust is an absolutely essential ingredient when we buy. We don’t part with hard earned dollars unless we convince ourselves the result will be worth it. We count on the salesman or the manufacturer to deliver real value. We trust their promises and fork over the cash.

Sure, Caveat Emptor, buyer beware. But companies that violate our trust once lose our business forever. In today’s markets there is so much choice, so many options, that once trust is gone, people are free to move on to another offer, and they do.

This is nothing new. Trust has always been critical. The information age changes the pace and volume of information, but not the criticality of trust. It seems to me that hundreds of years ago a single piece of information–”The King is dead.”–could change the course of a battle and turn the tide of a war. Today, a single piece of information is rarely so valuable. Instead, we assimilate, analyze, and respond to vast torrents of information. Diving into the Internet is like swimming in an ocean; daily RSS feeds from the Blogoverse are like drinking from a fire hose. It isn’t about any particular drop of water, it is about whether or not you can consume and respond quickly and intelligently enough to keep yourself from drowning.

What remains the same today and yesterday is the critical nature of how much trust you place in your sources of information. If news about the departed monarch of yesteryear arrived from the mouth of the town drunk, it wasn’t believed. If today’s assertions of male enhancement and unimagined wealth are delivered in attention assaulting SPAM, they too deserve little notice. In both worlds, trust moderates the flow of information. Without a source or a guide we can trust, no information can be depended on.

Sometimes that trusted guide is someone you know. Sometimes it is a brand that you’ve come to rely upon. Or perhaps it is someone referred to you by someone else.

Yet, when it matters, we rarely trust just one source, no matter how trustworthy. Our doctors tell us to get a second opinion. Buyers seek multiple bids. Journalists are trained to double check their facts. It is a matter of due course to seek out alternative perspectives to transform a one dimensional recommendation into a three dimensional panorama with scope and breadth of multiple experiences and views. Even at Amazon, we rarely read just one review; we want a wealth of perspectives so we can paint a richer picture of the product in question.

With the Internet, we regularly review and incorporate multiple sources in our decision making. We double check the prices at Expedia and Travelocity at Orbitz or United.com. We check online reviews and competitive offers from multiple sources for just about any item we can buy, from cars to computers to movies. We aren’t limited to any particular silo of information. We mix and match until we have convinced ourselves that we understand well enough to make a decision. This is the oceanic torrent of diverse information we sift through to discover truth.

This is complex search. We browse the web, using multiple search providers and track results across many many destination websites. It isn’t about the quality of any single search result, it is about managing the aggregate results from multiple services across the web.

When we search for a house, we don’t take the MLS listing at face value, we also check Zillow.com and want to learn about schools, crime, traffic, smog, and public transportation in the neighborhood. When looking for a job, we don’t just trust Monster.com or CareerBuilder.com, we also visit the company’s web page, scan Google and Technorati for buzz, and may even check out eTrade or Hoover’s for more in-depth financial data. We use the Internet to compile an aggregate view of our search target, a view that combines the perspectives of many different sources of varying levels of trust. The end result is a composite that we can trust because it represents a coherent representation of a broad range of diverse data, selected using our own judgment. We trust it more because it isn’t sole source, because we did the legwork, because we own it emotionally.

And yet, of the search solutions available today, none make it easy to manage this type of inherently complex process of double-checking, comparing, and seeking second, third and fourth opinions. What you type in at Expedia has to be re-typed at CheapTickets and United and Travelocity, and the results are often hard to retain for later comparison. What you find on eBay is isolated from the results at Amazon and shipping, handling, & taxes may be hidden until you proceed to checkout; even shopping sites like NexTag, TheFind, and ThisNext don’t make it easy to mix & match and compare multiple products fully priced from multiple merchants. Even Google and Yahoo! and MSN leave vast swaths of the Internet outside of their search database, commonly known as “The Dark Web.” Today’s search is a vast array of isolated islands, each offering a glimpse into their private database, their own private silo. But users don’t live in silos, they skip across the net from island to island, saving to bookmarks, printing to PDFs, opening links in new tabs and windows, even cutting & pasting into Word, trying to keep track of their journey. It’s a mess.

What we need is a tool or a system that lets us coordinate complex searches across the ‘net, using any search provider we want–indeed using any query-driven web service–all seamlessly integrated into a single interface and repository stored in the user context. We need the user to be in charge of their search, no matter where it might lead, how it might evolve, and what information is uncovered. We don’t need to be bound to any particular vendor, or isolated in any particular silo. We do need to be able to expand the search to include new search providers and to capture unforeseen data. And its needs to be both easy and powerful. And it all needs to work without interrupting, distracting, or confusing users.

That’s what we are working on at SwitchBook. Over the course of the next few months, I’ll be exploring what Complex Search means, highlighting how current and emerging tools succeed and fail to meet this need. Hopefully you’ll find it as compelling a problem as I do and will take a moment to share your thoughts on how we can shift the search world from the simple query-response of the Google era to an interconnected system of search technologies that empower users to resolve truly complex searches simply, quickly, and effectively. When we can do that, we can create a system that is accessible enough, flexible enough, and transparent enough for every user to trust.

While I’ve been off traveling, Google and Yahoo have been launching new or revving existing products that verge somewhere between Search VRM and Complex Search.

Matt Cutts (at his own blog) and Greg Sterling and Chris Sherman at Search Engine Watch, all have great writeups of the details of the powerful new Custom Search Engine service from Google. In short: Google is making it easier for websites to offer customized searches based on a focused subset of Google. The websites provide hand-crafted context through an explicit interface in the CSE design and Google kicks back a bit of the AdSense revenue to the website.

It’s a great package.

1. You get to integrate the market-leading search engine into your site
2. You get to direct Google as to how they can improve the search based on the context of your website
3. You get to monetize it

Brilliant. Google’s Custom Search Engines extends their service into affiliate websites in a way that empowers the individual webmaster’s expertise to improve the search. And they’ve made it so easy that literally anyone can do it.

Chris Sherman also has a nice write up of the improvements to the Yahoo toolbar. The main change: improvements to managing bookmarks, including integration with the search as well as making it easy to add a bookmark button to the toolbar.

Yahoo has also made it easier to organize your bookmarks with this new version. […] You can also add tags to your bookmarks, again either when you initially save one or later. If you check the “use tags” box as you’re saving a bookmark, Yahoo will suggest tags to associate with a page[…]

Alternately, you can save pages without worrying about categorization or tagging. By default, bookmarks not saved into a folder are placed in an “uncategorized” folder. To organize bookmarks, you can simply drag the thumbnail image of a page into a folder, or mark selected pages and move them all with a single command. There are also folders that display recently saved bookmarks and frequently used bookmarks.

You can edit bookmarks, adding additional information such as tags, a description, or comment. […] Yahoo has also integrated bookmarks into the search functionality of the toolbar. This means you needn’t take the trouble of organizing bookmarks into folders—you can simply rely on the search box. This works well, since Yahoo is storing the full-text of web pages when you bookmark them.

In short, Yahoo is making it much easier to keep track of interesting websites by saving entire pages, letting you tag them, and then reintegrating those pages into your subsequent searches. Google Desktop has been indexing and searching web history for a while, but Yahoo! is essentially bringing the Del.icio.us model onto your home computer.

Unlike Yahoo’s Del.icio.us and MyWeb, bookmarks saved using the new bookmarking feature are not intended to be made public and shared. While you can share bookmarks with others either by email or instant messenger with other individuals, they aren’t public in the way that a Del.icio.us bookmark is, for example.

What’s exciting to me about both these developments is the increase in control for the user. Google gives website owners incredible power to customize searches for their user base while Yahoo is making it much easier for users to manage complex pan-Internet search (and non-search) activity. This means more power for users, more effective searches, and quite likely more revenue for everyone.

Watch for this trend to continue.

Danny Sullivan over at Search Engine Watch has a great write-up about IE7’s use of search.

I particularly like this feature:

[The] Search Provider page also has an interesting box allowing you to visit any search engine, then do a copy-and-paste action to make your own search box. It’s very clever. You simply search for TEST on anything that gives you a search box. Copy-and-paste the resulting URL, and IE7 will automatically create the right way to access that search engine for you. I added Search Engine Watch as a search engine to my IE7 installation easily by doing this.

Smart. Kudos to the IE team for putting the user in greater control of their search interfaces. This basically means that the search provider doesn’t need to cut a deal with the PC manufacturer to get in the search provider list. And they don’t have to do anything special to their website, other than using a consistent URL-based query.

In some ways, this is a bit like Rollyo, although Rollyo is purely a Yahoo!-based aggregation service. Both make it super-easy for the user to control where they want to search, even when the search provider (which could be any database-driven website in IE) or searched website (with Rollyo) may not have the interface in place to support it.

This is a fairly elegant and lightweight form of VRM (Vendor Relationship Management), where the vendors are search providers (in the broadest possible sense). Nice that Microsoft is giving users increased control over their search relationships.

Great stuff.