Portals and KM

Specialized Search is a Next Big Move for Search

I have been discussing specialized search a bit (for example, see: Will Yahoo's Changing Search Strategy Cause it to Pick Up Niche Services?). Well, Yahoo is not the only mover in this direction. As reported by Venture beat, Google is planning to launch a music search service with music sites Lala and MySpace-owned iLike, according to multiple reports. Of course, Google offers other specialized search services such as blogs. There are also many specialized search tools entering the market such as Attivio, Exlead, and MindServer

The article states that with this new service, you will be able to search for a musician, then play songs streamed from partner sites like Lala and iLike. CNET describes the goal as providing ╥everything a music fan may need when searching online for a favorite artist, while TechCrunch says the pages will be modeled on Google results for public company searches.

It goes on to add that this sounds like a smart move for both Google and the music companies, giving Google a cool new service without requiring the it to host the music or strike deals with the record labels on its own, while bringing more exposure to Lala and iLike.

I certainly agree and feel that specialized search services are one of the next major directions with search. The Google page rank approach has been around for some time. It keeps getting refined but you can take it only so far.  It was a huge improvement over what was available. But you still get many false positives and lots of stuff to shift through. Specialized search can get you there faster. 

Darwin, a firm I am part of, offers both an alternative to the Page Rank approach but it also allows you to set your own level of specialization. You can search on a general basis like Google or set a very focused attractor and specify the area within the Web that you want to focus. It can be formal or informal.  You find out more at our Darwin Discovery Engine blog. 

Our Upcoming Session at Boston KM Forum on Chaos Theory and Information Discovery

Thierry Hubert and I will speak at the December 17 meeting of Boston’s KM Forum.  We will discuss: Deliver Us from Web 2.0 Content Chaos: Activity, Attraction and Personal Knowledge Create the Awareness Engine™.  Here is the session description.

The presentation will share with the audience a practical application of chaos theory for creating knowledge focal points from the chaotic mess of content resources distributed across the Web. Using a couple of examples, one from the Internet where socially generated content is exploding, and one from the enterprise where social tools are also contributing their own content bloat, We will discuss the ideas that are driving a new product idea at Darwin Ecosystem. The examples will illustrate how activity can make information more discoverable, and easier to explore through various visualization techniques. These techniques open the door for more self-discovery through intuitive recognition of patterns. They will help the explorer to find relationships among content objects and people that would not have been found using traditional search methods.

We hope to see you there. It will be from 4 to 6 PM at the Microsoft offices, 201 Jones Road, Waltham, MA. Here is the announcement post on the KM Forum blog with directions. Here is more about Darwin Ecosystem and how it works. 

My KM World and Enterprise Search Summit Notes: Is Semantic Technology Real?

This is another in a series of notes from the 2009 Enterprise Search Summit and 2009 KM World. It is titled: Is Semantic Technology Real? It is moderated by Rob Gonzalez, Platform Product Manager, Endeca Technologies and include: Michael J. Cataldo, CEO, Cambridge Semantics, Daniela Barbosa, Business Development Manager, Dow Jones Client Solutions, Dow Jones & Company, Lorenzo Thione, Founder / Principal Program Manager, Powerset / Bing Microsoft, Inc.  Here is the session description.

“Semantic technology is all the rage, sometimes even dubbed “Web 3.0.” However, many people—especially those making technology decisions for enterprises—wonder whether semantic technology has meaningful applications in the enterprise. Based on hands-on experience working with semantic tools, this panel of experts will establish the boundaries between reality and hype and help you understand what enterprises can gain from semantic technology in the here and now.”

Cambridge Semantics provides semantic development tools.  Dow Jones Client Solutions helps clients with taxonomy solutions, as well as other information needs. Powerset was acquired by Microsoft. It is Web semantic search tool.  Rob said, in answer to the panel title question, that semantic search has been applied successfully inside the enterprise and on the Web.

Rob asked the panel if there any examples of financially successful Web applications of semantic technologies. Lorenzo said yes. This happens more when you use semantic technologyto improve an existing site or product on such applications as consumer search. It seems harder for new sites such as Wolfram Alpha.

Daniela said in publishing there have been successful examples such as the Huffington Post and some Web advertising that use semantic web but do not promote it. The sites that promote that they are semantic Web are less successful.  Best Buy is using semantic web with their whole catalog but it is not promoted as a semantic web site. Google bought a semantic search company and it formed the foundation for Adsense and Adwords.

Semantic web appls let you combine different data sets like comparing the odds that you will be killed by sharks vs. vending machines.  It turns out that Vending machines are more deadly.  Semantic search helps people better define what they want in search.  The definition of semantic technology has been fluid. Rob said it is everything from natural language document processing, and sentiment analysis to geotagging so it is metadata on top of data.  It is finding new meaning in data. Michael said semantic technologies are based on emerging standards on ontologies, distributed query, and other related things. Semantics helps you find concepts even when there are not exact word matches. For example, you can find out how many sub-prime mortgages are in your loan portfolio even if they not are labeled that way. You define the characteristics and then find them. Here is a paper on semantic technologies and ontologies. 

The panel moved to semantics in the enterprise. There have been a number of successful examples where you can use a computer to connect data in two different places. How do you know that different things are similar? This is the answer that semantic technologies can address. It is about understanding concepts. FAST was given as an example. Consulting companies are starting practices around implementing semantic technologies.  At Dow Jones they are seeing companies interested in making better use of existing data through semantic technologies..

Daniela Is hoping that semantic technologies will get implemented like wikis, start small and spread.  This will be better than starting big.  Michael said that ontologies can grow over time. 

Some one asked what is new about semantic technologies? Daniela said the open vs. closed concept. You can find better connections now. Companies do not know they are already part way there so some stuff is not new.  Now you can get your siloed data connecting and get better access to data outside the enterprise. Semantic technologies are trying to find structure in data where it does not explicitly exist in the same way a person might.

Merck is using semantic technologies to automate the integration of research results from three different relational databases and several spread sheets so they can be uploaded into a common database. Prior to this people had to cut and paste content by hand into a template for the upload.

Semantic technologies support discovery in contrast to simple search when you know what you are looking for. There remains a lot to do to improve the interfaces of semantic technology tools. 

My KM World and Enterprise Search Summit Notes: From Birth to Billions: The Life Story of Google Enterprise Search

This is another in a series of notes from the 2009 Enterprise Search Summit and the 2009 KM World. This session is titled, “From Birth to Billions: The Life Story of Google Enterprise Search” by Cyrus Mistry, Enterprise Product Manager – Google. Here is the session description.

"This talk examines the state of Google’s Enterprise Search business, highlighting the origins of the iconic yellow box and the way the product and business search market has evolved over the past decade. Mistry highlights the challenges facing enterprises and providers of business search technology, as well as possible solutions and case studies of companies and organizations leading the way in search innovation.”

Cyrus began by describing the common problem of having too much information at work. The average worker creates over a gigabyte of information at work and much of this is in email. So Google were asked if they could bring Google into the enterprise.  At first they resisted but there was too much demand so they started Google enterprise search.

They found people wanted three things. First, people want simplicity.  The average number of words put into a Google search field is shrinking but people have the same expectations for good results.  They also want speed of response.  They did studies and found that the faster the results the more relevant people felt the results were. The third thing people want is relevance.

But things are different in the enterprise, Page rank is not relevant as there is not the same linking as on the Web.  In 2001 they launched Google for the enterprise in a small yellow pizza size box at a small price point that opened it up to more businesses. Cyrus said it also help lower the price of many other search tools. In 2004 they added more security. 

People felt Google was not connected so they did Open Source and established the connector framework.  Next after the basics were done they looked at the search experience.  They wanted to bring all types of results (text, images, etc.)  into a single screen.  This provided intelligent simplicity.  Next they looked at scalability so you can reach all documents. With Google you can add more boxes to get scale even up to a billion documents.

Now they are looking at social signals. Cyrus said this is where the innovation will be found.  The feature of user added results has been the most popular capability. People get really excited about this.  Another popular social feature is query suggestions to see related things that others are searching for.  They found a big jump is usage after this feature was adopted. They also look at where people click to see if a result should be elevated.  They use a complex algorithm to overcome any bias of current rank of this elevation.

 

My KM World and Enterprise Search Summit Notes: Enterprise Search Technologies

 This is another in a series of notes from the 2009 Enterprise Search Summit and the 2009 KM World. Enterprise Search Technologies was a preconference workshop. These notes are done near real time so please excuse any typos or spacing issues.  It was led by independent consultant Miles Kehoe from New Idea Engineering, Inc. Here is the session description.

“This workshop, by a vendor neutral consultant who has hands-on experience with a broad range of “out of the box,” open source, commercial, and home grown solutions, provides an overview of the enterprise search technology landscape. It reviews technologies currently on the market; discusses pros and cons, strengths and weaknesses, and specific requirements Kehoe shares case studies that illuminate how search technologies are leveraged in different types of organizations; and provides a good introduction to and understanding of the enterprise search world.”

Miles said the characteristics of great search include conversational capabilities, open ended, flexible, and smart.  Conversational search allows you to interact to focus your quest. This is especially important for enterprise search, as search is much harder inside the enterprise.  Never provide just - no hits. Ask more if you cannot find anything.  Every search engine is built around a set of indices. This even applies to Google who creates an index through its spiders. Different search engines just add different stuff around the indices.  Every search engine goes through a process. Some expose parts of it which gives you added flexibility to pull specific information out.

It used to be you got plain search results pages in enterprise search like basic Google Web search. Now you get what he called enterprise search 2.0 with visualization, navigation, people, facets, etc. strung around the basic results. 

There are two basic parts of search: indexing and the actual search. It is better to take time at indexing (when people are not waiting) than search when people are waiting. However, I asked if the real time capabilities of Twitter were changing those expectations.  People want to see stuff as soon as it exists.

Before he reviewed vendors, Miles said it is not the technology but the methodology. It is how you implement the search engine. I can agree with this.

As he started to review vendors Miles mentioned Lucene/Soir, a free open source search engine that is behind a number of search engines, including some commercial ones. It is Java based with an Apache license, prolific documentation, many implementations, and you have total control over search and relevance. However, there is some implementation work required and it is hard to find answers. There are limited enterprise support options. SearchBlox is packaged Lucene. Lucid Imagination is packaged Soir.

Miles’ tier one vendors are: Autonomy, Endeca, Exalead, Fast Search (the original independent version), Google, Vivisimo. I have reviewed Exalead (see Exalead’s CloudView Offers Integrated Search Capabilities and Exalead Provides Ability to Integrate with EMC Documentum).  His criteria are: broad enterprise presence, multi-platform search, market penetration, and clear product vision.  People like the Google brand so they have a perception that Google enterprise search works well. Not being in Tier One is not necessarily bad, just not meeting all the criteria. Other vendors I have reviewed that Miles also mentioned include Attivio (see Attivio Aligns with Traction and Releases New Features) that is newer and Recommind (see Recommind Provides Axcelerate eDiscovery 3.0 with New Features) which is more vertical focused.

Dates are important but web servers provide bad data so it is hard to trust what you get. Miles gave the example of a 1996 document appearing as new because it had just been re-indexed. 

The wifi started working so Miles showed us Web sites with good search capabilities. Globrix is a UK real estate site that uses FAST and you could see a lot of facets in home listings such as number of bedroom, bathrooms, price range, etc. Then we looked at Newssift that displays sentiment on topics. We looked at Kosmix that provides an example of exploratory search.  It shows things that are related and loosely related. 

Next we covered supporting technologies including document filters, connectors, social search, and federation.  Document filters are part of the indexing process that converts binary source documents (PDF, Office, etc.) into a stream of text for indexing.  Connectors are utility tools to provide a clearly defined interface between a search engine and external content. Some relate to indexing and others to display. Connectbeam is an example (see my reviews: Connectbeam Offers New Social Networking Application Integration Possibilities).

Social search is a popular term that applies to the capability to search corporate personal profiles to find people in an organization with certain skills or experience. It typically requires user to explicitly self-profile in order for searches to return accurate results. Some products now track user behavior to implicitly associate interest to users.

Federation refers to a program that can dispatch user queries to one or more external data sources (search engines, RDBMS systems, etc.) and present the combined results to the user. Federation from unsecure resources is fairly easy. Because relevance from each source is calculated differently, it is sometimes difficult to integrate results in a meaningful way.

Entity extraction recognizes people, places, or things during indexing. In unsupervised extraction entities are recognized through algorithms. In supervised extraction, the process is seeded by human operators prior to processing.

Sentiment analysis recognizes positive or negative sentiment algorithmically during indexing. It is easier to tell positive sentiment than negative.

Results clustering groups sets of documents into categories base don content. It looks like facets and entity extraction however clustering can be done independent of the query. Clustering is often used in search results to assist the user to discover additional related terms and content.

Facted search is the result of assigning documents in a search result list into a pre-defined taxonomy-like order. Unlike clustering, which can appear similar, facets are base don the query and populate pre-defined classes of content (authors location, etc.). Facets are often used to encourage interaction with user.

A key to having good search is to monitor it over time after the initial implementation. Look at what is happening and make corrections. Look at what people are searching for and accommodation them. You need to pull together a diverse collection of skills to have a great search function (e.g., business domain experts and corporate librarians, beyond just technical skills).

Miles mentioned two blogs on the topic that he writes: EnterpriseSearchBlog,com and SearchComponentsOnline.com. 

   

My KM World and Enterprise Search Summit Notes: Fundamentals of Enterprise Search

This is the first in a series of notes from the 2009 Enterprise Search Summit and the 2009 KM World. Please forgive typos as this is done in near real time. Fundamentals of Enterprise Search was a preconference workshop. It was led by Avi Rappoport, Principal - Search Tools Consulting Editor, SearchTools.com. She is an independent consultant not connected with a vendor. Here is the session description.

“Search engines, big and small, have certain standard elements and processes. The more you understand them, the easier to tune them to solve your real information needs. This practical overview provides a big picture view of how search fits within enterprise and websites, and a focused introduction to search technology and user experience. Elements of search covered include robot spiders, database connectors and other tools for locating content, indexing issues, query parsing, retrieval, relevance ranking, and designing usable search interfaces. The workshop addresses common search problems and solutions, security issues, languages, new interface elements, important (and unimportant) features as well as providing tools for choosing a search engine or evaluating an existing one.”

Avi began with similarities and differences of enterprise search and Web search. The differences include: limited scope, fewer meaningful hyperlinks for link analysis. Security and access control issues, content in databases, more control (for specifying value ranking, etc.), and no search spam.

Next she covered text search vs. data base search. Text search indexes multiple content sources and uses simple search commands instead of SQL. There is flexible indexing and retrieval and relevance ranking (major issue). There are new features such as spell check, auto completes, and facets. It works in the real world (e.g., eBay, Google).

Then she covered how information architecture works with search.  Information architecture is the art and science of organizing information for access and use. It creates order and systems and provides standard vocabulary. Search can supplement information architecture through user vocabularies and dynamic changes with new content.   

KM and search are opposite ways of approaching content. KM organizes stuff, and search finds it.   There are two main types of search: known item with short queries and “good enough” answers and exploratory search for research purposes. This is what Darwin Ecosystems excels at.  It can help you discover relationships between content that you did know exits and were not explicitly looking for. Avi said that search is an iceberg and people often see it as magic.

It is useful to index everything as it is hard to know in advance what people want.  Twitter has changed expectations for real time indexing, even for intranets. Three minutes is a good expectation. Here is another impact of consumer Web on enterprise computing.

Index security is an issue. (see my post: Attivio Aligns with Traction and Releases New Features) Without the right security you can see stuff you should not see. Need to work with security people on search issues to avoid this and have capabilities in the search tool.  The first step requires knowing what needs to be controlled and then you can determine how to do this.  Be aware of privacy laws.

After you determine what to secure, then deal with access control. Best to keep access control info as part of document store (what Attivio does - see post mentioned above). There are four levels of access control. One – access to search engine, two – collection-level access control (to portions of the search engine), three – locked results for a teaser for subscription, four – hit-level access control – link to a access control database at the point of display - hardest to do, useful when constantly changing rules.

Robot spiders start with a base URL for all hosts. For each page they repeat this process: read text info into internet format, save document in cache, save words into index, extract all links and check for rules, if they are new URLs add them to the list. It can repeat process over and over.

There are common problems with robots that SEO tries to avoid. Spiders can be disallowed by robots.txt or robots meta. Also cannot handle URLs with ? and & (but all spiders should handle these now), Javascript, forms, and interactive dynamic links, session IDs that change, multiple views of same data (wikis and Lotus Notes). 

External sources that have APIs like Twitter can be brought into enterprise search. You might want to partition this so it does not clutter standard search.  This is one way you can use Darwin Ecosysteminside the enterprise to look outside it. Relevance is relative. (sounds obvious but need to remember this when creating relevance listing).

Indexing multimedia needs to be dealt with now. There can be internal and external metadata to support this.  Best to use human judgment rather than automated systems. Automated systems can be a starting point but they need to be fine tuned by people. Speech to text and other automated capabilities are still buggy.

Stop words are common terms or ubiquitous terms. Traditionally you excluded them but there are consequences – such as copyright mentions. Best to index everything, especially since storage is much cheaper now. Avi gave a good example of excluding stop words by searching for phrase “whatever well be,” a song title.  On the other hand you can lots of irrelevant stuff.  Another example, the rock band, The Who, Here is where relevance can help so you get a lot if you include stop words but only need to see best ten examples. I tired this on Google and it did work as the top results related to the band even though there were 457,000,000. Avi said that Google may have set up an exception for this term. She also said you might get different results on wordpress.com.

Dealing with duplicate documents can be complex. First you need to decide what is a duplicate and then what is the primary if there are some slight differences (e.g. typo corrections).  Exact match is easy but similarity is more useful, harder but worth it.  Best to remove duplicates from index and hide results unless requested. This is what Google does.  Can create rules for handling duplicates. This is a good idea. However, you need human supervision but it is worth it.

Avi went through the search process: search form – query parser – query engine (goes to inverted index and back)  – relevance ranker (goes to document store, get stuff and brings back – formatter – search results.  This all happens very fast now. Queries come from many sources, not simply search fields. There are alerts, saved searches, automated searches, geographic information systems, and others. You need to balance relevance and completeness. You cannot have both.

Relevance ranking algorithms work differently. The most common is TF-DF or term frequencies : inverse document frequency. How often is the query word in document and how often is word in the index? There are others but this is most efficient. Look at title, metadata, and top of document. Remember relevance is task specific. There is no such thing as objective relevance. You can never please everyone. More like berry picking than hunting, Try different stuff instead of locking on single goal. Here is where Darwin Ecosystem can help with correlations of different topics with target key words.

Be sure to limit the user interface complexity. Google is a great example. Use familiar use interface elements. Put search into navigation so it appears everywhere. With auto-complete, use a drop down menu of matching words. Base this on search logs and use 7 - 10 most popular in alphabetic order

In summary, with enterprise search you have much more control on the capabilities and decisions with your search capability than on the Web. Make good use of these decisions. 

Darwin Ecosystem’s Awareness Engine™ Provides Enhanced eDiscovery within the Enterprise 2.0 and on the Web

The introduction of Web 2.0 social media into the enterprise creates large amounts of potentially useful information about the social side of business processes. Enterprise 2.0 also provides a wealth of social “data” as a byproduct of its use. We are only beginning to figure out how to harvest this content. The organizations that do this are truly working in the enterprise 2.0 space.

In this post I want to bring to your attention to a new tool that I have been involved with for the past two years. We have been operating somewhat quietly and are now making our work public with both an enterprise edition and a Web edition. The Alpha version of the Web addition is available on our Darwin Ecosystem site through free registration for your use. The enterprise version is also available to early adopters.

Increasing awareness in context and finding related content is the goal of Darwin Ecosystem. Its awareness engine™ gives users the ability to perceive and be conscious of events and patterns of activities captured in the enterprise and Web 2.0. The introduction of Web 2.0 tools into the enterprise also opens up possibilities for even more information silos if these tools do not connect. Darwin provides a way to see emerging patterns of related content across information generated through a variety of tools.

Darwin reduces the effort of keeping-up with enterprise 2.0 content (Information Overload Management). It classifies and correlates patterns of the business activities trapped in your Web. Through its Scan Cloud™ Darwin makes visible and measurable the value of the enterprise 2.0 content (Awareness and Monitoring). You can see related items in a tag cloud like visualization and this relationship shifts as you move through the cloud. This dynamic nature makes it easy for users to see the correlation across the enterprise 2.0 content (Discovering and Sharing).

Using correlation metrics based on Chaos Theory Darwin looks for the emergence of correlated themes within chaotic content. This moves Darwin away from the voting and popularity rankings used by many information aggregators and search engines to rank information. By using correlation metrics, it does not require a known process or taxonomy to discover useful and related information. Darwin allows for the emergence that Paula discussed. You can set up ongoing filters or “attractors” to explore emerging themes in specific topics of interest or simply look at the broad picture.

Darwin is a not a replacement of existing enterprise 2.0 technologies, dashboards, and document management tools. It is designed to complement and leverage these technologies by making their tacit knowledge more visible. Darwin looks at events, blogs and other Web 2.0 sources that may correlate with the enterprise actions, its competitors and the voice of its critics and customers. This allows the enterprise to be capable of more easily discovering its own knowledge assets as well as its market and competitive positioning.

For example, Using Darwin’s Awareness Engine, a program manager in Marketing becomes aware of concerns in R&D, Sales or other initiatives that are emerging as he/she tries to promote a product (all without having to wait and depend the next meeting or coffee break encounter). They can also see competitive moves that may correlate with their own plans or the efforts of their customers and prospects. Likewise management can have a high-level view of what topics are emerging across all divisions and initiatives to better steer the business or measure the effectiveness of the vision’s execution, in the process discovering emerging and stronger initiatives or employees that are noteworthy.

It is a Web browser application (Scan Cloud™) or it can become a custom solution through API access. It is delivered through a Web server with services and a database correlating the different Web 2.0 sources. For the enterprise there is an on-premise solution running on Ruby on Rails and making use of RSS feeds. Its Virtual Cortex™ database can be set on Oracle, MS-SQL or mySQL according to scalability needs. You can go from inside out so an enterprise can correlate their initiatives, knowledge and business intelligence against the pulse of the Web. The application is currently in Alpha stage and available for use.

Here is an example of Darwin in action. Using the Web edition. I set up a query on social media and saw the word – gain - in the cluster shown below. I looked to see why the word - gain - is associated with social media by selecting it and highlighting the cluster of correlations between social media and gain.

Then I saw three articles that matched this correction in the past few days as you see in the image below.

There was as story about how NASDAQ launches social network site as shown below. This was new news and very interesting to me.

I also saw an article that reported in the UK that social networking sites accounts for 25% of display ads. I had read elsewhere that in the UK online revenue has passed television ad revenue so this added further context to that story. There was also an article on using Facebook traffic to drive brand loyalty. None of these articles appeared on a Google search on social media and gain. Nor did they appear on Google News on the topic. Below is a screen shot of the complete Darwin interface so you can the relationship of the detailed components shown above. This exercise is a very brief sample of how you can discover new information that is often unanticipated through correlation that is hard, if not impossible, to find in traditional search.

You can also find stories about themes that are emerging in the Web, as well as images that correlate with content as shown in results about a very recent helicopter crash in the Pacific ocean.

Darwin also allows you to correlate content in traditional mainstream media with what is happening in the blogs and other social media. In 2008 Darwin won a Young Entrepreneurs Award from the Office for Science and Technology of the French Embassy in the United States. The Young Entrepreneurs Initiative (YEi) is a platform for mentoring and networking US-based entrepreneurs who understand the importance of internationalizing their vision and their activity, and who wish to set up a technology venture in France.

We recently started the Darwin Discovery Engine Blog to provide more background on their efforts, as well as commentary on content issues on the Web and enterprise 2.0. Please join us there. Our Twitter feed is: @darwineco. We look forward to hearing from you.

Yahoo Moving Focus from Basic Search to Presentation and Services

CDNet reported that Yahoo is pondering the meaning of search. It is in the middle of a company transition, with a pending deal to sell its search assets to Microsoft in exchange for a healthy portion of the revenue generated by searches on Yahoo's pages. To remain competitive in search it is looking at how it can bring added value for users.

Yoelle Maarek, a search expert who recently joined Yahoo Labs from Google, Israel, is quoted as saying that the advances that have taken place since the debut of Google's PageRank are relatively minor compared with the potential to improve how results are presented to users. She goes on to add that search will become more about divining the user's intent and improving the delivery of search results over the next decade.

Yahoo’s CEO Carol Bartz has said that the overwhelming majority of searches done on Yahoo are done by people who are already there, rather than those who head directly to Yahoo.com or set Yahoo as the search window in their browsers. Yahoo has big stakes here as it earned 359 million from search ads on its owned and operated sites during the second quarter of the year. So it makes sense to provide search related services so they continue to use yahoo as a destination. 

Will Yahoo's Changing Search Strategy Cause it to Pick Up Niche Services?

With the alliance with Microsoft's Bing, Yahoo's focus is no longer on the technology that powers search and trying to beat the competition with it - the "megawatt war," as Sam Diez reports at ZDNet. Instead, it wants to ride the back of Microsoft technology and channel it so users will want to come back to Yahoo every time they have a query. They want value added user experiences.

To accomplish this they need to offer value added niche content. One example, would the type of focused search experience that Tvissimo can offer, for example. Tvissimo lets you not only find what's happening on television, but you can also set up alerts for when your favorite shows, actors, movies, etc. come within a two week window so you will not miss them. That is a service that will keep you wanting to come back.

Sam concludes by wondering if "given the time and resources to focus on customer experience instead of back-end technology (Yahoo) gains some ground and starts building itself back up." Services such as TVissimo would help them. It can become a big factor in accessing television content, especially as it becomes available across computers and mobile devices.

Tom Wilde recent wrote in Search As Television’s Savior? that one critical need for the new multi-screen reality (when people use computers and TV at the same time) is “a new metaphor for programming, and this is where search can play a vital role.” He points out that users are increasingly in control of the content they consume so search is the perfect approach to delivering individualized content discovery. To be successful publishers need to be aware of how their content is discovered and organized across the multi-screen reality.

He goes on to outline some useful search optimization techniques. Having a robust search capability such as provided within TVissimo is another way to bring more control to users. Having both is the ideal way to go. Nielsen Research came out with interesting statistics on overall video consumption last week indicating that total TV consumption reached record levels when considering traditional broadcast video, time shifted video, web video and mobile video. People want to the content ad they do not want to miss what is important to them. The web gives them an increased desire for control. Search can put more of this control in their hands.

Expanding Web Search Part Two: Real Time Search

Yesterday I wrote about Focused Search-Based Applications as one way that Web and enterprise search is expanding. Here is another. The Wall Street Journal recently posted an interesting article on this topic, Internet Giants Look For Edge In Real-Time Search. It started by explaining that, “that real-time search helps Internet users find Web posts, including those from San Francisco's Twitter Inc., seconds after publication.” Twitter is one real-time search engine and others are joining the market.

The article goes on to add that making sense of real-time data is difficult for the big search players and their current algorithms. Their results are heavily weighed towards older Web pages that have established credibility and attracted large audiences, an approach at odds with real-time search.

Twitter users are constantly pouring out a flood of tweets, many are seconds old but from obscure users with little track record. How do you judge the quality and reputation that Google uses as part of its ranking? Also, tweets are written to take up as few characters as possible so spelling is often changed and acronyms, sometimes unique, are often used.

I see two different uses here. Google for time tested material and Twitter for what is happening right now. But how do you filter out the bad content in twitter like Google does? For now, the best way to determine the quality is to know the tweeter. On Twitter itself, Dion Hitchcliffe tweeted to me and @KMHobbie, “Twitter seems to be a solution at last for "filter failure." Rather have info endorsed by people I know.” I agree. I also look at the content, itself, and evaluate it on its own merits as most useful tweets point to expanded content in a blog or article. Meanwhile, there is a race to develop some relevant quality algorithms and spam filters for real time search.

I also asked the question here, When Do You Use Twitter Search vs. Google Web Search? I said that at times, I get better results on niche topics such as SOA or Agile development through Twitter search than through Google. I also get more current information.

Theron Kelso based on his own research added the useful comment: “Twitter search is better for: Current events, popular culture, and direct access to people for the purpose of continuing conversations.

Google is better for: current research findings (formal) in Science, Technology, Engineering, and Mathematics (with the possible exception of computer technology), how-to documents, and historical documents.”

Good points. However, the potentially fatal flaw for Twitter is that it dumps its data index after three months so you cannot search back after the ever rolling three month window. I use Twitter for personal knowledge management, just as I do this blog. I can go back and find stuff I blogged four years ago. The blog is a running record of what I feel is important. With Twitter I am saving useful links by tweeting or retweeting them. It is a great source to find information and to archive it for short term use. It complement sthe blog. I also tweet about events I attend but with the three emonth data index dump I cannot go back and find this information. So I am going to look for another real time search tool that can provide this, as long as they can migrate the traffic. Friendster lost followers to MySpace who lost some followers to Facebook. Will this happen to Twitter? You need to be able to go short and go long. Perhaps someone will build an archive app on top of Twitter just like TweetDeck works as an interface app.

So I discussed search-based applications and real-time search. Now where else will Web search go?

b

Expanding Web Search Part One: Focused Search-Based Applications

This is the first of a two part series on how the boundaries of search are being pushed beyond the familiar Google style broad Web search. Tomorrow I will discuss real time search. I recently spoke with Eric Rogge of Exalead about how search is playing an expanded role for enterprises inside and outside the firewall. He mentioned that the major players such as Google, Microsoft, and Yahoo are continuing their battle for broad based Web search. Microsoft has now brought Bing into play in this market.

Exalead is not competing here but focusing instead on the other end of the market, operating in the long tail by enabling businesses to create purpose-built search-based applications. I have written about them before (see Exalead's CloudView Offers Integrated Search Capabilities). CloudView is one way of attacking the question I raised in my post, We Need to Tear Down the Social Media Silos. Exalead is bringing search to new enterprise tasks and expanding its role through these search-based applications. See my post, Tweepz brings Exalead search to Twitter for an example of this. In our most recent conversation, Eric talked about other search tools they've built to find and refine all results from the ocean of Web content. These include audio, video, image-based, and even ranking tools.

For example, they and a systems integration partner built a search-based application for GEFCO, a French logistics company than manages the transportation and delivery of new cars from factory to dealer in Europe. They needed to be able to look at inventory and then keep track of delivery status. GEFCO tried to do this in the traditional manner with a SQL relational database but it was too complex. With a structured database you need to set up multiple data entry forms that require specific data types of data (e.g., car model, location, etc.). There needs to be many data entry forms to do the proper inquiry and car dealers and sales people get frustrated with these specific requirements.

Exalead was able to change the approach for GEFCO to a search based application where you typed in information in a single field as in Google and the search engine did natural language processing of the request without the requirements of a structured database. This greatly simplified the task and gave the car guys what they were used to on the Web. However, in this case you do not want thousands or millions of results as in Google but the single correct answer (e.g., where is the car? When will it arrive?). This application won an Europe award for IT innovation.

Eric said there are many applications, such as this tracking system, with very focused requirements. Another example is patient records in a healthcare system. Exalead also implemented a search-based application for the French postal system to track packages and people. They wanted to be able to answer such questions as where is the package or where is the postal worker? It integrated information from several databases to answer these questions through one interface.

He showed me several other examples. There was the Wikifier (see below), a use case example (wikifier.labs.exalead.com). It parses a web page and creates tags that aid in search. It will also recognize a topic and create a link to a relevant page such Wikipedia article or a search results page from Exalead's web search tool (www.exalead.com/search). It is a connecting tool or information mashup. As you go through the page you can hover over content and see relevant background information. This same concept could be applied to an enterprise application that covers business topics.

In another example, Eric showed me a restaurant locator tool called 'RestMiner'. Another use-case example, it mashes together a number of information sources to you get reviews from multiple review sites. A buzz score is calculated from data drawn from multiple sites. Here is another way to break down the information silos on the Web. Google Maps is also integrated to give you location information. Like the Wikifier, RestMiner was built as a proof of concept and could be applied to many business applications.

These examples shown some of the possibilities for search-based applications focused on very specific topics where you want more precise answers. Metrics such as the buzz meter can be build into the application to better aggregate and interpret the data. Many companies have data located in a variety of sources and formats. They do not have to rebuild these applications if a search-based application can work across them breaking down the silos. The Exalead concept of search-based applications has been well received. They had 80 percent growth from 2008 over 2007 and are on track to do it again in 2009. You can also read more about this topic on the Exalead blog.

When Do You Use Twitter Search vs. Google Web Search?

I recently came across an interesting article, Twitter and Local Search: A Status Update, by Michael Boland, Search Engine Watch. Michael conveyed the idea that Twitter search brings structure to all the noise on the Web with an index of real time conversations. I agree and now, at times, I get better results on a niche topics such as SOA or Agile development through Twitter search than through Google.

Michael points out that while Google spiders are quick to index new content, it still takes hours or days rather than seconds. He adds that's perfectly okay if you're researching a new flat-screen TV, or the history of Mongolia.

But it could be less effective in emergency situations or where to get the best pizza in San Francisco. Twitter search covers conversations, while Google search covers topics. When do you use Google search and when do you use Twitter search. What has been your experience?

Tweepz brings Exalead search to Twitter

I have written about Exalead before, see Exalead’s CloudView Offers Integrated Search Capabilities. They have a web search tool like Google that is used in widely in France and other parts of Europe. The CloudView product looked inside the entoerise. Now they have a new web tool addressed at Twitter.

Exalead’s Jochem Prins built Tweepz in about five man-days. It went live quietly on March 16, 2009. It currently indexes about a quarter of Twitter users. Tweepz, allows users to find people on Twitter by name, location, or bio and navigate and refine these searches further by extracted metadata, language, follower ranges, etc. It’s built on Exalead technology to employ full-text search, natural language processing, and smart classification.

You can also filter your results. You can narrow your results by number of followers, the language they speak or extracted entities (key words which show up in their bios) all of these divided into percentile ranges. RSS feeds for specific queries are available, as is the simple ability to tweet the results you are seeing.

Tweepz also hopes to index the content of Twitter users’ nominated URLs in the future, thus offering a larger scope and capacity to return meaningful search results. I wonder of they will take this into the enterprise also and look at enterprise micro-messaging tools?

Future of Search?

Here is an interesting bit from Liz Ruskin, contributing editor of SmartBrief on Leadership taken from a panel on Milken Institute Global Conference in Los Angeles.

“Greg Papadopoulos, CTO of Sun Microsystems, said that now we’re impressed when Google searches a sea of information for us in mere seconds but we should be dissatisfied with this mode of search. In the future, he said, search will be done for us. Every electronic device we buy will connect to a network and our search results will be based on things like our location to our preferences and past behaviors. Search engines will know what we mean, not just the definitions of the words we use to describe it.”

This is an interesting tease. I tried to find more on Google but only found references to his views on cloud computing and open source. He likes them. Does anyone know more about this?

67 Social Media Web Reputation Monitoring Tools and Sites

I short while back I posted on Ten Ways to Track Your Online Reputation. Marc Meyer went 57 varieties further and posted on How to listen in a bad economy - 67 social media/web/reputation management tools and sites. Some are free, some are cheap and some are pricey. I say generally you get what you pay for.

His first one was Google Alerts, free with many false positives and it misses a lot but the price is right. He also listed tow others I use with are better, Techrigy and Filtrbox. I use the free version of Techrigy and the least expensive version of Filtrbox. They both give me stuff that Goggle misses, including coverage of Twitter. I learned about them when I interviewed their CEOs for blog posts.

There are 64 over options including the ten in my prior post. What web monitoring tools work best for you?

The Evolving Google Rankings Algorithm

Rand Fishkin, SEOmoz's CEO, recently posted his personal opinion of how some of the key factors in Google's ranking system have changed through the years he has been in SEO. I learned about this from @o0x on Twitter. Now I do not claim to be a technical SEO expert but I have figured out how to get blog posts on the front page of Google. I did this first just for fun and now (warning commercial coming) to help others. So I was very interested in his opinions. He covered the four big factors. I provide a brief summary of what he said for each. Go to this blog for the details. These factors were the ones I knew so I was very interested in the graphic on his blog that shows their relative rise and decline.

Domain Trust/Authority – He said came this into consciousness in 2003 and over the last couple of years, has become the dominant factor in the success of rankings at Google. Anchor Text in External Links – He said while this is still a very powerful ranking tactic, it's not as it was before 2004. I really felt that it didn't carry the same power it did for those years afterwards. “Today, my belief is that anchor text has come to be regarded much as PageRank was after its dominance in the algorithm - as a technique that SEOs have focused on gaming to such an extent that much of it has become noise, and it's only really valuable when found in conjunction with other positive signals.”

On-Page Keyword Usage – He said that it not lost all of its ranking power, but it “still sits in the middle as an essential element, but one where "more" won't help you. This is in direct contrast to the other elements I've included in this diagram (where more does equal better rankings).”
_

Raw PageRank / Link Juice - PageRank used to be everything and it built industries of link sellers and buyers. PageRank has been on decline as a ranking factor so some time. Remember Google keeps its methods secret and is always changing them to stay ahead of the gamers. So this is just his opinion. However, it is consistent with my intuition and experience.

My advice is to not game but do what best communicates to your readers. I use the second and third factors above more but with the goal of providing better information to readers. My understanding is that the first goal is more something you achieve than technique. I stay away from gaming methods on both blogs and Twitter and just try to have meaningful conversations. Also, remember that you have to speak to two audiences, Google and your readers. If you only speak to Google you might get readers but they will not stay or come back.

What has been your experience?

Post Script Thanks to Searchnewz for reposting this.

Twitter Search Traffic Ready to Pass Google Blog Search

Steve Rubel recently posted, Twitter Search Traffic Poised to Eclipse Google Blog Search. He wrote, “As of February, Twitter Search attracted 1.35 million users while Google Blog Search, which has been plagued by relevance issues, sits at 1.38 million users.” Now this is impressive for Twitter search but I am sure there is no comparison to Google web search. I never use Google blog search since blogs show up quite well on Google web search and you get a more complete picture. So, if my actions are at all typical I am not surprised that Google blog search is flat.

I have recently started using Twitter search for a variety of reasons. I use it primarily to search on topics and not to find mentions of my own work or presence. I have other tools for this purpose that pick up Twittering (e.g., Techrigy and Filtrbox). But twitter search is useful to look at what is being said about events (e.g. FastFoward 09) or issues (e.g., knowledge management aka KM).

Steve closes with this thought, “Keep an eye on the social search space. It's not a short-term threat to Google, but it certainly represents a major shift in where and how we will search for relevant news and information by layering in trusted sources.” I think this is the future of gathering content from the Web, the mix of trusted sources with topic search.

Techrigy Brings Comprehensive Social Media Monitoring and Reporting

I am crossing posting this from AppGap as I am now using the free version of this tool and find it quite helpful. The new Web offers great transparency if you are able to find what is being said on a topic in the maze of information. I recently commented on ten reputation management tools see - Ten Ways to Track Your Online Reputation. A few days ago, I spoke with Aaron Newman, CEO of Techrigy about their offering in this space. As we started the conversation I realized I had his book, Enterprise 2.0 Implementations, on my desk. The review copy was given to me after I interviewed Aaron Fulkerson of MindTouch, the book’s technical editor. I have only read a few chapters so far but it looks like a great resource.

As we discussed, people are turning increasingly to the Web to help with decisions, including purchasing products. Companies and individuals are also becoming more aware of the need to monitor what is being said about them on the Web and to take action to respond to both positive and negative mentions. Techrigy is designed to address this need with a balance of comprehensive analytics, simplicity of use, and a modest price point. From what I have seen they have accomplished all three goals with their SM2 product.

They are building a warehouse of online conversation, adding ten million a day and have reached the one billion mark. With each conversation, they have added up to 35 fields of associated meta-data including profile information, sentiment and tone, demographics, links and more. This storage allows for more rapid search results that have associated demographics. I found it interesting that 800 million of these conversations come from blogs and 140 million come from Twitter feeds but they are not limited to these channels.

The SM2 analytics sit on top of this warehouse. SM2 has 2 types of searches: Basic and Advanced. A basic search is the preferred method, but only supports the AND operator. For example: “Obama” and “President”. From time to there is a need for other operators such as OR, NOT, NEAR, etc.

The advanced search feature has the ability to support these searches. For example: “(Obama or Bush) and “President”. SM2 searches also have the ability to exclude phrases from a search. Adding an exclude word will cause a result to not appear if it contains one or more of the phrases configured in the exclude phrases. “Obama” and “President” with an exclude word of “Inauguration” will exclude all postings with the word respectively.

SM2 produces a variety of reports and allows for extensive filtering of these reports. Below you can see a daily report with some of the possible filters indicated in the drop down menu. The left column also shows some of the many ways to look at the data by daily volume, themes, demographics, map overlay, sentiment, etc.

Sentiment was one of the views that caught my attention. You can see an example of the reports of positive and negative sentiment for brand below. You can drill down on each mention to see what was said whom and their demographics and also change the computer generated rating to correct any incorrect assumptions (e.g. pain might be positive or negative for a mention of aspirin depending on what else is said). This way you can fine tune the results.

In addition to looking at the perceived expressions sentiment you can look at the overall tone of the mention on a five point scale as seen below.

You can also sort the mentions by their channel to see where the conversations are occurring. The screen shot below shows an example where Twitter leads the pack for this brand with 190 mentions. Following are a number of blog tools (LiveJournal, Word Press, Blogger, Typepad that in aggregate total more than Twitter. Next in order are: Ning, Wikipedia, Flickr, MySpace, YouTube, and Facebook. This gives you are idea of the variety of channels covered. You can also sort results by the demographics of the conversation author, such as age or gender, as well as by the relative popularity of the site where the mention occurred.

In addition to looking at the channels mentioned above and others you can set up a custom search to look at any website that contains an RSS feed can be added to Custom Sources. Upon configuration, SM2 will run a full archive and pull posts from the entire site. After the full index a daily retrieval will pull data from the RSS feed, searching from the last point of collection to the current search time.

When you set up a search, you can filter by language, country, time range, and other factors. After the results come back you can mark unwanted sources as spam to avoid getting them again. You can also filter out results by key words that shown up a lot on false positives to more quickly clear out the irrelevant results. You can get email alerts on a daily basis or even a real time basis for especially sensitive monitoring.

There is a free version that allows you to play with Techrigy but it is limited to 1000 results that you can quickly use up. However, this version does serve as a nice introduction to the tool at no risk. As I mentioned above, I set up a free account and explored it after talking with Aaron. While the free version has a reduced feature set, it does give you a good feel for the power and depth of SM2. It is very addictive and it was hard for me to stop playing with tool to write this review. The Professional version provides the full power discussed here.

There is a Techrigy blog. Among other topics, it covers ways to use Techrigy. A recent post, Twitter lead generation workflow with SM2 Alerts, discussed how Techrigy, itself, uses use Twitter in conjunction with SM2’s Alerts function to generate interest, book demos and communicate with people in the social media marketing community. They are able to set up the SM2 search filters to only look at people who are likely to be interested and then get real time alerts. Techrigy finds that they are now booking many sales meetings daily via Twitter. This targeted marketing makes use of the transparency of the Web and is a good proof of concept for Techrigy.

TVissimo Now Integrates Online Video Search with Its Broadcast TV Search

We have now integrated online video search through three of the major sites with our brodcast TV search. You are looking for the newest episode of your favorite TV show and just missed it, but you suspect that it is available somewhere online. You love to watch reruns of your favorite TV series, but are not sure if and on which channel they may be airing again or if they are available on an online video site.

No problem, as TVissimo now combines broadcast TV search with online video. Enter a single search on TVissimo.com, and you can see the results for both broadcast TV and online TV, a two-week TV schedule as well as the available online videos from Joost, Hulu and TidalTV right now (more online sites are being added). (The Hulu indexing is in progress will be added within a few days.)

As far as we know. we are the only search provider that integrates broadcast TV and online TV. The way it works is when someone does a TVissimo search, based on the results, we also search the Hulu, Joost and TidalTV online video sites for matching entries and show them on the Video tab. When the user clicks on the online video provider icon, a new window opens up and streams the selected video from the providers site.

Each of these online video sources has a blog to follow their stores: Hulu blog, Joost blog, and TidalTV blog.

We are very pleased to add online video to provide a more comprehensive set of television resources for you. Let us know what you think. Here are two mentions, Digital Digest and Home Media Magazine. Thanks.

Attivio’s Thoughts on Improving the User Experience for Online Media Sites

Here is a comprehensive white paper on improving the user experience for online media sites from Sarah Meyer at Attivio. It offers this quote near the beginning:

“2009 PREDICTION(S): Five out of every 10 magazines and newspapers will go out of business, scale down their frequency or move entirely to the Web. This will not just be survival of the fittest, rather survival of the most willing and able to adapt to the changing media paradigm, and throw all of the old rules out the window.” - Andy Cohn, VP and group publisher, Fader Media

I have seen the same predictions for a number of sources so be warned that if you are a media company you need to get your web site in order to survive. This paper offers some excellent suggestions. Some apply to blogs, as well. In this era of user involvement, they rightly point out that you need both compelling content and an engaging user experience. They break their suggestions into three categories: focus on readers’ goals in the way you present content, address advertisers’ expectations, and optimize operations around innovation and cost control.

First, the readers view is covered. Sarah writes that most media company sites have a static design of site, low relevancy of search results, few services for building online communities, and limited access to historical content. As someone who frequently goes to media sites to track down stories I would agree with all these points.

Sarah then writes that advertisers are looking for significant page views to expose their ads, detailed reporting, ability to deliver ads to their specifically targeted audiences, and finally proof that you can offer something more compelling than the many alternative channels they have.

Operations need to be able to meet the needs of readers and advertisers in a cost effective and rapid manner. The paper goes on to offer suggestions for how to do this. These include adding opportunities for readers to contribute and explore (e.g., interactive charts that allow readers to change factors and see how those changes might deliver different results). Other suggestions include (in Sarah’s words):

“Aggregating content dynamically to meet reader and community interests.

Allowing more user-generated content that’s easier to read and interact with than a long list of user comments that are appended to a story.

Incorporating all relevant information in the content, opening up information resources – including all historical content – for content producers and readers. This will also provide incentive for syndication of your content.

Automatically generating links to relevant articles as new content is added to your site.

Replacing the simple search box with a more contextual exploration interface that iteratively helps readers find what they want, suggesting matches as they type. This ensures faster results and avoids the lost feeling users get when they have wandered through a series of pages without getting what they want.“

There is much more as the paper gets into specifics on how you can use technology to support these goals. However, Sarah also points out that is not simply technology. In addition to changes to your website, you need to look for enhancements to the way your staff works. This type of web site requires greater engagement by staff, as well as readers. Not all of the interactivity can be handled by software. This certainly applies to all the monitoring, analysis, adjustments, and responses. Excited engagement needs to occur on both sides of the site. It can be contagious. Getting the internal troops engaged can sometimes be the harder part for company sites, as well as company blogs.

I have written about Attivio before and they can provide the technology side to this equation (see Attivio Tightly Integrates Structured Data and Unstructured Content for a New Approach to Information Access). Be sure to also create a set of policies, practices, and governance for the people side. I recommend taking a look at the paper. It goes beyond simply pushing Attivio to offer general and useful suggestions.

iQuest: Putting Greater Transparency into the Enterprise

While a theme of the enterprise 2.0 is transparency, often the maze of different platforms, repositories, and metadata structures makes finding the information you need difficult, if not impossible. I recently spoke with Josh Rosenthal, founder and CEO at iQuest, about his experience with one of the top 5 international pharma companies that was struggling with enterprise search and had found it easier to re-invent a needed compound that try to find the existing one somewhere within their terabytes of data.

I have written about iQuest before (most recently - Extending Google Enterprise Search Through iQuest) and want to disclosure that I am a shareholder, but not an employee. It is a company that I have long believed in. Since that post, iQuest has completely revamped their product line to significantly improve its capabilities. Their reengineered software suite builds on the prior capabilities and extends them through a new software platform.

The phrama company we talked about had given up hope of being able to search their multiple terabytes of data in multiple repositories, diverse platforms, and different metadata structures with one tool until they ran a proof-of-concept trial with iQuest. iQuest gave them the capability to better employee a “find within” strategy rather than the ‘build all over again” strategy they had previously used.

Here’s what the company’s Associate Director for Worldwide Document and Collaborations Solutions said about the results of the trial. “I firmly feel iQuest has unique value, and takes an approach to enterprise search that is both novel and truly helpful in improving knowledge worker productivity. I sense that in less than 12 months your flagship product will be truly a breakout and market-changing event.”

iQuest now enables users to discover hidden patterns and relationships within and across large sets of data. In the past, discovering patterns in unstructured data was extremely difficult. iQuest’s approach to pattern recognition and matching combines a propriety data ingestion process with natural language processing, grammatical network analysis and Social Network statistical algorithms to create an accurate and fast data analysis process that delivers strong results across structured, semi-structured, and unstructured data.

iQuest greatly reduces the amount on manual management than is often found in enterprise search engines. You do not need to set up taxonomies or other data architectures. iQuest now provide a hardware/software solution that allows you to plug and play. Its “Contextual Search” semantic analysis then automatically generates connections and interrelationships across enterprise data repositories. It enables you to leverage the index of your enterprise information without having to build and maintain knowledge models using controlled vocabularies, taxonomies, and ontologies. 


iQuest leverages existing information repositories (DMS, ECM, CRM, ERP, etc.). It will sit on top of Sharepoint, for example, and find relationship across and within the documents and other data that shows up in a Sharepoint search. It will allow you to connect people and ideas within the documents through its text analytics. iQuest will do the same thing within any enterprise content management system. 


iQuest is designed to help your find what is missing or even did not know existed within your enterprise data. Its Intelligent Discovery™ uncovers thematic contextual links to content or raw information that's typically hidden or unknown, which results in actionable insights. Indexing content to better find information is a core function of search and content management systems. 
Content is ingested “as is.” It will look at dynamic social software content in wiki’s, RSS feeds, and chat. Though iQuest is largely targeted for use within the enterprise, it will also go out on the web and look at relationships between internal and web-based content.

iQuest is also designed for ease of use. It employs a Smart Architecture™ to eliminate the need to modify or add additional layers to the ECMS or existing index structures. This can save resources – people, time. and money. 



Josh writes Patterns, the iQuest blog where discusses the relationships between and among people, ideas, thoughts and words. It has some useful conceptual pieces that go beyond what is happening at iQuest.

TVissimo Relaunches Today with New Interface

Today, we have re-launched TVissimo™ with an improved interface. I am pleased to be part of this effort and continue to be the main author on our TVissimio - TV Schedule and Online Video Blog. Visit our blog for tips and techniques on using TVissimo. We also cover the television and movie industry and industry bloggers.

TVissimo.com is a free television schedule and online video search engine to help you find TV programs and movies you want to watch, quickly, easily and free. Whether you’re at home or away, you can instantly find the precise TV show schedules and your exact channel numbers. With hundreds of channels delivering thousands of programs daily, printed TV schedules have become pretty useless, while on-screen cable or satellite schedules are slow, difficult to use, and offer only limited details and functionality. Finding online (streaming or download) content requires searching many websites in different formats. TVissimo provides a single entry point for finding content everywhere.

TVissimo.com is doing for TV what Google™ has done for the web. Rather than focus on channels, TVissimo is designed around the concept that people have TV-Show-Type favorites, regardless of what channel the show is on. To see differences between various TV show search providers, go to TV-Show-Schedule.com. If the show you’re looking for exists, you’ll find it, unlike other top-ranked services that often come up empty.

Search any way you prefer, using Google-type keywords or phrases, for Categories such as adventures, football, children’s cartoons — or by any criteria, such as description or actor/star/show-type. You can even specify prime or rerun, length, or time slot, and display the schedule sorted by program name or start time. For example, you can browse through a list of comedy movies that are on tonight between 8:00 and 11:00 p.m., or check if a Nicholas Cage movie is on. And you can print out your own customized TV schedule, sequenced by start-time or program name.

TV show listings are available for the next 14 days, refreshed nightly, all year round. Online video availabilities are refreshed daily for the currently indexed sites (TidalTV), with more added in the near future.

Go to TVissimo.com, enter your zip code, and pick your TV service provider (cable, satellite or fiber). You can also save multiple locations (e.g. grandmother’s house or hotel during a business trip).

TVissimo.com’s innovation is based on searching the available TV show details using a patent pending database search technology developed over a decade and applied to meet the challenges faced today by people trying to find their favorite television shows quickly and easily.

You can readily send TV program alerts to yourself or to friends by email or Twitter message. TVissimo friends’ lists are created by entering your contacts’ email addresses or Twitter IDs and specifying which of your favorite programs to send to any person or group. You can send business related shows to co-workers, entertainment programs to friends, religious programs to religious group members, etc.

Once your Favorites are stored, future searches will bring up only the shows matching your preferences, which can easily be modified later. For the first time in the new TV world of hundreds of channels and dozens of online video sites, finding your favorite TV programs can be easy by putting an end to channel surfing, never again missing your favorite shows or starting half way through them.

Catch the Introductory video on TVissimo and watch “The Life of Remo”, the abused remote control through YouTube.

Fast Forward 09 Goes to Vegas

I had a great time at Fast Forward 08 - see Blogging Fast Forward 08 for the details. The only down side was the location - Orlando. Now they have greatly improved this factor as the show goes to Vegas. FastForward 09 will be there Feb 9 -11. I have not been there since I stopped by in 1967 on a cross country trip back to Boston from LA. 

The conference also has a great lineup of speakers and sessions. Speakers include Clay Shirky, Don Tapscott, Charelene Li, John Lervik, Bjorn Olstad, and others. The site says that FastForward 09 provides a much-needed forum for the discussion of best practices for developing and implementing search solutions as the evolution of the approach to user-centricity continues – both outside and inside the firewall. 

Here is how they elaborated on these themes. Outside the firewall, as users push to create, collaborate, and consume information and products in new ways, businesses are driven to find innovative ways to connect with this user-centric audience. On public- and customer-facing Web sites, search drives revenue by increasing customer loyalty and conversion, and monetizing existing assets. 

Inside the enterprise, as corporate data stores grow and as workforces become more mobile and geographically dispersed, businesses need to find effective ways to help their employees connect and collaborate. Search increases productivity inside the firewall by helping workers find relevant information and expertise faster. Search also improves business decisions by helping people make better use of what an organization already knows – connecting them to the latent and relevant information they need to make informed choices. 

There is an early registration discount at the FastForward 09 site. I hope to see you there. I will be blogging the event.

Google Trumps Microsoft on Yahoo Search

These Google guys are smart but you do not need me to tell you that. They killed off a potential competitive threat and increased their revenue stream at the same time. This is old news but I still had to post it, just to admire the symmetry. As reported in the New York Times, Google Ends Microsoft’s Yahoo Search, the following occurred.

Google’s amazing success and expansion on the web prompted Microsoft, the alpha technology company for two decades, to court Yahoo, a web search rival to Google. They already got Fast for enterprise search which is more in their sweet spot. Google’s success had weakened Yahoo enough to give Microsoft the confidence it could buy the company on the cheap.

Microsoft thought that combing with Yahoo would create a powerful competitor to Google. At first Google indicated they would play hard ball and fight the merger on antitrust grounds in Washington and Brussels.

However, they came up with a better idea. Google offered to let Yahoo use its killer search advertising technology, which by might add a $1 billion in additional cash flow a year for Yahoo and bring Google more revenue through a new channel.

Guess what? Now the prospect of such a partnership and new cash flow pumped up Yahoo’s board to demand more money from Microsoft and eventually caused Microsoft to abandon its quest. That was too easy. The market liked the play and added $4 billion to Google’s market value through a jump in stock price.

The Times article said that “Google has built a highly profitable $16.6 billion empire a dollar at a time,” referring to its cash cow the Google Ads. So this Yahoo move was a short cut to more riches. I get an undisclosed amount of small change a day from my Google ads but at the end of the year it adds up to something slightly more meaningful. Too bad they do not pay in stock.

Answers to Google Results Contest

Here are the ten terms I listed yesterday in the order of their Google search results. Wikipedia was often in the top ten but not always. How well did you do? What were the surprises for you? It is interesting to see that YouTube beats economics.

Music – 2,670,000,000
Books – 1,930,000,000
Sports – 1,840,000,000
Art – 866,000,000
Science - 781,000,000
Songs - 369,000,000
Politics - 351,000,000
Literature - 255,000,000
YouTube - 184,000,000
Economics - 113,000,000

Someone actually took the time to define a song on Wikipedia as "A song is a musical composition. Songs contain vocal parts that are performed with the human voice and generally feature words (lyrics)" and a book as "A book is a set or collection of written, printed, illustrated, or blank sheets, made of paper, parchment, or other material, usually fastened together."

Google Results Contest

See if you can place the following ten terms in the order of which has the most results from a Google search. This might roughly equate to how often they are mentioned on the web. The answers will appear tomorrow. No cheating and going to the source. The data was collected a few days ago.

Art
Books
Economics
Literature
Music
Politics
Science
Sports
Songs
Youtube

Microsoft Goes Big Time in Search with Fast Purchase

As reported in Channel Web: "Microsoft (NSDQ:MSFT) has offered to acquire Fast Search & Transfer, a Norwegian developer of enterprise search technology, for approximately $1.2 billion, Microsoft announced Tuesday. The acquisition would help Microsoft expand the information search and business intelligence capabilities of Microsoft SharePoint Server and other products."

Well, Sharepoint could sure use the help. I would not want to be one of the many third parties that developed search tools to support this weakness in Sharepoint. It is also a big stake (at least it is a lot of money to me) to place in the search area. To be clear, I think enterprise search is one of the key areas and no stake is too big, I was just referring to the pocket books of these firms. I should figure out something to sell them. I wonder what Google thinks? What will be their counter move?

The news goes on to say: "Fast Search & Transfer's two largest institutional shareholders that own 37 percent of the company's outstanding stock have accepted Microsoft's cash offer of 19.00 Norwegian Kroner per share, according to Microsoft." Thanks a lot of kroner. The must really want Fast given the premium they paid with the weak dollar. It reverses the trends of non-US firms looking for bargins here.

This also shows the increasing importance of enterprise search. Fast sponsors the Fast Forward group blog on enterprise 2.0 that Corante runs and I contribute to. I should have asked for shares. Here are some thoughts on the pending purchase by my Fast Forward colleague, Rob Paterson.

Fast is also sponsoring the conference by the same name, Fast Forward 08 on the theme of the user revolution. This news should add some spice to the event. It will be on Feb 18-20 in Orlando and I look forward to it. Meanwhile, if you want your company to obtain similar sale prices, send me an email and I will write for your blog also.

Do You Want Freshness with Your Google?

The NYT had an article recently on the inner workings of Google, Inside the Black Box. One interesting problem they were working on was how important to make freshness in a site versus an older site which had more links. They found that simply changing search formulas to display more new pages leads to lower-quality searches much of the time. So they came up with a more complex approach called QDF or query deserves freshness.

As the Times wrote. “THE QDF solution revolves around determining whether a topic is “hot.” If news sites or blog posts are actively writing about a topic, the model figures that it is one for which users are more likely to want current information. The model also examines Google’s own stream of billions of search queries, which Mr. Singhal believes is an even better monitor of global enthusiasm about a particular subject. As an example, he points out what happens when cities suffer power failures. “When there is a blackout in New York, the first articles appear in 15 minutes; we get queries in two seconds,” he says.”
Now this is a good idea. Google activity often reflects many things such as the most popular terms for a concept and, as reported above, what is hot. Most of my new readers come form Google and it is my default web site. Here is another example of the thinking that keeps them there and makes them the most popular web site. enterprise 2.0 web 2.0

More on Onomi: Social Bookmarking on a Corporate Intranet

Recently, I gave a presentation on tagging to the Taxonomy Community of Practice. I was joined by two speakers, including Jim Smallwood of Mitre. Jim is part of the Onomi team and provided a very useful description of Onomi and its development. I twrote about Onomi before but here is more detail. Mitre did a pilot for this social bookmarking system for six months. It was built for people working on real projects. The created a core group of individuals who set the foundation for creating tags. These core people will keep the system robust. A larger group uses the system but does not add to it. Use it to see what is new and locate experts.

There is already a lot of information sharing at Mitre including Sharepoint based discussion groups. Tagging was to supplement this existing sharing in many forms. Hoped to build more communities around common interests. At the same time a podcasting effort – Mitrecast was underway. The podcast people contacted them to share tags. Also an internal RSS aggregator linked to them. They want to leverage the folkonomy created with Onomi to update the corporate taxonomy. Mitre is still exploring these issues.

The Mitre team took an exiting open source application, Scuttle. It was chosen for ease of use, extensibility, and existing feature set. They went to core users to get what features to add. Moved it form Sequel to oracle and implemented single sign-on. Help to recognize people and use LDAP. They added ability to easily email bookmarks to people.. provide default email text to add context to tag. They added ability to browse bookmakrs by users, or by corporate affiliation. RSS was also added.

You can provide a title for each bookmark and 250 characters to describe it and add tags. You can see other resources with this tag are listed and number of users who have bookmarked it. You can search by tags and by the words in the description field. You can add additional tags to your search to narrow the results.

The average tagger has about 35 bookmarks and uses 3 to 4 tags for each item on average. They have promoted it in several ways. The posting of banner ads on the corporate intranet have led to immediate increased use. They also held meetings to promote the system. They posted use cases for the system to give others ideas and motivate them to use it. enterprise tagging enterprise social media

Adding Social Network Analysis to Search

I wrote a piece for Enterprise Search Center on Adding Social Network Analysis to Search. It recently became available on line so I wanted to share this. I opened with, “The web today is about participation and participant-created content. The most effective web search tools take this participation into consideration in the process of delivering relevant results. A look at these techniques (and some of the problems with them) can lead to insights into exploring the relationship between social context and search results inside the firewall as well.” And then went on to compare Google and iQuest and other search tools. You can get the complete article for free by registering at the link above.

There are a lot of other useful features, white papers, research reports and other stuff at the site.

Reverse Geocoder – Great Tool

This is new tools week. I learned at a recent Berkman Blog group meeting that John Coryat has used a US ZIP code database to reverse geocode the continental United States along with Alaska, Hawaii, and Puerto Rico. Check out his reverse geocoder site. If you click on a location on his version of a Google Map it gives you the longitude and latitude. I was reported in the Googe Map API Blog.

This is just another cool tool that emerging from the open APLs that Google provides. It seems to be the biggest source of mashups. I zoomed in and found the longitude and latitude for my home. Have fun.

The Social Bookmarking Faceoff – Read/WriteWeb

I came across this useful assessment of social bookmarking tools through looking at the Digg Labs swarm capability. This is worth a post in itself and John Maloney pointed me to it. Digg Labs swarm is a way of visualizing the connections between blog posts and other web content.

But back to social bookmarking. Alex Iskold reviews a number of tools. He concludes. “The social bookmarking market is in a steady state with two dominant players - del.icio.us and StumbleUpon. The rest of the pack, including Yahoo MyWeb, appears to be substantially behind.”

Alex did not profile social news sites (like digg) or social shopping sites (like Kaboodle), I would agree that while they do have a book marking function they belong to different class, more like the rating sites.

I think that tools like del.icio.us are useful to individuals an dI have used del.icio.us for a poject. But I feel they have limited use within enterprises because they are so open ended. The lists get quickly unmanageable.

I like the H20 Playlist that I have written about before. It provides much more structure so you can more easily divide and chunk lists so they do not get so unmanageable. I am using it to document my web 2.0 research and I will be writing more about this soon.

Key Word Analysis Tools

For most sites over 80% of traffic is the result of search engines. I see that for myself in my blog. So determining the right key words to focus on (using the methods above) can be helpful. Amanda Watlington did a presentation on this at the Berkman blog group a while back. I was looking at it again recently, Here is what I found.

The more expensive tools are for people who buy key words for ads like in Google to see what to spend money on. If you are not an advertiser but want to use the right key words on your blog you can try Overture Key Word Selector to get a feel for them and it is free. This may be all you want to do. I do not claim to be an expert here but there are many including Amanda. Here are some tools.

Overture- Keyword Selector Tool This is free tool. You can compare terms for example, knowledge management got 13,040 in April 2006 and KM got 5255 so I would want to use knowledge management.

WordTracker – They say that” with Wordtracker you'll know which are the best keywords to drive more traffic to your sites.” It is not free or even cheap but you can take a free trial.

Build traffic – seems less expensive that Word tracker but still not free – they do offer a free trial

Good Key Words - They have several tools including Keyword Suggestion Tools that help you identify additional keywords. You type in one or more base keywords and the suggestion tool will provide you related keywords and phrases that are typed in by online searchers. They also link you to keyword top lists that allow you to obtain statistical data relating to keyword searches in engines like Google, Yahoo, and Ask Jeeves. These lists are offered by the search engines, themselves.

A Number of Responses on Tagging and Search

I have been been getting some very useful responses to my posts on tagging. In this post I want to list a few of them for easy reference and briefly record my follow up to their links. Here they are listed in no particular order.

Steve Eisner, VP of Engineering at AskMe Corporation, writes the blog, A Social Life. He recently wrote about Tagging vs. Searching in which he wrote, “A lot of people (myself included) have tended to more or less equate tagging with “del.icio.us”. Delicious doesn’t search well, and google doesn’t tag well. That creates some kind of artificial distinction between “tagging” and “searching”. As a result, people get distracted trying to demonstrate the (re-)findability value of social software (perhaps because search is so generally useful that impact on search has become the easiest measurement of value?).” and more.

Stephanie Lemieux documented a discussion on The Value of Social Tagging in a Corporate Setting in the Taxonomy Community of Practice wiki. Among other things she related a dialogue around Raytheon’s internal tagging efforts so here is another example of tagging behind the firewall to go with IBM and Mitre.

John Lemke aka Lumpy shared this Slacker Manger Article, The Several Habits of Wildly Successful del.icio.us Users, which looks great. He said that he found himself searching much, much less after reading it and found himself learning much more as well. He wrote this post on the topic, Delicious vs. Search, in which eh said that tagging is not a replacement for search but it is an alternative. Lumpy shared this in his comments on this blog, “Search, Knowledge Management and tagging are related but each one is very different. I don't think tagging will replace search but I feel it fits into the concept of knowledge management much better than search.” I agree with this point. Here is an interesting post he wrote on one of his blogs on the topic, Tagaholics Unite! — More on the joy and chaos of tagging. There are a lot of useful links in it.

Andy Havens added a very long comment in which he concluded, “I'm not saying tagging is the "death of search," but I've found that once you start using del.iciou.us and get used to the quirks and personality issues that come with referring to how *people* rank and judge sites rather than how a machine does it... you get not just better results, but deeper and more interesting and fun results. It's especially interesting to go through the tags of one user once you've found a really spot-on site from them. It's like finding a lost friend and then picking his pockets for notes.”

I agree with what Andy wrote but I would still use Google for quick searches and del.icio.us when I have the time to explore a topic in depth and see what others are reading on it. enterprise tagging

Even More Tagging Articles

I received some useful responses to my request for tagging articles and blog posts. Thanks to those who responded. I am not going to comment on these just yet but offer them as a resource for others as I found all the ones listed below to be useful. New favorites are welcomed.

I am still getting responses to the first request so look back in those comments to see if there is more stuff. Also, when I looked, the wikipedia article on social bookmarking is more useful that the one on tagging.

From Patrick Cormier, Information Management Now:

More on Folsonomies by Parick

Tidying up Tags?" by Marieke Guy and Emma Tonkin

The Evolution of Social Tagging - InfoSpaces, a blog by Emanuele Quintarelli

Folksonomies: a Web 2.0 lesson learned, InfoSpaces, a blog by Emanuele Quintarelli

Folksonomies 2.0 - The Chaotic Order - InfoSpaces, a blog by Emanuele Quintarelli

The Several Habits of Wildly Successful del.icio.us Users - Slackermanager

Best Web 2.0 Explanatins Dion Hitchcliffe (okay not exactly on topic but good anyway)

From Amanda Watlington, Blogs and Feeds:

Where Tagging Works- Search Engine Lowdown – Chris Sherman enterprise tagging

Tagging Not Likely The Killer Solution For Search – Search Engineowdown – Danny Sullivan

From Niall Cook, Cognez

The case for enterprise bookmarking - From Niall Cook, Cognez

Making libraries more delicious: Social bookmarking in the enterprise – URLgreyhot

From Scale/Free: List of tools for the internal blogosphere

From Jack Vinson - A social and cognitive analysis of tagging - Rashmi Shinha (I wrote about this great post before)

Checkmates: Mobile Social Mapping Mashups Continued

Tomoaki Sawada pointed me to another mashup tool. They are springing up everywhere. This one is called Checkmates. It combines mobile phones, people, and location. Edward Ho, one of the developers, talks about it at his blog, will work for doughnuts. There is likely a lot of parallel development because it is so easy and the open APIs of Google Maps plus the utility of the links to Google Maps data.

Checkmates sounds like what Michael Fedor of the Monitor Group did. He writes about how he linked his Treo smart phone to other services to extend Google Maps’ functionality through "Location Aware Services,” adding a new dimension to how he can us his cell phone. He does this through the mobile mapping program Kmaps. It allows him “to tag locations on a map, snap photos, edit notes, and upload location-based photo blogs to the web with the click of a couple buttons.”

Michael gave a nice story of its use that I wrote about a few weeks ago, Michael Fedor’s Excellent Adventures with Mobile Mash-ups. Now what Edward and his friends are doing is also a good thing. It also seems that Checkmates is continuing to extend its functionality. I see all this as cumulative innovation as people keep connecting applications in new ways and building on what others have done.

Beth Shankle (see yesterday) also told me about another twist on this theme,. “Helio users will be able to do such things as use their phones to take a photo of themselves at a club and post it to MySpace with an invitation to join them. Users will be able to use their phones to post blog entries on MySpace or access their MySpace contacts and photos.”

You can run but you cannot hide. The mashups will catch up and tell everyone what you look like and where you are. Now there might be a lot of creative applications of this capability.

Good search is good knowledge management: James Robertson

Here is a good brief article on the importance of search for effective knowledge management. James Robertson writes, Good search is knowledge management, as one of a series of reports he produces at Step Two designs, an Australian content management consultancy. In “Good search is knowledge management,” James also links to two earlier useful reports on the subject: Intranet search reports and Search engine 'best bets.'

He makes several good points. First he says that “search is often implemented as a 'second thought' within organizations. First the intranet or document management system is installed, and then search is added to help users 'find stuff' within the growing repository of information.”

I have seen this happen many times. I have also seen poor search capability listed as one of main reasons intranets and knowledge managements systems are not used. This even occurred for one major telecom where there was documented evidence that people performed better using the knowledge management system. It was just hard to get some to use it because they were frustrated with the search and the interface.

James suggested that one of the first steps is to implement search reports. He says that the two most useful are: “most popular search terms” and “failed search terms.” This will let you see what people and want and what they need to paraphrase the Stones. Taxonomies also help with the standardization of key words. This was one of the initiatives we did to improve usages on the underused telecom knowledge management system mentioned above.

I recently met with Carl Frappaolo and Dan Keldsen from the Delphi Group which offers workshops on developing taxonomies, along with many other good things. They agree with James’ point that you cannot just plug and run search engines to get the most effective results. Here is one of their events, Proving Ground for Information Architecture & Taxonomy: Introduction to be held in Florida on March 7-9. Dan has an interesting new blog, BizTechTalk from the Edge to the Core.

Trimergent- Build Your Own Initiative Specific Portal Through Search

Here is an interesting idea. Build a desktop portal through search using a new tool, Trimergent. A nice thing is that you can create a custom desktop for a project team and then share it with others. You can have as many desktops as projects and archive them for future use or to serve as templates. Buzz Bruggeman of Active Words introduced me to Ashwani Sirohi who showed me a bit on how Trimergent works and it seems quite useful. You identify and aggregate content from a variety of sources to meet specific goals. They refer to the outcome as Personal Information Networks (PIN) and good security is also included.

Another nice aspect of PINs versus enterprise portals is that they are in control of the user, like blogs. This is another example of the bottom-up user control trend in software. It must have some of the old school IT guys going nuts. But the new school ones will recognize that putting tools like this in the hands of users and not fighting the trend can actually make their job easier and their budgets more manageable.

It addition to sharing your initiative specific search results with colleagues, they can comment on what you found and you can also do the same to their efforts. Here is another example of the recognition and movement toward better participation and collaboration through the newer entries in the enterprise software area.

Take a look.

Google Big Time Blog Search

As Search Engine Watch reported, “Google has introduced its long awaited blog search service, becoming the first major search engine to offer full-blown blog and feed search capabilities…Google's new service (in beta, naturally) is available both at google.com/blogsearch and search.blogger.com. Google blog search scans content posted to blogs and feeds in virtually real-time, according to Jason Goldman, Google product manager for blog search…Google defines blogs as sites that use RSS and other structured feeds and update content on a regular basis” There is much more in the article.

Google likes blogs and they often show up high on regular Google searches, all without quotation marks. However, I found some real differences in simple comparison. I tried – knowledge management – and did not find a single blog on the front page of regular Google with 312,000,000 hits. The blog search, with 72,775 hits, had one of my posts on the front page, my comment on Dave Pollard’s piece on “Why knowledge management is so important.” I then tried – portals and km – and the regular Google found 999,000 hits. This blog was at the top but there wa snot another post from it on the front page, although there were several posts by others about this blog. The blog search had 377 hits and each item on the front page was a post form this blog. The high nature of hits is not a surprise because I was searching on my blog title. I am sure many other blogs will have similar results on their title. Just shows the importance of picking titles. Searching on my name in quotes had similar results as - portals and km - in both tools. I am surprised and the big difference in number of hits between the two. You should try some comparisons yourself. I will try some and be less egocentric next time.

Thanks to Lisa Nadile for pointing this out.

Google It – Even More Ubiquitous?

As the New York Times reports today in Google Profit Soars Again, Beyond High Expectations, “Google continued its rapid growth in the second quarter, posting a profit of $343 million, or $1.19 per share, more than a fourfold increase from the same period a year ago. Revenues doubled to $1.38 billion from the quarter a year earlier. Earnings per share, adjusted for certain stock-based compensation, reached $1.37, far exceeding analysts' expectations of $1.21.”

It continues to surpass its main rivals, Yahoo and Microsoft. I use Google as a given like the telephone, whenever I search and search is the second most common internet activity after email. I have written about Google a few times and its influence on many things like, Does Google Write the Next Dictionary?, Google Defines Stuff, Google Desktop Search has Arrived, and A Diller a Dollar, a Google Scholar. There are many more and if you “google” my blog you can find them. To google is becoming the generic term, like to Xerox but with an greater potential for expansion than photocopying. Where will this all go? Can anyone break into this spreading set of related capabilities? If the influence becomes too pervasive, will it become regulated like the telephone?

Buzztracker - Find the Hot Spots in the News

Here's an appealing application, Buzztracker, built on top of Google News. It presents a visualization of geographic connection in the day's news so you can see where the hot spots are located. Thanks to Jennifer Kurkoski for sharing this with the SocNet email group

Looking into the Messages within Blogs – iQuest

Blogs contain a lot of content to shift through for a variety of reasons (trends, major issues, etc.) My friend Josh Rosenthal and his colleagues at iQuest Analytics have developed a tool to help with this task. As their site says:

“iQuest can rapidly analyze anything with words in any language — emails, call center transcripts, message boards and blogs, text responses to survey questions, media coverage, documents of any kind — and surface patterns and associations without being prompted where to look or what to look for. Used as a stand-alone application or deployed in concert with iQuest Analytics' battery of survey and assessment vehicles, iQuest produces an in-depth data map of what your important constituencies feel, think and believe.”

Now what does this mean? The concept is that iQuest lets content speak for itself. The tool surfaces the top n (usually 10) concepts or themes that are being discussed in a given body of text such as a blog. This reduces much manual analysis and allows you to look deeper into what people are saying in large amounts of content. It also looks at everything and is thus does not let bias creep. Most manual analysis by people will have preconceived notions that consciously or even unconsciously sift out what might seem important. The automated tool is not “smart” enough to do this which is a good thing.

It displays the data in a “Contextual Network Graph style using nodes and links. The nodes represent objects and/or concepts, and the links represent the relationships between the nodes.” You can see some examples on their site. I have seen examples of the different outputs and the possibilities are quite robust. There is a lot more detail in their Under the Hood Section.

This could be a very useful tool for looking at what your market, customers, and competitors are writing about.

Google Blogs and Google Maps

I looked at the official Google Blog and found a new service, Google Maps. You can find a spot and get directions for how to get from one place to another. You can also get things like hotels near somewhere.

“We think maps can be useful and fun, so we've designed Google Maps to simplify how to get from point A to point B. Say you're looking for "hotels near LAX." With Google Maps you'll see nearby hotels plotted right on a crisp new map (we use new rendering methods to make them easier to read). Click and drag the map to view the adjacent area dynamically - there's no wait for a new image to download.”

I tried it and it works like Map Quest but with more aesthetic maps.

Google Adds Libraries - What’s Next?

They can look at your desktop, your blog, is your home next? Will they be able to find your missing sox in those orphan pairs? The New York Times reports that Google now plans to begin converting the holdings of leading research libraries, such as Harvard, into digital files that would be searchable online.

Even Blogs Need Keyword Research – Amanda Watlington

Amanda Watlington writes in an e-Marketing News article, Even Blogs Need Keyword Research, that bloggers, like other web site operators, should do keyword research to improve their blog's search engine visibility and usability. She has given me some useful advice so it is nice to see her thoughts in a public space. She notes that “many search engine marketers complain that blogs are crowding out quality search results (the pages they have optimized), and that one can hardly conduct a search without getting blog content in the results. My advice is quit complaining and consider why search results have so much blog content.”

Amanda argues against creating blog content just to promote keywords (e.g. keyword spam), but rather to consider keywords as you write meaningful content so more people will see the content you write. She provides some very specific suggestions for this in the article which I recommend.

I guess she included the right key words because it has already been commented on by Search Engine Lowdown, Cre8pc blog, Top Rank Online Marketing Blog, and jurgle.com.

Jessica Baumgart on Google Answers

I recently wrote briefly on Google Answers, a pay by answer service that Google provides. The service only finds existing answers without any commentary of the value of these definitions, which is all that can be expected for the small amount they charge. I felt that the ones they found for “knowledge management” were a bit superficial, but reflected some of the misconceptions in the market.

Jessica Baumgart goes into great detail on Google Answers. Jessica provides her commentary from her perspective as a professional librarian. She writes, “I'm not sure what kind of resources Google Answers' people use or have access to. I probably would not begin this search on the Internet and may not use the Internet except to learn if (and if so, how) (the example used) deals with this rumor on their Web site, another biased source that may not have the complete, honest story.” I agree. Google answers finds what is on the web, which can be useful but not necessarily accurate or comprehensive.

Search Engine for Handwritten Documents – University of Massachuestts

Andy Beal’s Search Engine Lowdown reports that Slashdot readers pointed to new search engine technology created by the University of Massachusetts, Amherst which is able to search handwritten documents.

The UMass article says: “Historians and researchers searching through handwritten documents, such as the 140,000 pages that make up George Washington’s personal papers in the Library of Congress, now have a new powerful tool to aid their work – a first-of-its kind manuscript retrieval system developed at the University of Massachusetts.” Documents are first scanned and the tool searches these scanned images. Seems amazing to me. I will have to dig up all my old handwritten journals from the 60s.

Jessica Baumgart looks at Google Definitions

Jessica, a librarian, takes a closer look at the Google definition function I posted on last week. She writes:

"What sticks out in my mind is what's missing from the results. It seems obvious to me that there should be links to pages in sources like online dictionaries, Wikipedia or other online encyclopedias, About.com pages, or some online reference sources. Why aren't they there? I thought I even read that Google has an agreement with Dictionary.com; and, if you search using "define term," a definition from Dictionary.com appears at the top of the results. For the RSS search, it seems like pointing to one of the standards would be a good choice, but it isn't there."

There is more in her post.

Google Defines Stuff

This is a week for Google features. John Maloney pointed me to Google defines. Here is the link to Google defines RSS as an example. It provides links to various definitions of RSS which actually stands for a number of things including: Repetitive stress syndrome, Regional Subscription System, Radio Science Subsystem, Rastriya Seva Sangh, Root Sum Square, Rural Stewardship Scheme, Remote Storage Service, as well as our blog related Really Simply Syndication. There is an explanation for each of the terms if you want to go there.

This service may have greater significance as I posted on a few months ago, Does Google Write the Next Dictionary? Google hits may be the latest statistical validation of language usage and a guide for the next generation of dictionary scholars.

I tried the Google defines for knowledge management. Most of the definitions were focused on document management and ignored the collaboration and access to experts and peers that many would include. Here is a typical one that completely misses the mark like more of the others, “The use of computer technology to organize, manage, and distribute electronically all types of information, customized to meet the needs of a wide variety of users.” Garbage in, garbage out.

Google Answers – Just what you wanted to know and more

John Maloney pointed me to Google Answers, an interesting service that you pay a bit for. They have recent questions and answers posted. I really liked the answer to Heathrow Airport (London) flight paths. I lived in central London a few years ago and watched to overhead flights and I have also come into Heathrow on many occasions. There are several routes which give you a nice aerial view of the city. Here is just the first part of the $15 answer.

“This is a simple map of London Heathrow Airport ‘LHR’ with the standard instrument departure routes marked out. You will note they all have various names and abbreviations. These will be referred to in the later sources.

This web site belongs to Garry Lewis, an Air Traffic Controller in London. This page and the following four, set out the structure of the United Kingdom airspace (maps included).

This page sets out in some detail, the standard routes into LHR and how air traffic is managed.”