펌:The BI-Search Evolution

source: http://www.information-management.com/newsletters/business_intelligence_bi_search-10017332-1.html?zkPrintable=true


The BI-Search Evolution

By David Caruso


MAR 10, 2010 5:31am ET


Several years ago, I was involved in a consulting engagement with the IT operations of a multibillion dollar firm. The focus was on the areas of budgets, ROI achievement, effective system use and user satisfaction. While the IT staff did a good job of managing budgets, they often struggled to deliver strategic business benefits to the users. One of their biggest problems was the inability to deliver timely information to executives and business users who needed to make ad hoc decisions. Consequently, expensive MBA types scrambled to extract data from business intelligence reports and spreadsheets in order to prepare analyses for the managers.

A recent Forrester Research survey noted that when it came to BI environments, 59 percent of the respondents indicated that users were unable to access 100 percent of the data needed for reporting and analytic work. Additionally, 78 percent of the respondents indicated that their BI environment did not enable exploration and analysis with features such as adaptive data models, unlimited dimensionality and guided analysis.



Information access and exploration has become more challenging as more people attempt to make more decisions based on more data. The people who need answers can’t find what they need – nor can they easily use what they found. Companies have been implementing and using information systems for decades, so what makes this so difficult and expensive? The answer is apparent when you look at the underlying source systems and the process of bringing the data from those sources together:



•Too many disparate data sources. New data sources are being introduced every day, and existing sources undergo constant changes. This complicates the task of unifying data and allowing for decision-making based on the most up-to-date information.

•Evolving user needs. The information needs of users change as rapidly and continually as the business needs evolve. Users also have expanded data universes, moving beyond just spreadsheets to include all company-wide and even Web-based data.

•Expensive and time-consuming data modeling. Often, users don’t know what information they need, so it’s difficult for them to articulate what data they require for decision-making. Because IT attempts to anticipate all the answers a user will ultimately want, this process often requires extensive data modeling in order to get the right answers.

•Power tools for the everyday user. Many analytic tools are intended for sophisticated power users, but these users are only a small fraction of the decision-making population in a company. Today, almost every business user is expected to make informed decisions.

Fortunately, easy-to-use search and the power of BI are finally merging, so IT can now deliver on the promise of providing users with all the data necessary to make strategic business decisions, and the power to discover, explore and analyze.



Taken separately, traditional BI and enterprise search tools were each designed to solve problems other than ad hoc decision-making. BI was originally developed for reporting on structured data while search was designed for retrieving unstructured documents. As a result, each technology fall shorts in different ways in several key areas – a good user experience, the types of accessible information and the ability to respond to rapid change.

Because of its focus on reporting and structured data, BI tools are good at answering predictable questions and reporting on key performance indicators. However, they aren’t as effective at answering new or ad hoc questions, requiring the user to request custom reports and cubes from IT analysts.

This is because the rigid, hierarchical data models in BI tools only allow limited exploration and are often complicated to use. Even with rigorous data modeling, most BI tools cannot access unstructured content, and adding new data sources requires analysis and redesign of the data models as well as the reports, analytics and dashboards they drive.

On the other hand, everyone uses search today because, in many ways, it has become the simplest form of computing. Googling someone or using search on an e-commerce site or even on a company home page is the starting point of many users’ regular computer use. However, basic search, with its incomplete data model and document-centric retrieval, also allows for only limited exploration, depriving users of necessary context. Structured data is an afterthought. And, while new data sources can be easily added, providing context and exposing relationships to existing data is difficult.

However, BI and search can be combined to preserve the strengths of both and mitigate the drawbacks of each.

Enabling Discovery

To understand how combining search and BI can bring a richer solution to bear on business decisions, we have to first consider how humans actually make an ad hoc decision.

In daily decision-making, people formulate their next question based on the answer to a previous question. In the process, people often need help formulating good questions because they want to understand what the alternatives might be when making trade-offs. As they gain insight into their problem, they can use additional filters, graphs and visualizations to drill down and explore deeper.

In the business world today, people typically rely on BI systems to get an answer to a known business problem and rely on search to find information. Unfortunately, answers to either structured BI queries or text searches are only as good as the question posed. Although users might glean insight from the results returned in either case, they might never know if the question asked was the right one. Only a few business questions are simple enough to be served with a "hole-in-one" answer.

The convergence of BI and search technologies can enable a user to expose relationships in data that can often lead to an unanticipated answer or new revelation – without the necessity of the perfectly formed question.



Unification of the Data

First, we must note that BI is based on a schema-driven model. That schema holds the key to what can be searched or navigated. But BI usually doesn’t accommodate complex data or unstructured content. In addition, in order to navigate the data effectively, applications must be created specifically for the query at hand.

Data-driven exploration and query refinement, allowing for search on both structured and unstructured data, is important when the data is heterogeneous and hard to understand for users – such as when unstructured content is being included in the search process. This flexibility allows IT to unify heterogeneous, changing data and content from multiple sources without the headaches and expense of traditional data modeling. Likewise, it enables IT to incorporate data from any format, structured or unstructured, and makes it possible to navigate across unstructured documents by automatically extracting structure from them.

Now, because the data is self-describing, it is able to build a dynamic data model; in effect, automating the data modeling task. This enables faceted search and navigation (or guided navigation) which allows the user to elaborate a query progressively, seeing the effect of each choice in the results set. The real power in this kind of search is that someone can expose new data relationships that help drive an unanticipated answer without having created the perfect query. For example, guided navigation enables users to find products or categories via attributes such as part numbers and commodity groups, size and weight. This allows users to enter a few keywords to locate information. The returned information is organized by category (or dimension) and then, using a graphical interface, users navigate the data and its relationships to locate their necessary answer.



Benefits for Users and IT

With search capabilities opening up the data and bringing a new level of ease of use to the analytic power of BI, new benefits include:


•Users will be able to execute data queries via a search box using natural language. Users can start a decision process with a simple search such as “machined parts” and be able to refine the data on a point-and-click basis. This helps users who don’t know how to structure queries find all data available to them, because the faceted search will organize the dimensions and open up the data for ad hoc interaction.

•A single point of entry for multiple BI systems, operational data stores and additional content. A data-driven model simplifies the data preparation and opens up the universe of data that can be explored. Users will be able to combine and explore data from any system, making federated queries against multiple BI systems and underlying source systems easy. This enables users to go to a single entry point to access all the data and content they need, no matter where it may have originated.

•Access to all the data, regardless of type. The explosion of information sources means businesses can make better decisions. Users want to access all relevant data without regard for whether it’s structured, unstructured, new or old. By unifying structured data and unstructured content, and making it all searchable, a search-enabled BI environment allows access to more data more quickly, with more flexibility and less modeling or integration overhead.