DATA WAREHOUSES MAKE YOUR DATA PROFITABLE AND USEFUL
Companies have loads of data - for loads of reasons. Imagine tapping into all that data and making it work for you. Data warehouses let you.
Imaging converts paper documents and images into electronic and digital forms. A data warehouse turns that data into useful information.
Often data warehouses are confused with data marts or stores. Data marts and stores usually serve smaller environments like departments or single servers. You use data warehouses in enterprises. Data marts and stores take data and let you access it. This data is usually from one source if you're dealing with a single department.
Data warehouses extract data from many sources / departments in a company and let you perform complex analyses.
The true benefit of a data warehouse is it lets you look at data you might not normally associate together. Simultaneously look at information from an invoice and data found in customer service records. This might let you know that a customer bought lots of products and which ones gave them trouble. Use that information to better serve your customers and evaluate the products you sell.
The key is that once the data is in the warehouse you can access this information in real time. You can let anyone in your company access the data to or set up security parameters. The information is usually available through an OLAP (Online Analytical Processing) server.
This runs like a regular database -- only all of the data is clean. There are no corrupt fields, duplicate records or records with multiple names of the same company. OLAP servers sometimes have terabytes of data. They include all aspects of data and are constantly growing.
Now that you know what a data warehouse is, here are five things you should know about how they work and what they do for you:
How do I implement a data warehouse?
A data warehouse isn't just one product. It's a combination of hardware and software products. It combines data from multiple servers or just one -- this depends on your environment.
One of the first things you need is a road map. Data warehouses are something that you continue to use. You need a solution that grows. Don't base the road map on one vendor's products.
A data warehouse is a combination of solutions -- even though one company may dominate your solution, don't bet all your data on it. There are no guarantees that a company will be around as long as yours. Technology changes daily. Don't lock yourself into a single technology that might not last.
The road map should have milestones and guide posts to help you judge how effective your solution is. Include in the road map the sequences of data warehouse activities. This encompasses details like where the data comes from first and when. The road map tells you where to start. It's usually where you need it most. It also provides a way to tailor the data warehouse to your unexpected needs.
There are companies that help you do this. Actium (Conshohocken, PA 610-832-1000) is a service provider that works with you every step of the way to implement the data warehouse. They work with many data warehouse technology vendors. The benefit of going this route is they have experience with enough products to help find the best one for you.
Look at the open system vendor Siemens Pyramid (San Jose, CA 408-428-9000). They provide open mainframe-class type solutions that are a mix of products from a multitude of vendors. They offer scalable solutions and can start you off one department at a time.
Building a data warehouse is simple in theory. It goes like this:
1. Extract data from all the operational systems in the enterprise. This includes shipping, accounting, marketing and others. Each department probably has its own server. The goal is to bring all the information together into one place. Data arranged differently in each system isn't conducive to decision making.
2. Clean and integrate the data from multiple sources. You need to strip data of all the information associated with it, like pre-defined fields. If it's customer information, you want to have consistent spelling of customer names and information. The goal is to make it clean, raw data. Clean data is void of corrupted fields and inconsistent naming. This data is for complex analysis and must be accurate.
3. Bring all of your clean accurate data into the data warehouse.
4. Provide users access to the data in the warehouse. This is the pot of gold at the end of the day. There are tools that let you do this. The most commonly used tool is an OLAP server specified to meet your company's needs. Set them up for specific applications, jobs, job levels and expertise. It's your call. The vendor you work with to implement your warehouse should be able to give you exactly what you want.
Cognos (Burlington, MA 617-229-6600) has two products that help you do queries and analysis. The multi-dimensional analysis tool PowerPlay lets you analyze and understand data over a period of time. This helps you find trends. It works with many different servers or lets you create your own.
Lots of companies offer software packages that claim to do everything. Oracle (Redwood Shores, CA 415-506-7000) has a slew of products that take care of the data warehouse from start to finish.
Letting a company do it for you is another option. Data warehouse service bureaus like Acxiom (Conway, AR 501-336-1000) do this. They take your data on any media form and clean and process it. Then they consolidate it for you. They'll return the data on any media you request. If you want, they'll put the clean data in an Oracle database. This eliminates duplicate data in a new database.
Acxiom will manage your data for you or put it on an open system at your site. They provide a dedicated account team to each customer to help you with implementation.
How long should it take to implement a data warehouse?
This is debatable. There are many factors to take into account. Predicting how long it takes before you get a full functioning data warehouse is a challenge.
Factors to consider:
Data. How much do you have? If you only have a few gigabytes it might not take long. Chances are you might not be implementing a data warehouse either. You might only be putting in a data mart.
The more data you have the longer it will take.
Vendor reliability. Some companies promise a fully implemented data warehouse in three months. It can be done, but look at their track record. Find out what they promised other customers in similar situations. Check to see if they made good on those promises?
What type of information do I want in my data warehouse?
By nature we are recorders. We record everything. We just don't always record it in the same way each time or the same place. So we have lots of records in lots of places. A data warehouse is your chance to make your computers truly work for you. Just give it all the data you've been recording. Let the data warehouse do the work and spit out useful information.
Make a top 10 list of questions you can't answer now. The answers will help you improve your business and the way you do it. This will help you define the type of data you should put in the warehouse.
These questions don't have to be product specific. They can be generic about every product you make or sell. They can address customer service issues like: "I want to anticipate my customers needs before they do. I want to be the first one to provide them a solution. When will they need it?" You'll see that you already have all the information needed. You just can't get to it quickly.
Logic Works (Princeton, NJ 609-514-1177) has products that help you get started. Irwin 2.6 is a tool that makes sure you're getting the right data into the warehouse. It has a new feature "Complete Compare." This lets you compare a physical model of your data warehouse in production to a representative model.
It goes through both models. Compares them and lets you know what's missing. It's imperative that you get the right data into the warehouse. If you don't, when you extract data for analysis you're not going to get an accurate report. Then the warehouse is useless.
An exploding source of data is the Web. Use Acrue Software's (Mountain View, CA 415-969-9031) Red Brick for Web site analysis. This is great if you have a large Web site. People log onto Web sites constantly. Some sites get thousands of hits a day.
The way people surf through your Web site provides you with valuable data. It identifies the information your potential customers want so you can market directly to them. It also tells you which pages people like and don't. It defines frequently visited pages of your site. It lets you know if people are visiting that page because you're giving something away or because it has valuable information.
Information Advantage (Minneapolis, MN 612-820-0702) also has a Web product -- WebOLAP. It lets companies put their data warehouses on the Internet or onto corporate intranets. Tap into it and perform relational OLAP processing of the database content using a standard web browser like Netscape. WebOLAP turns the content into interactive HTML documents.
What do data warehouses
do for you?
Data warehouses help you enhance and promote your business. You build them for a return on investment. The initial benefit of a data warehouse isn't to save you money, it's to increase your profits. That's what differentiates it from most storage solutions.
Use them to better automate the solutions and practices you're already using. Use data warehouses to break down every piece of information that goes into your organization and where it originates. You might find a lot of extra steps you don't need. That saves you time and money in the long haul.
What are businesses getting from data warehouses today?
Financial Groups. They use data warehouses to try and understand the risks associated with customer portfolios in terms of profits. The ability to look at all accounts and information associated with an individual lets financial institutions, like banks, truly understand and service their customers better.
Information from a data warehouse helps banks predict customer needs even before the customer realizes that need. For example, they recognize when a customer has been making a car payment for four years. After four years, many people think of buying a new car. The bank can anticipate this, and approach the customer about an auto loan for a new car.
NCR (Dayton, OH 513-445-5000) provides complete data warehouse solutions for retail, financial and communication industries. They set customers up with the hardware and software needed to create and implement a data warehouse.
Manufacturers. They use information to understand end users' product preferences. Do they prefer knobs or buttons? How often do they buy the particular product you make?
Promotional Marketing. In sales, the statistics tend to dictate you'll get one hit for every set of 10 calls you make. Information from data warehouses used can dramatically increase those odds. Customer analysis helps you identify other opportunities to sell them different products.
Another promotional marketing opportunity is direct mail. How many times has a catalog arrived in your mailbox that you'd never order from in a million years? That's a waste of money and time. Both for you and the company that sent it to you. This catalog would yield better profits if it went to a more focused audience. A data warehouse helps you do that.
Airlines. They use the information extracted from a data warehouse to judge the capacity and frequency of flights. It helps them schedule the most profitable routes at specified times of the day.
Decision Makers. It provides them with sophisticated analysis quickly. They use this to support business decisions that give a company a competitive edge.
Your Sales Force. They can access a data warehouse to find prospects in a certain area. This is more than just demographics. Depending on the type of data they search for, they can predict buying habits and times when prospective customers buy that type of product or service.
Insurance Companies. They're always looking for new ways to detect fraud. Data warehouses help them put together even more pieces of the fraudulent puzzle.
Customer Service. These people deal with time critical applications. They have someone on the phone demanding to know something immediately. A data warehouse provides information about the customer on the phone that could enhance the way you serve them.
Network Imaging (Hernon, VA 703-478-2260) has done a lot of work with data warehousing applications that are geared toward customer service applications. They focus on providing detailed information rather than statistics. Time is critical for their customers. Increased information increases their sales revenue. That's important.
