Business processes

Happy Uniqueness

Henrik Gabs LiliendahlLeave a comment

When making the baseline for customer data in a new master data management hub you often involve heavy data matching in order to de-duplicate the current stock of customer master data, so you so to speak start with a cleansed duplicate free set of data.

I have been involved in such a process many times, and the result has never been free of duplicates. For two reasons:

  • Even with the best data matching tool and the best external reference data available you obviously can’t settle all real world alignments with the confidence needed and manual verification is costly and slowly.
  • In order to make data fit for the business purposes duplicates are required for a lot of good reasons.

Being able to store the full story from the result of the data matching efforts is what makes me, and the database, most happy.

The notion of a “golden record” is often not in fact a single record but a hierarchical structure that reflects both the real world entity as far as we can get and the instances of this real world entity in a form that are suitable for different business processes.

Some of the tricky constructions that exist in the real world and are usual suspects for multiple instances of the same real world entity are described in the blog posts:

The reasons for having business rules leading to multiple versions of the truth are discussed in the posts:

I’m looking forward to yet a party master data hub migration next week under the above conditions.

Bookmark and Share

Customer Relationship Mess (CRM)

Henrik Gabs LiliendahlLeave a comment

I have several times witnessed how a sales department for a lot of good reasons has forced the implementation of a CRM (Customer Relationship Management) software package disconnected from the ERP (Enterprise Resource Planning) system and other applications where customer master data have been handled until then.

The good reasons have been that the current applications didn’t fit the business processes in a dynamic sales department and perhaps that the current monolithic enterprise solution was too inflexible for the business needs in sales.

While this move may have been a great success in sales force automation the downside is often that the single customer view has been limited to a single customer view seen from the windows in the sales department offices.

In order to have a 360 degree view of customer you have to cover all the view points in the enterprise embracing all departments being in contact with the customer and thereby accessing and maintaining customer master data.

Those who feel the pain when a company doesn’t maintain such a view is the customer and those who enjoys when a company have that view is the customer.

Lately I had two experiences as a customer. A bad experience facing a lousy approximately 110 degree customer view from a phone company and a well executed 360 degree view from an insurance company. Both cases haven’t been around one of my favorite subjects being identity resolution. Both companies have my citizen ID.

It is just so that some companies cares more about single department business needs than true customer relationship management. IT’s a mess.

Bookmark and Share

Multi-Channel Data Quality

Henrik Gabs Liliendahl4 Comments

When I hear terms as multi-channel marketing, multi-channel retailing, multi-channel publishing and other multi-channel things I can’t resist thinking that there also must be a term called multi-channel data quality.

Indeed we are getting more and more channels where we do business. It stretches from the good old brick and mortar offline shop over eCommerce and the latest online touch points as mobile devices and social media.

Our data quality is challenged by how the way of the world changes. Customer master data is coming from these disparate channels with various purposes and in divergent formats. Product master data is exposed through these channels in different ways.     

We have to balance our business processes between having a unique single customer view and a unified product information basis and the diverse business needs within each channel.  

Some customer data may be complete and timely in one channel but deficient and out of date in another channel. Some product data may be useful here but inaccurate there.

I think the multi-channel things makes yet a business case for multi-domain (or multi-entity) master data management. Even if it is hard to predict the return on investment for the related data quality and master data management initiatives I think it is easy to foresee the consequences of doing nothing.

Bookmark and Share

Things Change

Henrik Gabs Liliendahl7 Comments

Yesterday I posted a small piece called So I’m not a Capricorn? about how astrology may (also) be completely wrong because something has changed.

On the serious side: Don’t expect that because you get it Right the First Time then everything will be just fine from this day forward. Things change.

The most known example in data quality prevention is probably that it is of course important that when you enter the address belonging to a customer, you get it right. But as people (and companies) relocates you must also have procedures in place tracking those movements by establishing an Ongoing Data Maintenance program in order to ensure the timeliness of your data.

The other thing, so to speak, is that having things right (the first time) is always seen in the context of what was right at that time. Maybe you always asked your customers for a physical postal address, but because your way of doing business has changed, you actually become much more interested in having the eMail address. And, because What’s in an eMail Address, you would actually like to have had all of them. So your completeness went from being just fine to being just awful by following the same procedure as last year.

Predicting accuracy is hard. Expect to deal with Unpredictable Inaccuracy.       

Bookmark and Share

Technology and Maturity

Henrik Gabs Liliendahl2 Comments

A recurring subject for me and many others is talking and writing about people, processes and technology including which one is most important, in what sequence they must be addressed and, which is my main concern, how they must be aligned.

As we practically always are referring to the three elements in the same order being people, processes and technology there is certainly an implicit sequence.

If we look at maturity models related to data quality we will recognize that order too.

In the low maturity levels people are the most important aspect and the subject that needs the first and most attention and people are the main enablers for starting moving up in levels.

Then in the middle levels processes are the main concerns as business process reengineering enables going up the levels.

At the top levels we see implemented technology as a main component in the description of being there.    

An example of the growing role of technology is (not surprisingly of course) in the data governance maturity model from the data quality tool vendor DataFlux.

One thing is sure though: You can’t move your organization from the low level to the high level by buying a lot of technology.

It is an evolutionary journey where the technology part comes naturally step by step by taking over more and more of the either trivial or extremely complex work done by people and where technology becomes an increasingly integrated and automated part of the business processes.

Bookmark and Share

Christmas at the old Bookstore

Henrik Gabs LiliendahlLeave a comment

Once upon a time (let’s say 15 years ago) there was a nice old bookstore on a lovely street in a pretty town. The bookstore was a good shopping place caring about their customers. The business had grown during the years. Neighboring shops have been bought and added to the premises along with the apartments above the original shop.

Also the number of employees had increased. The old business processes didn’t fit into the new reality so the wise old business owner launched a business process reengineering project in order to have the shop ready for a new record selling Christmas season. All the employees were more or less involved from brainstorming ideas to the final implementation. All suggestions were prioritized according to business value in supporting the way of doing business: Handing books over the fine old cash desk in the middle of the bookstore.

Even some new technology adoptions were considered during the process. But not too much. As the wise old business owner said again and again: Technology doesn’t sell books. Ho ho ho.

Unfortunately something terrible happened somewhere else. I don’t remember if it was on the other side of the street, on the other side of the river or on the other side of the ocean. But someone opened an internet bookstore. During the next years the market for selling books changed drastically due to orchestrating a business process based on new technology.

The wise old business owner at the nice old bookstore was choked. He actually had read the best management books on the shelf in the bookstore telling him to improve his business processes based on the way of doing business today; rely on changing the attitude of the good people working for him and then maybe use technology as an enabler in doing that. Ho ho ho.

Now, what about a happy ending? Oh yes. Actually some people like to buy some books on the internet and like to buy some other books in a nice old bookstore. Some other people like to buy most books in a nice old bookstore but may want to buy a few other books on the internet. So the wise old business owner went into multi-channel book selling. In order to keep track on who is buying what and where he used a state of the art data matching tool. Ho ho ho. Besides that he of course relied on the good people still working for him. Ho ho ho.

Bookmark and Share

Entity Revolution vs Entity Evolution

Henrik Gabs Liliendahl8 Comments

Entity resolution is the discipline of uniquely identifying your master data records, typically being those holding data about customers, products and locations. Entity resolution is closely related to the concept of a single version of the truth.

Questions to be asked during entity resolution are like these ones:

  • Is a given customer master data record representing a real world person or organization?
  • Is a person acting as a private customer and a small business owner going to be seen as the same?
  • Is a product coming from supplier A going to identified as the same as the same product coming from supplier B?
  • Is the geocode for the center of a parcel the same place as the geocode of where the parcel is bordering a public road?

We may come a long way in automating entity resolution by using advanced data matching and exploiting rich sources of external reference data and we may be able to handle the complex structures of the real world by using sophisticated hierarchy management and hereby make an entity revolution in our databases.

But I am often faced with the fact that most organizations don’t want an entity revolution. There are always plenty of good reasons why different frequent business processes don’t require full entity resolution and will only be complicated by having it (unless drastic reengineered). The tangible immediate negative business impact of an entity revolution trumps the softer positive improvement in business insight from such a revolution.

Therefore we are mostly making entity evolutions balancing the current business requirements with the distant ideal of a single version of the truth.

Bookmark and Share

Bilateral Master Data Management

Henrik Gabs Liliendahl4 Comments

There is an issue I have come over and over again when creating a master data hub, making a golden copy, establishing a single version of the truth or whatever we like the name to be. The issue is about the scope of data sources.

Basically you take (practically) all the master data sources from within your organization and consolidate these data. Often you match with external sources as business directories and so. But what you often miss is the master data operated by your partners. These are partners like:

  • Your suppliers of products, be that raw materials or finished products for resale
  • Your sales agents and distributors
  • Your service providers as direct marketing agencies and factoring partners

These partners are part of your business processes and they often create and consume master data which are only shared with you in a limited way via some form of interface.

I know that even handling master data from within most organizations is a complex issue. Integrating with external reference data doesn’t add simplicity. But without embracing the master data life at your partners, the hub isn’t complete; the copy is only made of plated gold and the single version of the truth isn’t the only truth.

My guess is that many master data programs in the future will extend to embrace internal (private) data, as well as external (public) data and bilateral data as described on the page about Data Quality 3.0.

Bookmark and Share

instant Data Quality

Henrik Gabs Liliendahl2 Comments

My last blog post was all about how data quality issues in most cases are being solved by doing data cleansing downstream in the data flow within an enterprise and the reasons for doing that.

However solving the issues upstream wherever possible is of course the better option. Therefore I am very optimistic about a project I’m involved in called instant Data Quality.

The project is about how we can help system users doing data entry by adding some easy to use technology that explores the cloud for relevant data related to the entry being done. Doing that has two main purposes:

  • Data entry becomes more effective. Less cumbersome investigation and fewer keystrokes.
  • Data quality is safeguarded by better real world alignment.

The combination of a more effective business process that also results in better data quality seems to be good – like a sugar-coated vitamin pill. By the way: The vitamin pill metaphor also serves well as vitamin pills should be supplemented by a healthy life style. It’s the same with data management.

Implementing improved data quality by better real world alignment may go beyond the usual goal for data quality being meeting the requirements for the intended purpose of use.  This means that you instantly are getting more by doing less.

Bookmark and Share

Top 5 Reasons for Downstream Cleansing

Henrik Gabs Liliendahl16 Comments

I guess every data and information quality professional agrees that when fighting bad data upstream prevention is better than downstream cleansing.

Nevertheless most work in fighting bad data quality is done as downstream cleansing and not at least the deployment of data quality tools is made downstream were tools outperforms manual work in heavy duty data profiling and data matching as explained in the post Data Quality Tools Revealed.

In my experience the top 5 reasons for doing downstream cleansing are:

1) Upstream prevention wasn’t done

This is an obvious one. At the time you decide to do something about bad data quality the right way by finding the root causes, improving business processes, affect people’s attitude, building a data quality firewall and all that jazz you have to do something about the bad data already in the databases.

2) New purposes show up

Data quality is said to be about data being fit for purpose and meeting the business requirements. But new purposes will show up and new requirements have to be met in an ever changing business environment.  Therefore you will have to deal with Unpredictable Inaccuracy.

3) Dealing with external born data

Upstream isn’t necessary in your company as data in many cases is entered Outside Your Jurisdiction.

4) A merger/acquisition strikes

When data from two organizations having had different requirements and data governance maturity is to be merged something has to be done.  Some of the challenges are explained in the post Merging Customer Master Data.

5) Migration happens

Moving data from an old system to a new system is a good chance to do something about poor data quality and start all over the right way and oftentimes you even can’t migrate some data without improving the data quality. You only have to figure out when to cleanse in data migration.

Bookmark and Share