Complicated Matters

A while ago I wrote a short blog post about a tweet from the Gartner analyst Ted Friedman saying that clients are disappointed with the ability to support wide deployment of complex business rules in popular data quality tools.

Speaking about popular data quality tools; on the DataFlux Community of Experts blog Founder of DataQualityPro Dylan Jones posted a piece this Friday asking: Are Your Data Quality Rules Complex Enough?

Dylan says: “Many people I speak to still rely primarily on basic data profiling as the backbone of their data quality efforts”.

The classic answers to the challenge of complex business rules are:

  • Relying on people to enforce complex business rules. Unfortunately people are not as consistent in enforcing complex rules as computer programs are.
  • Making less complex business rules. Unfortunately the complexity may be your competitive advantage.

In my eyes there is no doubt about that data quality tool vendors has a great opportunity in research and development of tools that are better at deploying complex business rules. In my current involvement in doing so we work with features as:

  • Deployment as Service Oriented Architecture components. More on this topic here.
  • Integrating multiple external sources. Further explained here.
  • Combining the best algorithms. Example here.

Bookmark and Share

2 thoughts on “Complicated Matters

  1. Dylan Jones 6th September 2010 / 10:11

    You raise a good point here Henrik, the reason most rules are not managed is because they are an absolute nightmare!

    There is also an issue that business rules (if they exist in some electronic workflow system for example) are rarely reflected in the data quality rules that should align to them.

    For example, a business rule may state that a customer who already has an existing account must be linked to a parent record, a business rule may be enforced to perform basic checks via workflow software but unless the workflow software has the power of modern matching technology it may still fail.

    What we need is a unification of business and data quality rules so that there is no distinction.

    • Henrik Liliendahl Sørensen 6th September 2010 / 10:30

      Thanks for commenting Dylan. Indeed we need a unification of business and data quality rules.

      In relation to the data quality tool market I think we have two trends for that:
      • Mega vendors embracing all disciplines
      • Specialized vendors making best-of-breed plug-ins

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s