A while ago I wrote a short blog post about a tweet from the Gartner analyst Ted Friedman saying that clients are disappointed with the ability to support wide deployment of complex business rules in popular data quality tools.
Speaking about popular data quality tools; on the DataFlux Community of Experts blog Founder of DataQualityPro Dylan Jones posted a piece this Friday asking: Are Your Data Quality Rules Complex Enough?
Dylan says: “Many people I speak to still rely primarily on basic data profiling as the backbone of their data quality efforts”.
The classic answers to the challenge of complex business rules are:
- Relying on people to enforce complex business rules. Unfortunately people are not as consistent in enforcing complex rules as computer programs are.
- Making less complex business rules. Unfortunately the complexity may be your competitive advantage.
In my eyes there is no doubt about that data quality tool vendors has a great opportunity in research and development of tools that are better at deploying complex business rules. In my current involvement in doing so we work with features as:
- Deployment as Service Oriented Architecture components. More on this topic here.
- Integrating multiple external sources. Further explained here.
- Combining the best algorithms. Example here.