Metadata

Cross Border Product Data Flows

Henrik Gabs LiliendahlLeave a comment

The below figure shows the cross border data flows on this planet. There are inter-regional data flows and there are flows between the worldwide regions:

cross-boarder-data-flows

Now, a small part of this data will be product data exchanged between trading partners participating in global business ecosystems. While I have no data on if product data are distributed by the same proportions as data in general, it will be a qualified guess, that the picture will look somewhat the same.

Exchanging product data across borders has some challenges:

  • Language is an issue. Product data will eventually have to be translated into the language of the end buyer, if this is not the language in which the product data originally are provided. The definitions (metadata) of product data will also be subject to translation. Even the language of the transmission tools would not be in English all over.
  • Regulations around product data are different from country to country.
  • The cultural content of the optimal data describing a product in structured data elements and related digital assets are different between countries and regions.

At Product Data Lake, we are, from the center of the largest green bubble, looking for ambassadors around the world who are able to link the in-house product information management solutions at trading partners.

Interested? Get in contact:

Three Must Haves for your Data Lake

Henrik Gabs LiliendahlLeave a comment

Whether the data lake concept is a good idea or not is discussed very intensively in the data management social media community.

LakeThe fear, and actual observations made, is that that a data lake will become a data dump. No one knows what is in there, where it came from, who is going to clean up the mess and eventually have a grip on how it should be handled in the future – if there is a future for the data lake concept.

Please folks. We have some concepts from the small data world that we must apply. Here are three of the important ones:

Metadata

In short, metadata is data about data. Even though the great thing about a data lake is that the structure and all purposes of the data does not have to be cut in stone beforehand, at least all data that is delivered to a data lake must be described. An example of such an implementation is examined in the post Sharing Metadata.

Data Lineage

You must also have the means to tag who delivered the data. If your data lake is within a business ecosystem, this should include the legal entity that has provided the data as told in the post Using a Business Entity Identifier from Day One.

Data Governance

Above all, you must have a framework to govern ownership (Responsibility, Accountability, Consultancy and who must be Informed), policies and standards and other stuff we know from a data governance framework. If the data lake expand across organizations by incorporating second party and third party data, we need a cross company data governance framework as for example highlighted on Product Data Lake Documentation and Data Governance.

Visions for a Global PIM

Henrik Gabs LiliendahlLeave a comment

Yesterday Gautam Sood of Riversand blogged about that One Size doesn’t fit all – The Complexities of a Global PIM.

In this blog post Gautam examines the challenges, the key questions and the concept options an organization have when embarking on a journey to go from a national (or regional) scale to an international scale in Product Information Management.

A recent blog post here on the blog also had that theme for the Master Data Management (MDM) realm.  This post is about Cross Border Master Data Management.

In his post Gautam states: “A Global PIM is not a consolidation exercise. Variance is the reality, and it has to be supported.”

globalThis resonates very well with my findings. Very low practical this means that you will not win by translating all product descriptions into English. Even the metadata has to be multilingual, as you will interact with trading partners using different languages. While one public standard for product information may be king in one region, this will most likely not be the case in another region, which again effects how you collaborate with trading partners in different geographies.

In my eyes the global PIM journey does not end with consensus and a common platform of either concept inside your organization. You have to embrace your business ecosystem of trading partners. How to do that is explained in the post What a PIM-2-PIM Solution Looks Like.

What a PIM-2-PIM Solution Looks Like

Henrik Gabs LiliendahlLeave a comment

The importance of having a viable Product Information Management (PIM) solution has become well understood for companies who participates in supply chains.

The next step towards excellence in PIM is to handle product information exchange (product data syndication) in close collaboration with your trading partners. Product Data Lake is the solution for that. Here upstream providers of product information (manufacturers and upstream distributors) and downstream receivers of product information (downstream distributors and retailers) connect their choice of in-house PIM solution or other product master data solution as PLM (Product Lifecycle Management) or ERP.

The PIM-2-PIM solution resembles a social network where you request and accept partnerships with your trading partners from the real world.

pdl-how-1

After connecting the next to set up is how your product attributes and digital asset types links with the one used by your trading partner. In Product Data Lake we encompass the use of these different scenarios (in prioritized order):

  • You and your trading partner uses the same standard in the same version
  • You and your trading partners uses the same standard in different versions
  • You and your trading partner uses different standards
  • You and/or your trading partners don’t use a public standard

Read more about that and the needed data governance in the post Approaches to Sharing Product Information in Business Ecosystems.

pdl-how-2

Then it is time to link your common products. This can be done automatically if you both use a GTIN (or the older implementations as EAN number or UPC) as explained in the post Connecting Product Information. Alternatively, model numbers can be used for matching or, as a last option, the linking can be done in the interactive user interface.

pdl-how-3

Now you and your trading partner are set to start automating the process of sharing product information. In Product Data Lake upstream providers of product information can push new products, attribute values and digital assets from the in-house PIM solution to a hot folder, where from the information is uploaded by Product Data Lake. Downstream receivers can set up pull requests, where the linked product information is downloaded, so it is ready to be consumed by the in-house PIM solution.

pdl-how-4

This process can now be repeated with all your other trading partners, where you reuse the elements that are common between trading partners and build new linking where required.

pdl-how-5

If you have any questions, please contact me here:

Bookmark and Share

Sharing Metadata

Henrik Gabs LiliendahlLeave a comment

In short, metadata is data about data. Handling metadata is an important facet of data management including in data governance, data quality management and Master Data Management (MDM). When it comes to the new trends in data management as big data and handling data in data lakes, the importance of metadata management will in my eyes become even more obvious.

In a current venture (Product Data Lake) we are working on building in metadata management for business ecosystems, meaning that trading partners can share product information either using the same metadata or linking their different metadata.

Using international, national and industry standards for product information will be the perfect solution within business ecosystem sharing of metadata and indeed this is the preferred option we support. However, there are many competing standards for product information and they come in developing versions, so having everyone on the same page at the same time is quite utopic.

Add to that everyone do not speak English – and even not the same variant of English. Metadata originates and should exist in the languages that is used in trading partnerships.

pdl-menu-manage-attributeIn Product Data Lake we have started out with these principles:

  • Product attributes can be tagged with an attribute type telling about what standard (if any) in terms of product identification, product classification or product feature it adheres to. More about that in the post Connecting Product Information.
  • Attribute short and long descriptions can be represented in different languages.
  • Trading partners can link their product attributes and have visibility in the Product Data Lake of the standards and descriptions used in the different languages they exist.

I will very much welcome your input to this quest and if you want to be involved please do not hesitate to be in touch with me here or on Xing, Viadeo or LinkedIn.

Bookmark and Share

From Business Glossary to Full-Blown Metadata Management or Vice Versa

Henrik Gabs Liliendahl3 Comments

A very common starting point for producing tangible outcomes in a data governance programme is setting up a business glossary. The alternatives, or next/previous steps, for a business glossary were discussed in the post Metadata Musings by a Nerd.

First, in my eyes a business glossary (or whatever you call such a thing) is indeed a useful deliverable in its own right. In order to support a data governance programme you will need to add things besides definition of terms. One important element is documenting business rules as reported by Nikki Rogers at The University of Bristol here.

metadata

A business glossary should ultimately morph into full-blown metadata management or meet data dictionary and/or metadata repository initiatives that also may grow in your organization. What full-blown metadata management means was touched recently by Brian Brewer in a blog post called Gartner Says More Metadata. This post cites a blog post by Darin Stewart of Gartner (the analyst firm). The Gartner post is called Big Content Needs More Metadata.

How did you develop your business glossary?

  • Did you start with a business glossary and then morphed into the data dictionary and metadata management discipline and lingo?
  • Did the business glossary grow from the metadata management work?
  • Is the business glossary just sitting there doing what it does?

Bookmark and Share

Metadata Musings by a Nerd

Henrik Gabs Liliendahl2 Comments

We all know the problem: We use the same term, but means two different things. Or: We use two different terms, but actually mean the same thing.

Within data management this is a huge challenge. The solution is ….. Well, there are different terms:

Business Glossary is one term. The term is explained in an artcle on B-eye-Network by Lowell Fryman here. Using the term business glossary implies that you have a business approach to the issue. Implementing a business glossary is often mentioned as a part of a data governance framework.

Data Dictionary is explained on Wikipedia here. Using the term data dictionary implies that you have a technical approach to the issue. Having a data dictionary is sometimes mentioned as a part of a Master Data Management (MDM) solution.

Metadata Repository is also explained on Wikipedia here. Using the term metadata repository implies that you have a somewhat nerdish approach to the subject as seen in the post Metadata Meatballs. Addressing metadata is often stated as an important subject within the data quality discipline as shown in the post Perfect Wrong Answer.

Metadata terms

Bookmark and Share

Data being Defective by Design

Henrik Gabs Liliendahl4 Comments

In a blog post yesterday on the Melissa Data blog Elliot King wrote about Classifying Data Quality Problems. The post suggests that there are three different kinds of data quality issues:

  1. Operational
  2. Conceptual
  3. Organizational

This classification revolves around the root cause of bad data.

As examined in my post yesterday sometimes bad data quality isn’t bad data. A good deal of problems doesn’t relate to the raw data itself, but is linked to how data are structured,for example in data models, and how data are categorized, for example by (not) using metadata.

Flaws in data structure seem to have similar root causes as the suggested categorization, for example:

  1. Operational: Data are structured and labeled to fit capturing systems which may not fit further downstream purposes of use.
  2. Conceptual: The term conceptual data models (or similar approaches) pop up here. We miss them, not at least the enterprise-wide ones, very much in IT landscapes made up by popular off-the-shelf software.
  3. Organizational: We are usually not very well in talking the same language about the same data.

Frisendal bookBy the way: One good book about overcoming these challenges I read recently is by Thomas Frisendal and is called Design Thinking Business Analysis.

Bookmark and Share

When Bad Data Quality isn’t Bad Data

Henrik Gabs Liliendahl7 Comments

PopeThere has been a quiz running on this blog with the question: What is the name of the current Pope of the Catholic Church?. Find the current standing of answers in the figure to the right.

It’s good to see a lot of different answers and indeed, a problem with the quiz is that all answers may be correct. While Francis is the name as pope in English chosen by Jorge Mario Bergoglio, the pope has other names in other languages as Frans in Danish and Norwegian, François in French, Franziskus in German and Francesco in Italian.

The quiz is actually bad as it has not included other good answers as Franciscus, the latin name, Francisco, the Spanish name, and Franciszek, the Polish name. The question in the quiz is too simple. What is meant by “the name” should be clarified: Is it the birth name, the chosen name as Pope in a given language or what?

Such problems are in fact very common related to what we often see as bad data quality, as it reflects two frequent issues which aren’t about the raw data:

  • Data models are too simple. In this case we could be able to reflect different types of names: Birth name and what (sorry, believers) resembles a screen name. And names in various languages.
  • Metadata is too weak. In this case it could be more precise what name we are collecting, if it is only one of the name types we need, for example chosen name in English. More about metadata on Wikipedia.

What other issues have you encountered seen as bad data quality, but which isn’t bad raw data?

Bookmark and Share

Data Accessibility

Henrik Gabs Liliendahl2 Comments

When solving data quality issues we are usually dealing with important data quality dimensions as uniqueness, completeness, timeliness, accuracy, consistency and integrity.

One other data quality dimension I have been addressing lately is accessibility.

Data accessibility is a universal feature within information technology as recently emphasized in the big story about that the Spanish bank BBVA is to move +100,000 employees to the cloud (using Google Apps). The main driver for that is data accessibility.

The increasing adoption of cloud services will in my eyes contribute positively to solving many data quality issues through improved accessibility as discussed in the post Data Quality from the Cloud.

The fact that data is available and in principle accessible doesn’t however mean that the case is solved. The Achilles Heel is how to smoothly integrate accessible data into business processes, not at least how to integrate and present data from many different sources be that external and internal sources.

Data accessibility must be seen in conjunction with the other data quality dimensions. Fulfilling one dimension doesn’t make the day. Accessibility to data that isn’t satisfactory unique, complete, timely and accurate isn’t that much worth. Making data consistent across multiple sources isn’t a walkover. Securing data integrity between more and more accessible data will be paramount.

Metadata management is also closely related to data accessibility. The importance of a common understanding about what is in the accessible data can’t be overestimated.

My guess is that we will see data accessibility as an increasingly important data quality dimension in the years to follow.

Bookmark and Share