Product Data Syndication

Three Essential Trends in Data Management for 2024

Henrik Gabs LiliendahlLeave a comment

On the edge of the New Year, it is time to guess what will be the hot topics in data management next year. My top three candidates are:

  • Continued Enablement of Augmented Data Management
  • Embracing Data Ecosystems
  • Data Management and ESG

Continued Enablement of Augmented Data Management

The term augmented data management is still a hyped topic in the data management world. “Augmented” is here used to describe an extension of the capabilities that is now available for doing data management with these characteristics:

  • Inclusion of Machine Learning (ML) and Artificial Intelligence (AI) methodology and technology to handle data management challenges that until now have been poorly solved using traditional methodology and technology.
  • Encompassing graph approaches and technology to scale and widen data management coverage towards data that is less structured and have more variation than data that until now has been formally managed as an asset.
  • Aiming at automating data management tasks that until now have been solved in manual ways or simply not been solved at all due to the size and complexity of the work involved.

It is worth noticing that the Artificial Intelligence theme lately has been dominated by generative AI and namely ChatGPT. However, for data management generative AI will in my eyes not be the most frequently used AI flavor. Learn more about data management and AI in the post Three Augmented Data Management Flavors.

Embracing Data Ecosystems

The strength of data ecosystems was latest examined here on the blog in the post From Platforms to Ecosystems.

Data ecosystems include:

  • The infrastructure that connects ecosystem participants and help organizations transform from local and linear ways of doing business toward virtual and exponential operations.
  • A single source of truth for ecosystem participants that becomes a single source of truth across business partner ecosystems by providing all ecosystem participants with access to the same data.
  • Business model and process transformation across industries to support agile reconfiguration of business models and processes through information exchange inside and between ecosystems.

In short, your organization cannot grow faster than your competitors by hiding all data behind your firewall. You must share relevant data within your business ecosystem in an effective manner.

Data Management and ESG

ESG stands for Environmental, Social and Governance. This is often called sustainability. In a business context, sustainability is about how your products and services contribute to sustainable development.

When working as a data management consultant I have seen more and more companies having ESG on top of the agenda and therefore embarking on programs to infuse ESG concepts into data management. If you can tie a proposed data management effort to ESG, you have a good chance of getting that effort approved and funded.

Capturing ESG data is very much about sharing data with your business partners. This includes getting new product data elements from upstream trading partners and providing such data to downstream trading partners. These new data elements are often not covered through traditional ways of exchanging product data. Getting the traditional product information through data supply chains is already challenged so adding the new ESG dimension is a daunting task for many organizations.

Therefore, we are ramping up to also cover ESG data in the collaborative product data syndication service I am involved in and is called Product Data Lake.

From Platforms to Ecosystems

Henrik Gabs Liliendahl2 Comments

Earlier this year Gartner published a report with the title Top Trends in Data and Analytics, 2023. The report is currently available on the Parsionate site here.

The report names three opportunities within this theme:

  • Think Like a Business,
  • From Platforms to Ecosystems and
  • Don’t Forget the Humans

While thinking like a business and don’t forget the humans are universal opportunities that have always been here and will always be, the move from platforms to ecosystems is a current opportunity worth a closer look.

Here data sharing, according to Gartner, is essential. Some recommended actions are to

  • Consider adopting data fabric design to enable a single architecture for data sharing across heterogeneous internal and external data sources.
  • Brand data reusability and resharing as a positive for business value, including ESG (Environmental, Social and Governance) efforts.

Data Fabric is the Gartner buzzword that resembles the competing non-Gartner buzzword Data Mesh. According to Gartner, organizations use data fabrics to capture data assets, infer new relationships in datasets and automate actions on data.

Data sharing can be internal and external.

In my mind there are two pillars in internal data sharing:

  • MDM (Master Data Management) with the aim of sharing harmonized core data assets as for example business partner records and product records across multiple lines of business, geographies, and organizational disciplines.
  • Knowledge graph approaches where MDM is supplemented by modern capabilities in detecting many more relationships (and entities) than before as explained in the post MDM and Knowledge Graph.

In external data sharing we see solutions for:

External data sharing is on the rise, however at a much slower pace than I had anticipated. The main obstacle seems to be that the internal data sharing is still not mature in many organizations and that external data sharing require interaction between at least two data mature organizations.

How MDM Inflates

Henrik Gabs LiliendahlLeave a comment

Since the emerge of Master Data Management (MDM) back in 00’s this discipline has taken on more and more parts of the also evolving data management space.

The past

It started with Customer Data Integration (CDI) being addressing the common problem among many enterprises of having multiple data stores for customer master data leading to providing an inconsistent face to the customer and lack of oversight of customer interactions and insights.

In parallel a similar topic for product master data was addressed by Product Information Management (PIM). Along with the pains of having multiple data stores for product data the rise of ecommerce lead to a demand for handling much more detailed product data in structural way than before.

While PIM still exist as an adjacent discipline to MDM, CDI mutated into customer MDM covering more aspects than the pure integration and consolidation of customer master data as for example data enrichment, data stewardship and workflows. PIM has thrived either within, besides – or without – product MDM while supplier MDM also emerged as the third main master data domain.   

The present

Today many organizations – and the solution providers – either grow their MDM capabilities into a multidomain MDM concept or start the MDM journey with a multidomain MDM approach. Multidomain Master Data Management is usually perceived as the union of Customer MDM, Supplier MDM and Product MDM. It is. And it is much more than that as explained in the post What is Multidomain MDM?

As part of a cross-domain thinking some organizations – and solution providers – are already preparing for the inevitable business partner domain as pondered in the post The Intersection of Supplier MDM and Customer MDM.

The PIM discipline has got a subdiscipline called Product Data Syndication (PDS). While PIM basically is about how to collect, enrich, store, and publish product information within a given organization, PDS is about how to share product information between manufacturers, merchants, and marketplaces.

The future

Interenterprise MDM will be the inflated next stage of the business partner MDM and Product Data Syndication (PDS) theme. This is about how organizations can collaborate by sharing master data with business partners in order to optimize own master data and create new data driven revenue models together with business partners.

It is in my eyes one of the most promising trends in the MDM world. However, it is not going to happen tomorrow. The quest of breaking down internal data and knowledge silos within organizations around is still not completed in most enterprises. Nevertheless, there is a huge business opportunity to pursue for the enterprises who will be in the first wave of interenterprise data sharing through interenterprise MDM.

Extended MDM is the inflated next scope of taking other data domains than customer, supplier, and product under the MDM umbrella.

Reference Data Management (RDM) is increasingly covered by or adjacent to MDM solutions.

Also, we will see handling of locations, assets, business essential objects and other digital twins being much more intensive within the MDM discipline. Which entities that will be is industry specific. Examples from retail are warehouses, stores, and the equipment within those. Examples from pharma are own and affiliated plants, hospitals and other served medical facilities. Examples from manufacturing are plants, warehouses as well as the products, equipment and facilities where their produced products are used within.

The handling of all these kind of master data on the radar of a given organization will require interenterprise MDM collaboration with the involved business partners.

Organizations who succeed in extending the coverage of MDM approaches will be on the forefront in digital transformation.

Augmented MDM is the inflated next level of capabilities utilized in MDM as touched in the post The Gartner MDM MQ of December 2021 and Augmented MDM. It is a compilation of utilizing several trending technologies as Machine Learning (ML), Artificial Intelligence (AI), graph approaches as knowledge graph with the aim of automating MDM related processes.

Metadata management will play a wider and more essential role here not at least when augmented MDM and extended MDM is combined.    

Mastering this will play a crucial role in the future ability to launch competitive new digital services.

What is Collaborative Product Data Syndication?

Henrik Gabs Liliendahl1 Comment

Product Data Syndication (PDS) is a sub discipline within Product Information Management (PIM) as explained in the post What is Product Data Syndication (PDS)?

Collaborative PDS can be achieved at scale with a specialized product data syndication service where the manufacturer can push product information according to their definitions and the merchant can pull linked and transformed product information according to their definitions.

With Collaborative Product Data Syndication, you can get the best of two worlds:

  • You can have the market standard that makes you not falling behind your competitors.
  • However, you can also have unique content coming through that puts you ahead of your competitors.

The advantages of collaborative PDS versus other PDS approaches was examined in the post Collaborative Product Data Syndication vs Data Pools and Marketplaces.

The Product Data Lake solution I am involved with utilizes that data lake concept to handle the complexities of having many different data standards for product information in play within supply chains and encompass the many different preferences for exchange methods.

Our approach is not to reinvent the wheel, but to collaborate with partners in the industry. This includes:
·       Experts within a type of product as building materials and sub-sectors in this industry, machinery, chemicals, automotive, furniture and home-ware, electronics, work clothes, fashion, books and other printed materials, food and beverage, pharmaceuticals and medical devices. You may be a specialist in certain standards for product data. You will link the taxonomy in use at two trading partners or within a larger business ecosystem.
·       Product data cleansing specialists who have proven track records in optimizing product master data and product information. You will prepare the product data portfolio at a trading partner and extend the service to other trading partners or within a larger business ecosystem.
·       System integrators who can integrate product data syndication flows into Product Information Management (PIM) and other solutions at trading partners and consult on the surrounding data quality and data governance issues. You will enable the digital flow of product information between two trading partners or within a larger business ecosystem.
·       Tool vendors who can offer in-house Product Information Management (PIM) / Master Data Management (MDM) solutions or similar solutions in the ERP and Supply Chain Management (SCM) sphere. You will be able to provide, supplement or replace customer data portals at manufacturers and supplier data portals at merchants and thus offer truly automated and interactive product data syndication functionality.
·       Technology providers with data governance solutions, data quality management solutions and Artificial Intelligence (AI) / machine learning capacities for classifying and linking product information to support the activities made by other delegates and subscribers.
·       Reservoirs, as Product Data Lake is a unique opportunity for service providers with product data portfolios (data pools and data portals) for utilizing modern data management technology and offer a comprehensive way of collecting and distributing product data within the business processes used by subscribers.

What is Product Data Syndication (PDS)?

Henrik Gabs LiliendahlLeave a comment

Product Information Management (PIM) has a sub discipline called Product Data Syndication (PDS).

While PIM basically is about how to collect, enrich, store and publish product information within a given organization, PDS is about how to share product information between manufacturers, merchants and marketplaces.

Marketplaces

Marketplaces is the new kid on the block in this world. Amazon and Alibaba are the most known ones, however there are plenty of them internationally, within given product groups and nationally. Merchants can provide product information related to the goods they are selling on a marketplace. A disruptive force in the supply (or value) chain world is that today manufacturers can sell their goods directly on marketplaces and thereby leave out the merchants. It is though still only a fraction of trade that has been diverted this way.

Each marketplace has their requirements for how product information should be uploaded encompassing what data elements that are needed, the requested taxonomy and data standards as well as the data syndication method.

Data Pools

One way of syndicating (or synchronizing) data from manufacturers to merchants is going through a data pool. The most known one is the Global Data Synchronization Network (GDSN) operated by GS1 through data pool vendors, where 1WorldSync is the dominant one. In here trading partners are following the same classification, taxonomy and structure for a group of products (typically food and beverage) and their most common attributes in use in a given geography.

There are plenty of other data pools available emphasizing on given product groups either internationally or nationally. The concept here is also that everyone will use the same taxonomy and have the same structure and range of data elements available.

Data Standards

Product classifications can be used to apply the same data standards. GS1 has a product classification called GPC. Some marketplaces use the UNSPSC classification provided by United Nations and – perhaps ironically – also operated by GS1. Other classifications, that in addition encompass the attribute requirements too, are eClass and ETIM.

A manufacturer can have product information in an in-house ERP, MDM and/or PIM application. In the same way a merchant (retailer or B2B dealer) can have product information in an in-house ERP, MDM (Master Data Management) and/or PIM application. Most often a pair of manufacturer and merchant will not use the same data standard, taxonomy, format and structure for product information.

1-1 Product Data Syndication

Data pools have not substantially penetrated the product data flows encompassing all product groups and all the needed attributes and digital assets. Besides that, merchants also have a desire to provide unique product information and thereby stand out in the competition with other merchants selling the same products.

Thus, the highway in product data syndication is still 1-1 exchange. This highway has these lanes:

  • Exchanging spreadsheets typically orchestrated as that the merchant request the manufacturer to fill in a spreadsheet with the data elements defined by the merchant.
  • A supplier portal, where the merchant offers an interface to their PIM environment where each manufacturer can upload product information according to the merchant’s definitions.
  • A customer portal, where the manufacturer offers an interface where each merchant can download product information according to the manufacturer’s definitions.
  • A specialized product data syndication service where the manufacturer can push product information according to their definitions and the merchant can pull linked and transformed product information according to their definitions.

In practice, the chain from manufacturer to the end merchant may have several nodes being distributors/wholesalers that reloads the data by getting product information from an upstream trading partner and passing this product information to a downstream trading partner.

Data Quality Implications

Data quality is as always a concern when information producers and information consumers must collaborate, and in a product data syndication context the extended challenge is that the upstream producer and the downstream consumer does not belong to the same organization. This ecosystem wide data quality and Master Data Management (MDM) issue was examined in the post Watch Out for Interenterprise MDM.

Privacy and Confidentiality Concerns in Interenterprise Data Sharing

Henrik Gabs LiliendahlLeave a comment

Exchange of data between enterprises – aka interenterprise data sharing – is becoming a hot topic in the era of digital transformation. As told in the post Data Quality and Interenterprise Data Sharing this approach is the cost-effective way to ensure data quality for the fast-increasing amount of data every organization has to manage when introducing new digital services.

McKinsey Digital recently elaborated on this theme in an article with the title Harnessing the power of external data. As stated in the article: “Organizations that stay abreast of the expanding external-data ecosystem and successfully integrate a broad spectrum of external data into their operations can outperform other companies by unlocking improvements in growth, productivity, and risk management.”

The arguments against interenterprise data sharing I hear most often revolves around privacy and confidentiality concerns.

Let us have a look at this challenge within the two most common master data domains: Party data and product data.

Party Data

The firm CDQ talk about the case for sharing party data in the post Data Sharing: A Brief History of a Crazy Idea. As said in here: The pain can be bigger than the concern.

Privacy through the enforced data privacy and data protection regulations as GDPR must (and should) be adhered to and sets a very strict limit for exchanging Personal Identifiable Information only leaving room for the legitimate cases of data portability.

However, information about organizations can be shared not only as exploitation of public third-party sources as business directories but also as data pools between like-minded organizations. Here you must think about if your typos in company names, addresses and more really are that confidential.

Product Data

The case for exchanging product data is explained in the post The Role of Product Data Syndication in Interenterprise MDM.

Though the vast amount of product data is meant to become public the concerns about confidentiality also exist with product data. Trading prices is an obvious area. The timing of releasing product data is another concern.

In the Product Data Lake syndication service I work with there are measures to ensure the right level of confidentiality. This includes encryption and controlling with whom you share what and when you do it.

Data governance plays a crucial role in orchestrating interenterprise data sharing with the right approach to data privacy and confidentiality. How this is done in for example product data syndication is explained in the page about Product Data Lake Documentation and Data Governance.

Four Ways You as a Merchant Can Exploit Product Data Syndication

Henrik Gabs LiliendahlLeave a comment

Product Data Syndication has become an essential capability to manage within digital transformation at merchants as wholesalers and retailers. There are 4 main scenarios.

1: Inbound product data syndication of resell (direct) products

The process involves getting the most complete set of product information available from the supplier in order to fit the optimal set of product information needed to support the often self-service based buying decision by your customers.

This can be done by direct feeds from suppliers or through feeds via the various data pools that exist in different industries and geographies.

2: Inbound product data syndication for indirect products

You also need product data for parts used in Maintenance, Repair and Operation within facility management around logistic facilities, offices, and other constructions where products for MRO are needed. With the rise of the Internet of Things (IoT) these products are becoming more and more intelligent and are operated in an automatic way. For that, product information is needed in an until now unseen degree.

Every organization needs products and services as furniture, office supplies, travel services and much more. The need for onboarding product data for these purchases is still minimal compared to the above-mentioned scenarios. However, a foreseeable increased use of Artificial Intelligence (AI) in procurement operations will ignite the requirement for product data onboarding for this scenario too in the coming years.

3: Outbound product data syndication to marketplaces

Selling products on marketplaces has become a popular alternative to selling via ones own ecommerce site. While price and delivery options are main drivers here there are still more business to win via this channel if you can provide better and more unique product information than other resellers of the same product.

4: Outbound product data syndication to customers using products as parts

Your business-to-business (B2B) customers may also need product data for parts used directly in production or in Maintenance, Repair and Operation in production facilities and within facility management around logistic facilities, offices, and other constructions where products for MRO are needed. With the rise of the Internet of Things (IoT) these products are becoming more and more intelligent and are operated in an automatic way. For that, product information is needed in an until now unseen degree.

The Need for Collaborative Product Data Syndication

The sharp rise of the need product data syndication calls for increased collaboration through data partnerships in business ecosystems.

In the Product Data Lake venture I am working on now, we have made a framework – and a piece of Software as a Service – that is able to leverage the concepts of inbound and outbound Product Data Syndication and enable the four mentioned ways of utilizing product data syndication to create better business outcomes for you as a merchant.

Product Data Lake acts as a single point of digital contact for suppliers and customers in the product data supply chain which also provide you as a merchant with single place in the cloud from where your Product Information Management (PIM), ERP and eCommerce applications get and put external product data feeds.

This concept enables automated self-service by suppliers and customers who also can subscribe to Product Data Lake. In The Product Data Lake platform you can control the product portfolio and the product attribute set you are sharing with each business partner.

Learn more about Product Data Lake here.

Product Data Supply Chain Management in Resell

Five Ways You as a Manufacturer Can Exploit Product Data Syndication

Henrik Gabs LiliendahlLeave a comment

Product Data Syndication has become an essential capability to manage within digital transformation at manufacturers. There are 5 main scenarios.

1: Outbound product data syndication for finished products

As a manufacturer you need to ensure that self-service buying decisions by the end customer through the channel partner point-of-sale will result in choosing your product instead of a product provided by your competitor.

This is achieved through providing complete product information in a way that is easy onboarded by each of your channel partners – as well as direct customers and marketplaces where this apply.

2: Inbound product data syndication for 3rd party finished products

As a manufacturer you often have a range of products that are not produced inhouse but are essential supplements when selling own produced products.

The process involves getting the most complete set of product information available from the supplier in order to fit the optimal set of product information needed to support the buying decision by the end customer where your own produced products and 3rd party products makes a whole.

3: Inbound product data syndication for raw materials and packaging

Here the objective is to get product information needed to do quality assurance and in organic production apply the right blend in order to produce a consistent finished product.

Also, the increasing demand for measures of sustainability is driving the urge for information on the provenance of the finished product and the packaging including the origin of the ingredients and circumstances of the production of these components. 

4: Inbound product data syndication for parts used in MRO

Product data for parts used in Maintenance, Repair and Operation is an essential scenario related in running the production facilities as well as in facility management around logistic facilities, offices, and other constructions where products for MRO are needed.

With the rise of the Internet of Things (IoT) these products are becoming more and more intelligent and are operated in an automatic way. For that, product information is needed in an until now unseen degree.

5: Inbound product data syndication for other indirect products

Every organization needs products and services as furniture, office supplies, travel services and much more. The need for onboarding product data for these purchases is still minimal compared to the above-mentioned scenarios. However, a foreseeable increased use of Artificial Intelligence (AI) in procurement operations will ignite the requirement for product data onboarding for this scenario too in the coming years.

The Need for Collaborative Product Data Syndication

The sharp rise of the need product data syndication calls for increased collaboration through data partnerships in business ecosystems.

In the Product Data Lake venture I am working on now, we have made a framework – and a piece of Software as a Service – that is able to leverage the concepts of outbound and inbound Product Data Syndication and enable the five mentioned ways of utilizing product data syndication to create better business outcomes for you as a manufacturer.

Product Data Lake acts as a single point of digital contact for suppliers and channel partners in the product data supply chain which also provide you as a manufacturer with single place in the cloud from where your Product Lifecycle Management (PLM), ERP and Product Information Management (PIM) applications get and put external product data feeds.

This concept enables automated self-service by suppliers and channels partners who also can subscribe to Product Data Lake. In The Product Data Lake platform you can control the product portfolio and the product attribute set you are sharing with each business partner.

Learn more about Product Data Lake here.

Product Data Supply Chain Management in Manufacturing

2021 Data Management Mind Map

Henrik Gabs LiliendahlLeave a comment

Disciplines come and go in the data management world. Here is a mind map of the disciplines on top of my mind today. Some of the disciplines goes back to the emerge of IT in the previous millennium and some have risen during the latest years.

The Role of Product Data Syndication in Interenterprise MDM

Henrik Gabs LiliendahlLeave a comment

Interenterprise Master Data Management is on the rise as reported in the post Watch Out for Interenterprise MDM. Interenterprise MDM is about how organizations can collaborate by sharing master data with business partners in order to optimize own master data and create new data driven revenue models together with business partners.

One of the most obvious places to start with Interenterprise MDM is Product Data Syndication (PDS). While PDS until now has been mostly applied when syndicating product data to marketplaces there is a huge potential in streamlining the flow of product from manufacturers to merchants and end users of product information.

Inbound and Outbound Product Data Syndication

There are two scenarios in interenterprise Product Data Syndication:

  • Outbound, where your organization as being part of a supply chain will provide product information to your range of customers. The challenge is that with no PDS functionality in between you must cater for many (hundreds or thousands) different structures, formats, taxonomies and exchange methods requested by your customers.
  • Inbound, where your organization as being part of a supply chain will receive product information from your range of suppliers. The challenge is that with no PDS functionality in between you must cater for many (hundreds or thousands) different structures, formats, taxonomies and exchange methods coming in.

Learn more in the post Inbound and Outbound Product Data Syndication.

4 Main Use Cases for Collaborative PDS

There are these four main use cases for exchanging product data in supply chains:

  • Exchanging product data for resell products where manufacturers and brands are forwarding product information to the end point-of-sale at a merchant. With the rise of online sales both in business-to-consumer (B2C) and business-to-business (B2B) the buying decisions are self-service based, which means a dramatic increase in the demand for product data throughput.
  • Exchanging product data for raw materials and packaging. Here there is a rising demand for automating the quality assurance process, blending processes in organic production and controlling the sustainability related data by data lineage capabilities.  
  • Exchanging product data for parts used in MRO (Maintenance, Repair and Operation). As these parts are becoming components of the Industry 4.0 / Industrial Internet of Things (IIoT) wave, there will be a drastic demand for providing rich product information when delivering these parts.
  • Exchanging product data for indirect products, where upcoming use of Artificial Intelligence (AI) in all procurement activities also will lead to requirements for availability of product information in this use case.  

Learn more in the post 4 Supplier Product Data Onboarding Scenarios.

Collaborative PDS at Work

In the Product Data Lake venture I am working on now, we have made a framework – and a piece of Software as a Service – that is able to leverage the concepts of inbound and outbound PDS and enable the four mentioned use cases for product data exchange.

The framework is based on reusing popular product data classifications (as GPC, UNSPSC, ETIM, eClass, ISO) and attribute requirement standards (as ETIM and eClass). Also, trading partners can use their preferred data exchange method (FTP file drop – as for example BMEcat, API or plain import/export) on each side.

All in all, the big win is that each upstream provider (typically a manufacturer / brand) can upload one uniform product catalogue to the Product Data Lake and each downstream receiver (a merchant or user organization) can download a uniform product catalogues covering all suppliers.