I’m not sure that there’s any perfect approach to integrating data and planning a data integration process prior to knowing what the source content is going to look like. That being said, there are always some standard methods and practices that we recommend when data integration activities are expected to be commonplace:
- Establish standard business terminology and value standards for each subject area.
- Develop a business data dictionary that is owned and maintained by a series of business-side data stewards. These individuals should ensure that all terminology is kept up to date and that any associated rules are documented.
- Document the data in your core systems and how it relates to the standard business terminology. This will include data transformation and conversion rules.
- Establish a set of data acceptance criteria and correction methods for your standard business terminology. This should be identified by the business-side data stewards and implemented against each of your core systems (where practical).
- Implement a data profiling program as a production process. You should regularly measure the data quality (and value accuracy) of the data contained within each of your core operational systems.