Because the business development and IT technology evolution of an enterprise are usually progressive, the computerizaion of the business services must begin from scratch, from simple to complex, and then gradually form a number of application systems related to each other (often a certain degree of data exchange must be build) applications. In order to keep the consistency and integrioty of the data used dispersely in each business,the business intelligence field has came out a very important application – the Master Data Management (MDM); especially after the changing variety of BigData has been introduced into the enterprise analysis, clearly its role is becoming more important. Regardless of the extent of the businesses invested in MDM or what solution is chosen, the data quality is indeed the core issues.
Data quality management cycle
In addition to the MDM demands, for the complex external data sources, the data quality management mechanism during the creation of BI systems is also extremely important. The TrinityDQ data quality management system can be integrated with TrinityETL to manage data quality by the cycle of four modules: “Data Profiling”, “Data Auditing”, “Data Cleansing”, and “Data Quality Monitoring”.
It can identify the profile of the data, understand the information demand and ways to obtain information, analyze the information environment and to assess the levels of data quality and its impact.
According to the preset rules, it can audit data, list the ratio and distribution which do not comply with the rules; for the data which do not match the data rules, it can investigate the causes and the develop the improvement plans.
It can clean up the problems in the existing data, filter the data in a predetermined manner in accordance with the ETL procedures, and to correct it; or perform an exception procedure, such as handing over the problem to the data owners to decide the cleaning method.
4.Data Quality Monitoring
It can design the control mechanisms for continuously monitoring of data quality to see whether there is a gap, and then to generate a report for health status on a regular basis. Through the monitoring dashboard, it can allow the user to quickly and clearly understand the current data quality as the basis of the adjustment strategy for data quality maintenance.