The data metadata syntax for business, technical, and master metadata common to all IRI software is the "data definition file" or "DDF" format. DDF is the primary form of field (column, key/value) layout metadata used in IRI software, and supports the wide range of data handling functions and sources in IRI software.
More specifically, DDF is the data metadata subset of the process metadata for the SortCL (.scl) data transformation and reporting program in IRI CoSort, which is also the basis for metadata-compatible tasks in:
- IRI Voracity ETL, BI, and other jobs it supports in .flow / .bat / .sh files, including
- IRI NextForm (.ncl) data migration/replication jobs
- IRI FieldShield (.fcl) data discovery and masking jobs
- IRI RowGen (.rcl) test data generation and population jobs
The job script design and execution options for all these tools are front-ended in the IRI Workbench graphical IDE, built on Eclipse. Metadata modeling and metadata management are facilitated by the Eclipse modeling framework (EMF), built-in data mapping diagrams, and graphical editors for the metadata.
The DDF files are also the metadata repository for all structured data sources and jobs, all of which also:
Application Independence
Define job commands and data layouts together or separately in explicit, logical text files -- metadata repositories. Centralize the layouts for re-use.
Use the same source layouts in multiple applications. Save and re-use mapped target layouts as metadata sources for new applications.
Specify multiple targets and layouts at once to reflect multiple layouts and views of the same data.
Specify existing or custom-defined master data formats in files or tables within DDF repositories. Use those layouts in IRI Voracity ETL jobs, as well as any CoSort/SortCL-based transformation, cleansing, federation or reporting application.
Create DDF files manually, or automatically in the IRI Workbench graphical Metadata Discovery wizard.
Manage your DDF files, master data, and other metadata assets in a secure repository for team sharing, version control, and lineage tracking.
Convert to SortCL DDF from 3rd-party file layouts using command line utilities in the CoSort package, the Workbench, or MIMB.
Governance Goals
IRI metadata supports field-level protection functions like data masking and encryption along with data integration and reporting.
The DDF metadata -- and its inclusion in audit logs -- allow you to map these processes, and help satisfy the requirements of your risk and controls framework.
IRI's metadata and operational infrastructure allow you to replace multiple processes with one product, job script, and I/O pass. Simplicity, transparency, security, compliance, and auditability are included.
