IT and data departments depend on metadata to understand their own systems, and to align their tools and processes. Some tools, however, use:
- cryptic metadata that is not easily understood
- hidden metadata so it cannot be modified
- binary metadata other tools cannot use
The metadata used in the IRI CoSort package, the IRI Voracity data management (and ETL) platform, and their fit-for-purpose subset products (IRI NextForm, IRI FieldShield, and IRI RowGen), is in a familiar and self-documenting 4GL for data definition, manipulation (transformation, masking, cleansing, etc.) and mapping (reformatting and reporting) called SortCL.
Named for its legacy sort utility roots in 1992 after the "Sort Control Language" of VMS, SortCL has benefited from the clarity, velocity, reliability, and extensibility of this syntax ever since. And today SortCL syntax and job management is fully supported in the free Eclipse IDE for all IRI product users, IRI Workbench.
SortCL allows application, DW, business, and data governance teams to describe their structured data sources, manipulations, and reports in a simple, common language. SortCL reduces definition complexity and the need for elaborate metadata maintenance through a simple, shared syntax managed in text files and Eclipse, shareable in the cloud, and compatible with mapping automation and metadata conversion technologies from DataSwitch, AnalytiX DS (now Erwin) and MITI (MIMB).
SortCL defines the layout and manipulation of your data sources, processes, and targets for a wide range of applications and IRI software products. Whether you use IRI software to perform, speed, and/or combine data transformation, migration, protection, or reporting, SortCL metadata defines it.
SortCL data definition files (.DDF) define the structure and appearance of source files and database tables. SortCL specification files (.SCL), or job scripts, refer to the .DDF layouts. .SCL files use the source field names as symbolic references for target field manipulation and mapping.
SortCL metadata also supports the FAIR principles of data, aligning it with data warehouse and data mesh architectures, in that DDF is:
Findable -- as an Eclipse metamodel and text file, with metadata stored in project folders or user directories and shared in any repository like Git.
Accessible -- SortCL metadata is expressed in simple, self-documenting 4GL, saved in text files and run on the command line with the SortCL executable. The data definitions or job scripts explicitly name and locate data sources and targets and all mapping and manipulation functions.
Interoperable -- SortCL metadata describes all structured relational databases and flat file formats, including CSV, COBOL, LDIF, flat JSON and XML, and even Excel and ASN.1 CDR. SortCL DDF and job scripts work identically in IRI Voracity, CoSort, FieldShield, NextForm and NextForm jobs,
Reusable -- DDF metadata can re used in any number of SortCL-compatible application job scripts to define source and target data layouts for data integration, transformation, cleansing, masking, conversion, wrangling, reporting, synthesis, and more.
