Challenges
High volume data transformations may represent a performance bottleneck for some users of the ETI Solution. The current remedy, if any, may be a separate product, pass, and place for processing the data.
Separate transformation programs or tools offer no ready way to leverage existing ETI metadata. This represents inconvenience in job design, inefficiency in execution, or both.
Solutions
It is easy to improve high-volume data transformation performance; call IRI CoSort directly within ETI Solution.
Use ETI Solution code templates to populate CoSort's Sort Control Language (SortCL) field layout and data manipulation commands, based on existing ETI metadata and transformation requirements.
Specify a start function with SortCL job controls that can call other specifications. The start function can call other functions.
The results combine with existing ETI Metadata to produce run-ready SortCL job scripts containing the correct file names, sort keys, field attributes, and so on. In fact, by using SortCL, you can perform, speed, and/or combine data:
- filtering (select, scrub, links to DQ tools)
- transformation (sort, join, aggregate, calc, etc.)
- conversion (data-type and file-format migrations)
- presentation (batch, summary, and delta reports)
- protection (field encryption, de-ID, masking)
Ask us for a combined ETI-CoSort/SortCL code sample.
