Quick Links
Challenges
The bin sort tools in Unix and Linux use sorting algorithms that do not perform well (scale) as input volumes increase. Those native data sorting tools also use sorting techniques that cannot:
- handle many data types
- filter, reformat, mask or cleanse data at the same time
- replace mainframe, COBOL or ETL sort functions
- address DW ETL sort, aggregation or join needs
Nevertheless, you're familiar with Unix sort syntax, and may have an investment in jobs using /bin/sort commands. You need a more robust sort engine under the hood on Unix, or the same functionality on Windows.
Solutions
IRI CoSort packages for Unix, Linux, and Windows include a faster, more robust drop-in replacement for the /bin/sort program.
Use the same Unix sort syntax (but with the CoSort engine) on the command line, or in batch jobs, to reorder huge files in parallel. The CoSort engine outperforms the Unix system sort by several orders of magnitude, scales linearly in volume, and does not fail.
After seeing dramatic improvements in large file sorting speed, CoSort users move onto a more powerful interface - the Sort Control Language (SortCL) program. SortCL combines sorting operations with data:
- Transformation (scrub, sort, join, group, etc.)
- Conversion (data types, record layouts, files)
- Protection (field-level encryption, de-ID, etc.)
- Reporting (custom detail, delta and summaries)
in addition to legacy sort migrations, Oracle unloading, ETL acceleration, and some of the related solutions outlined throughout this site, like: data validation and scrubbing, pattern matching, complex transforms, etc.
