Single-Pass Sort, Join and Aggregation Transformations [video]
CoSort can sort, join, and aggregate in the same job. In fact, CoSort’s SortCL Program supports the consolidation of these three major data warehouse transformation functions in the same job script and I-O pass, along with reporting, protection, and many other data manipulations that would otherwise require separate steps, and possibly separate products. This kind of task consolidation is not only more efficient at runtime, it saves disk space, simplifies job design, and minimizes code maintenance.
In this demonstration, we are completing a sort, join, and aggregation in the same job script and in one I-O pass. We are using this demonstration to show how a user might pull the components together for a billing statement by using the combined work of the sort, join, and aggregation processes. We take transaction files from a store, join those files with customer information records and then aggregate the sums of transactions for each customer so that the output records could be used for input into billing statements.