Affordable BIRT Acceleration via CoSort

by Jackie Sabbagh

Business Intelligence Reporting Tool (BIRT) is a free, open source, visual reporting platform for Eclipse, and the commercial foundation for rich analytic extensions available from Actuate. More than 2 million people around the world use BIRT. However, as with most BI tools, data visualization, not data integration, is BIRT’s strength.

Actuate suggests the use of Hadoop, in-memory databases, or other new ‘big data‘ technologies to prepare data for BIRT analytics. But those are complex, costly roads to travel, and often overkill.

Fortunately, BIRT now also works directly with the SortCL program engine in IRI CoSort (and thus IRI Voracity) in the same Eclipse GUI for single-pass data preparation and presentation. Using the Eclipse Open Data Access (ODA) protocol, BIRT users can specify an “IRI Data Source” to rapidly produce reports from fit-for-purpose data sets that CoSort produces.

CoSort is the traditional big data engine, performing and combining bulk data transformations in Unix and Windows file systems for 4- or 5-figure perpetual use license fees.

According to Athena Solutions founder Rick Sherman,

… the CoSort SortCL program prepares (or “franchises”) data in disparate file and database sources for BI by transforming, cleansing, masking, and re-formatting the data. SortCL jobs are described in simple text files that use centralized metadata, and are supported by job design and management facilities in the IRI Workbench GUI, built on Eclipse™.

SortCL can produce CSV files that BIRT can rapidly ingest as a data source. Moreover, SortCL can feed output records directly in memory to industry-driven BIRT templates via ODA.

In the dynamic workflow paradigm, both the metadata and the output data from SortCL jobs are provided seamlessly to BIRT templates for use at reporting time. Data integration occurs during visualization, so custom views from big data can materialize much faster.