An IRI Voracity Use Case for CDR Data Processing
This is the final article of 5 articles regarding ASN.1 support in SortCL and IRI Workbench. To begin reading from the first article in the series, click here.
Creating a Report on Filtered TAP3 CDRs:
Consider an input file that is based on the TAP3 schema standard (written in ASN.1) and BER (Basic Encoding Rules)-encoded. Trying to get information from this data is not simple. The file is binary, encoded and not human-readable.
SortCL capabilities can help sort, filter, aggregate and ultimately generate a report to focus on the important data. This input file can even be generated itself with RowGen capabilities, such as in this example(note that the fields for both the INFILE section and the OUTFILE section have been truncated for brevity):
/INFILE=rfp
/PROCESS=RANDOM
/INCOLLECT=1000
/FIELD=(transferBatch_batchControlInfo_sender, POSITION=1, SEPARATOR=",",SET={ GBRCN,GBRVF,DEUD1,DEUD2})
/FIELD=(transferBatch_batchControlInfo_recipient, POSITION=2, SEPARATOR=",",SET={ GBRCN,GBRVF,DEUD1,DEUD2})
/FIELD=(transferBatch_batchControlInfo_fileSequenceNumber, POSITION=3, SEPARATOR=",",TYPE=WHOLE_NUMBER,SIZE=5)
/FIELD=(transferBatch_batchControlInfo_fileCreationTimeStamp_localTimeStamp, POSITION=4, SEPARATOR=",",SET={20050907120940,20060907120940})
……
/REPORT
/OUTFILE="TAP-0310.asn;tapgen.ber"
/PROCESS=ASN1
/FIELD=(transferBatch_batchControlInfo_sendera=encode_hex(transferBatch_batchControlInfo_sender), POSITION=1, SEPARATOR=",")
/FIELD=(transferBatch_batchControlInfo_recipienta=encode_hex(transferBatch_batchControlInfo_recipient), POSITION=2, SEPARATOR=",")
/FIELD=(transferBatch_batchControlInfo_fileSequenceNumber, POSITION=3, SEPARATOR=",")
/FIELD=(transferBatch_batchControlInfo_fileCreationTimeStamp_localTimeStampa=encode_hex(transferBatch_batchControlInfo_fileCreationTimeStamp_localTimeStamp), POSITION=4, SEPARATOR=",")
……
This example has an input file containing 1000 CDRs (Call Detail Records). It is filtered to a certain timestamp, sorted by call duration, and only relevant fields are output to the report, which is limited to 10 records by the /OUTCOLLECT=10 statement.
/INFILE="TAP-0310.asn;tapgen.ber" # 1000 ASN.1 BER encoded CDRs /PROCESS=ASN1 /SPEC=tap3.ddf /OMIT WHERE TRANSFERBATCH_CALLEVENTDETAILS_GPRSCALL_GPRSBASICCALLINFORMATION_CALLEVENTSTARTTIMESTAMP_LOCALTIMESTAMP NE "3230323030393037313230393430" /SORT /KEY=TRANSFERBATCH_CALLEVENTDETAILS_GPRSCALL_GPRSBASICCALLINFORMATION_TOTALCALLEVENTDURATION /OUTFILE=tap3.data.out /PROCESS=RECORD /HEADREC="Timestamp Duration Network V_In V_Out Item\n" /SPEC=tap3formattedoutput.ddf #select few fields /OUTCOLLECT=10
This example shows a flat file report, but the report can also be output to other sources, including XLSX and XLS spreadsheet formats, even from the same script. To do that, add an outfile target for XLSX or XLS. The resulting script may then look like this:
/INFILE="TAP-0310.asn;tapgen.ber" # 1000 ASN.1 BER encoded CDRs /PROCESS=ASN1 /SPEC=tap3.ddf /OMIT WHERE TRANSFERBATCH_CALLEVENTDETAILS_GPRSCALL_GPRSBASICCALLINFORMATION_CALLEVENTSTARTTIMESTAMP_LOCALTIMESTAMP NE "3230323030393037313230393430" /SORT /KEY=TRANSFERBATCH_CALLEVENTDETAILS_GPRSCALL_GPRSBASICCALLINFORMATION_TOTALCALLEVENTDURATION /OUTFILE=tap3.data.out /PROCESS=RECORD /HEADREC="Timestamp Duration Network V_In V_Out Item\n" /SPEC=tap3formattedoutput.ddf #select few fields /OUTCOLLECT=10 /OUTFILE=”HEADER;tap3data.xlsx” # output header from field names - or specify manually in SDEF /PROCESS=XLSX /SPEC=tap3formattedoutput.ddf #select few fields /OUTCOLLECT=10
Here is an IRI Workbench view of the script and mapping diagram for this job:
The data can be more robustly visualized from within one of these spreadsheet formats.
Wrap Up
Thanks to ASN.1 support in IRI’s SortCL program, users of the Voracity data management platform and its component products can now process CDRs from TAP3 encoded data, along with any combination of schema and ASN.1 encoding rules.
A secondary license for the Compile-and-Go Library (CAGL) from OSS Nokalva and your encoding rules are also needed for SortCL scripts processing ASN.1 data, or for the asn1_2ddf metadata converter to work.
The bottom line is that a wide array of data manipulation and generation functions can be performed with IRI support for ASN.1-encoded data in SortCL-compatible jobs … in ways other solutions for processing data (and much less this type of data) cannot. If you would like to discover, integrate, migrate, govern and/or analyze data in ASN.1 files directly, please set up a discussion or demo with us here.
Other articles in the series:
