Challenges
You need realistic test data files and test report generation to:
- develop and stress JCL and COBOL programs
- build and simulate new BI and analytic applications
- evaluate CRM, ERP, EAI, EII, and other systems
- prototype ETL operations and bulk load scenarios
- benchmark new hardware or software platforms
The data you have in production files or reports may be inaccessible, restricted by company policy or data privacy laws, or may not even exist. And if you do have access to test data files or test reports, they may not contain the range or volume of data that may be involved in the future.
And finally, the test data management tools market is not exactly rife with solutions for use cases requiring:
- test data generation for mainframes
- synthetic data for software testing
- automated test data for ETL and BI
- COBOL test data generation tols
- data masking for test reports
Solutions
IRI RowGen software can synthesize and customize test data generation directly in many structured file formats, as well as in detail and summary report targets:
- Record, line, or variable sequential
- CSV and other delimited files, JSON, and XML (click for task examples)
- Excel (XLS and XLSX, in row or column format) - see examples # 1&2 here
- LDIF
- Structured (flat/fixed-length) text files
- COBOL index (MF-ISAM, Vision)
- Mainframe blocked
- Detail/summary reports with custom formatting
RowGen randomly creates field values in more than 100 data types. It can also randomly select (or mask) data from real datasets, so the test data values can contain either artificial and real (but scrambled or otherwise de-identified) data. RowGen also applies intelligent selection and ranging criteria, as well as desired data frequency distributions, to improve data realism.
Targets in these formats can persist in standalone files written to on-premise folders or cloud buckets in Amazon S3, Azure Blob and Google Cloud Platform (GCP) storage. But they can also stream through pipes, procedures, or message queues to other test targets.
Embedded support for standard and complex data transformations, set file (value) lookups, conditional filters, and custom formatting helps RowGen simulate production file and report formats for a variety of applications. And once these files are built, or streaming test data sets are built, your test file management (test data provisioning) possibilities include both static, automated, and virtual test data targets, as well as test data management hubs and CI/CD platforms like Jenkins, Amazon Code Pipeline, Azure DevOps and GitLab.
You can also combine RowGen with IRI DarkShield to find and replace real values with synthetic values in semi-structured sources (like JSON, XML, FHIR, HL7, and X12 files, as well as NoSQL databases) and unstructured sources (like free text and image files). This allow you to produce realistic test documents, EDI files, NoSQL collections, and images for a variety of development or sharing use cases. Indeed, if you need a test data masking tool for files -- in structured, semi-structured and unstrucutured formats -- take a close look at IRI DarkShield.
