Abstract: IRI has partnered with Windocks to provide an on-demand, test-ready virtualized repository that delivers masked, subsetted or synthesized databases in seconds for Docker containers, conventional instances, or workstations. These sanitized clones support any on-premise or cloud infrastructure, any database, and any block level, object, or file storage system.

 

Application DevOps continues to advance with automated continuous integration pipelines, containers, and package repositories. These tools allow identical environments to be delivered on demand.

Test data management, on the other hand, lacks on-demand delivery of test-ready databases. Fortunately, containers and on demand virtualized databases are now available, for on demand delivery of test ready databases, sufficient to keep pace with modern application DevOps.

This article outlines steps to apply IRI FieldShield data masking or IRI RowGen synthetic data operations designed in IRI Workbench to a secure database image, for delivery of writable copies in seconds. In the Windocks cloud provisioning environment, each database consumes only 40 MB of storage.

Combining IRI data masking and synthetic data with virtualized databases

The containers for database clones can now directly leverage the data cleansing and protection operations available in the IRI Voracity data management ecosystem thanks to a tie up with Windocks technology. Windocks operates on premise or in the cloud to deliver writeable refreshed database clones to users.

Database virtualization combined with containers is attractive for greater dev/test agility and automation, but also for improved data security and governance, and cost savings. A single VM can host up to 50, or even 100 simultaneous environments and allows for consolidation of dev/test VMs and instances.  Organizations average a 50 to 70% reduction in database dev/test infrastructure costs.

Virtualized database images

Virtualized databases are delivered in seconds, sourced from a block-level storage, or from Windows Virtual Hard Drive (VHD) images.   Both approaches deliver writable multi-database environments in seconds, with each consuming only 40 MB on delivery.  Images can be updated as needed to deliver point-in-time or near real-time production databases for debug and support purposes.

Database images are built from a plain text configuration file (dockerfile), that specifies the database environment, databases, and IRI data masking, data quality, and synthetic data job parameters. The sample dockerfile below includes a single full database backup, but could specify dozens of databases.

IRI data masking, synthetic data, and data cleansing jobs are typically created in IRI Workbench, saved as SortCL job scripts (SCL file extension), and executed during the image build by calling an IRI batch file.  A batch file can be generated from a flow diagram in IRI Workbench, which is a visual representation of the flow of actions to be executed.

The IRI Workbench screenshot below shows some of the Voracity job design activity in this example:

• a flow diagram of the setup batch file to be executed on the database image, shown top left.
• a glimpse of the FieldShield data masking script and its model outline, shown top right.
• a mapping diagram of the same data masking script, shown bottom left.
• part of the original low-quality data in the database that will be cleansed upon delivery of the virtualized database image, shown bottom right:

Dockerfiles can also specify IRI jobs during database delivery, to support dynamic data masking or masking of databases that were updated following the initial image build. Here is a dockerfile configuration that would run Windocks database cloning and IRI Voracity jobs all at once:

With the IRI jobs created, building the database image starts with the Windocks web application, navigating to the Build tab, selecting the dockerfile and assigning an image name, and “build.”

On completion, the image is available to authorized users via the Images tab:

Self-service and automated delivery

Authorized users access images on the Image tab, enabling delivery of fresh environments with options to specify container ports and passwords. The same image supports delivery of databases to either database containers or conventional VMs and workstations.

A dropdown tool allows users to select a subset of databases from a complex image. Restful API calls support Jenkins, Azure DevOps, and other CI automation.

Environments are delivered in seconds, regardless of size or complexity, with the IRI data masking and other enhancements built in to each database. Each environment consumes only 40 MB on delivery, and supports writes with a Copy-on-Write from the image.

A real-time view of database environments is displayed on the Live Secondaries tab, with each container assigned a unique port.

SQL Server containers are identical to conventionally installed instances, compatible with Active Directory and Windows Authentication, and include full SQL Server services. Containers are accessed with normal tools, such as SQL Server Management Studio (SSMS) using the servername, and instance name (ie., Windocks\Instance10002).

This image shows the cleansed data that results after instantiating a container that calls IRI jobs through a batch file, which may be exported from IRI Workbench.

Affordable database virtualization with IRI is here

Database virtualization is featured by IBM and other legacy solution vendors, but comes at a high purchase price and requires substantial resources to adopt. The solution highlighted in this article is available via a software download or preconfigured VM images on the cloud, allowing evaluations to be up and running in minutes.

Windocks database virtualization is also unique in the marketplace, by supporting existing infrastructure, including Active Directory, and enterprise storage systems.  In addition to enabling greater data agility, with automated testing cycles measured in seconds, data security and governance are improved over traditional database provisioning.

Near real-time access to production database environments is also easily achieved, with database images updated with differential or transaction log backups. Dev/test infrastructure costs can also be reduced by half or more, with a 5-10x reduction in VMs, and 99% reduction in storage costs.

Inquire with either Windocks or IRI to access a supported, no-cost evaluation. All that is needed is a Windows Server (2012 r2, 2016, or 2019), with a minimum 2 vCPU cores, and 4 GB of RAM.  Visit www.windocks.com or www.iri.com for more information about the technologies described here, and email info@iri.com if you would like to register for an onsite evaluation or cloud-based pilot and test drive the solution.