IRI’s data management tools share a familiar and self-documenting metadata language called SortCL. All these tools — including CoSort, FieldShield, NextForm, and RowGen — require data definition file (DDF) layouts with /FIELD specifications for each data source so the data is mapped to the metadata.

Because these DDF files can define both sources and targets, they are easily reusable. Part of the flexibility is that field attributes that are not applicable for a file type are ignored. Therefore, this allows the use of the same DDF for a CSV file, a database table,  a JSON file, or any other data source provided the base field structure is the same.

Using the Discover Metadata wizard in IRI Workbench (WB) produces the following DDF from a CSV file (shown in the bottom half of the screen shot). The name of the fields are generated from the header row.

The target for the job is a JSON file that is created from this csv file. However, since the JSON file does not exist yet, the Discover Metadata wizard cannot be used to create the metadata. Instead, a context menu item “Add Location Definition” can be used on the DDF created from the csv file to add definitions for how the JSON file will be formatted.

There are currently five types of Location Definitions (DEFs): ODEF for databases, JDEF for JSON files, XDEF for XML files, LDEF for LDIF files, and MDEF for MongoDB. Each DEF has its own format based on standard industry paths. After selecting the type of location definition that is needed, the table of suggested names is filled out by using the existing name of the field from the DDF.

However, the format needs to change so that an object is output for the first and last name fields. By using JSON path syntax, an object called “NAME” with two objects called “LAST” and “FIRST” can be created. Additionally, an “ADDRESS” object is created with four detail objects within it.

Upon clicking OK, the DDF is modified to include the specified JDEFs.

Now the same DDF can be used for both source and target in the migration wizard below. The source csv file has no use of the JDEF; therefore, it is ignored in source.

Running the produced script creates the following JSON file. Note how the names and address details are grouped.

Creating and modifying the DDF files before starting any jobs allows greater flexibility in their use. Various context menus are available for making the modifications; however, hand editing can be used as well.

If you have any questions or need help, contact your IRI representative here.