Web Decisions Drives Marketing Data Faster with CoSort
Category
Data Quality, Profiling & Augmentation
Reviewer
Christopher Addington, founder and president of Web Decisions.
Background
Web Decisions provides multichannel marketing solutions for retail, mail order and e-commerce businesses. By utilizing its advanced, online marketing system, Web Decisions helps companies reach their targeted audience with personalized messages based on their customers' interactions across multiple channels. Web Decisions has extensive experience in the area of constructing and maintaining marketing databases while providing the means for one-to-one communications utilizing various media.
Platform
Web Decisions' 360i eMarketing system currently operates on Windows 2000/2003 and multiprocessor Intel servers utilizing SAN disk arrays.
Problem Solved
The 360i eMarketing system was developed by Web Decisions to provide the marketer with near real- time reporting and extremely fast data processing capabilities on millions of records. Several of the 360i capabilities involving very large marketing databases require that the database records be sorted in a particular order before other proprietary processes can be initiated. Speed is crucial when working with client databases that contain millions of records. Accordingly, performance was important to Web Decisions in its selection of a sort utility. CoSORT's speed exceeded our expectations. An absolute requirement of the 360i application was that a sort utility must be easily incorporated into the back- end scripting that is launched as a batch process. The CoSORT sort control language (SortCL) is a flexible and powerful scripting language that can be executed from the operating system's command-line interface. By including CoSORT scripts in Windows .bat files, Web Decisions easily accomplished its back-end processing requirement.
Product Functionality
The CoSORT SortCL is a powerful 4GL language that describes and exposes the extensive functionality of this product. While this product also ships with a GUI, the sort control language provides capabilities only limited by a user's ability to construct complex scripts that can perform multiple functions in a single pass of one or more input files. SortCL provides the ability to join, select, filter, sort, translate and aggregate data from multiple files in different formats. Similar to relational database joins, SortCL can match two or more files and produce merged output. Records can be selected and filtered based on conditional criteria, and reformatted to a new record layout before being output. Additionally, fields can be translated and data can be aggregated or summarized, thus providing an ideal method for producing quick reports. The bonus is that CoSORT performs all of these functions extremely fast and, if we want it to, in the same script and/or pass through the data.
Strengths
While Web Decisions has just started implementing CoSORT version 8.1, our initial tests indicate that performance has increased substantially. The CoSORT SortCL continues to expand with improved support for files that have field delimiters. Web Decisions works extensively with very large Microsoft CSV (comma separated values) files and is looking forward to these added features.
Weaknesses
Based on Web Decisions' data processing requirements, CoSORT did not handle some of the Microsoft CSV files received from outside sources. Inconsistencies in fields being surrounded or not surrounded by quotes caused undesirable output. However, Innovative Routines International (IRI) quickly responded with SortCL scripts that worked around the issue.
Selection Criteria
Web Decisions chose CoSORT because of its SortCL and the ability to easily incorporate sort functionality into its backend batch processing. This made CoSORT a clear winner because the major competitor does not support a command-line interface. Had the competitor met this requirement, CoSORT would still have been selected based on performance and price.
Deliverables
Today, Web Decisions is primarily utilizing SortCL to sort, aggregate and reformat millions of records. However, the posture at Web Decisions is to first evaluate CoSORT as a solution for any data processing tasks, before submitting the task to programming. After utilizing the product for two years, CoSORT has saved Web Decisions weeks and possibly months of potential proprietary programming tasks.
Vendor Support
Web Decisions has been very pleased with the support IRI has supplied. With regards to an issue of handling Microsoft CSV files, IRI has now released CoSORT version 8.1 that has enhanced support for all types of field delimited files. The support staff is currently working closely with us to implement the new release.
Documentation
Web Decisions received both hard-copy and online documentation and found each useful while developing SortCL scripts. All of Web Decisions' real-world sort routines were created by reviewing the examples provided in the documentation.
