Use Case Accelerators > Cleansing Data Using RegExReplace, Conditional Expressions, and Filters

Cleansing Data Using RegExReplace, Conditional Expressions, and Filters

Article #: Product: Version:

Summary

Many businesses are prioritizing efforts to cleanse or standardize data to improve data quality. One way to cleanse data is to identify invalid data and write it to an invalid data file, and write valid data to a separate, valid data file. With this approach, you can analyze and correct the invalid data in the invalid data file while continuing to process data from the valid data file.

You can facilitate data cleansing by using the DMExpress RegExReplace function, conditional expressions, and filters to create two targets, one each for valid and invalid data.

Resolution

One example of data cleansing can involve identifying non-alphabetic characters in fields where only alphabetic characters are expected, writing the associated records to an invalid data file, and writing records with valid data to a valid data file as shown in the following Input Data file, Valid Data file, and Invalid Data file:

To find invalid data in the Input Data file, the FirstName field value is evaluated for non-alphabetic characters:

DMExpress Processing

In the attached example, this processing is done in DMExpress using the RegExReplace function, conditional text, and filters to create the two targets.

Attachments

215_SeparatingInvalidDataNames.zip, compatible with DMExpress version 7.0.0 or higher

Additional Information

For additional information on DMExpress functions, see DMExpress functions reference in the DMExpress Help.

Last updated: