FAQ > How to Determine whether the DMExpress Hadoop Sort was Used

How to Determine whether the DMExpress Hadoop Sort was Used

Article #: Product: Version:

Summary

DMExpress can seamlessly plug into the Hadoop architecture in order to accelerate sorting and improve the performance of existing MapReduce jobs. You can confirm that DMExpress was invoked as a sort accelerator for MapReduce jobs by viewing the Hadoop job logs.

Resolution

When DMExpress is invoked as a Hadoop sort accelerator, the stderr logs of each Hadoop task will include standard DMExpress execution logs and statistics. For example:

If no DMExpress log output similar to the above appears in stderr, then DMExpress was not invoked as the sort accelerator.

The DMExpress copyright will not appear in the log when a null sort is used on the map side. See the "DMX-h Sort User Guide" for details on the null sort.

Additional Information

For information on how to find the DMExpress logs within Hadoop, see Finding DMExpress Hadoop Logs on MRv1 or Finding DMExpress Hadoop Logs on YARN (MRv2).

Last updated: