Use Case Accelerators > Processing Twitter JSON Data
Processing Twitter JSON Data
JSON data format is widely used in modern client/server applications such as Twitter due to its compactness over XML and its data-centric design.
DMExpress can process UTF-8 JSON input data via the JSON Reader task. The output from the JSON reader can subsequently be used in other downstream tasks within a DMExpress job.
The DMExpress JSON Reader task enables JSON data to be treated as a DMExpress source. Specifically, the JSON Reader converts a UTF-8 JSON source into a UTF-8 CSV (comma-separated values) output, which can then be read by other DMExpress tasks.
In this example, the JSON Reader takes the JSON file tweets.txt as its source, and writes the CSV output directly to stdout. Create the JSON Reader task to perform the JSON to CSV conversion as follows:
Select Edit ->Add JSON Reader… to create the task.
In the Source pane, select File path and enter the path to the file tweets.txt.
In the Target pane, select Standard output.
In the Target layout pane, generate a delimited record layout corresponding to the source layout and save it to a new DMExpress metadata file:
Enter metadata.dxt in the Link to DMExpress metadata file field.
Enter tweetLayout in the Map input fields to delimited layout field.
Click on Create layout by scannping input file.
This layout will be linked in as external metadata in the subsequent task. The populated window should look similar to this:
Click OK, then click to place the task on the job canvas.