Split and Transform Files

Overview

In this lab, you develop Camel routes that process CSV files, split them into individual records, and transform them into Java objects that are written as individual JSON files. The transformation is done using the Fuse Integration Editor. You also develop exception handling logic to process incorrectly formatted records.

Goals

  • Build an Apache Camel route that consumes CSV files and transforms the records into Java objects using the Bindy transformation component

  • Split the list of received Java objects and apply a Java-to-Java transformation using the Fuse Integration Editor (from org.acme.Customer to org.globex.Account)

  • Map the fields as defined within the table

  • Transform the org.globex.Account Java objects into individual JSON files

  • Implement the DLQ pattern to move erroneous CSV records into an error folder

  • Design a unit test to validate the data transformations

file transfer.png

camelContext

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="        http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd        http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd">
    <camelContext id="_camelContext1" xmlns="http://camel.apache.org/schema/spring">
        <propertyPlaceholder id="properties" location="fabric8/route.properties"/>

        <!--
             Transformation Endpoint doing csv2java conversion
             Step 1 : csv record is converted java using camel bindy
             Step 2 : Dozer transformation of Customer to Account
             Step 3 : Java to Json conversion using json jackson
        -->
        <endpoint id="csv2json" uri="dozer:csv2json2?sourceModel=org.acme.Customer&amp;targetModel=org.globex.Account&amp;marshalId=json&amp;unmarshalId=csv&amp;mappingFile=transformation.xml"/>
        <dataFormats>
            <bindy classType="org.acme.Customer" id="csv" type="Csv"/>
            <json id="json" library="Jackson"/>
        </dataFormats>
        <route id="_route1">
            <!-- Consume files from input directory -->
            <from id="_from1" uri="file:?fileName=customers.csv&amp;noop=true"/>
            <onException id="_onException1">
                <exception>java.lang.IllegalArgumentException</exception>
                <handled>
                    <constant>true</constant>
                </handled>
                <to id="_to1" uri="file:?fileName=csv-record-${date:now:yyyyMMdd}.txt"/>
            </onException>
            <split id="_split1">
                <tokenize token="\n"/>
                <to id="_to2" ref="csv2json"/>
                <to id="_to3" uri="file:?fileName=account-${property.CamelSplitIndex}.json"/>
            </split>
        </route>
    </camelContext>
</beans>

本地运行

$ cd file-split-transform/
$ mvn clean install
$ java -jar target/file-split-transform-1.0.0.jar

results matching ""

    No results matching ""