I want load the duplicate records to seperate table

[Geek_Guest]

Question asked by visitor lokendra

i import a flat file as a source, it contains duplicate records. i want to load the duplicate records in to one table and remaing are loaded in to another target table?for this which transformations i need to use?

[jdoshi110]

Question asked by visitor lokendra

i import a flat file as a source, it contains duplicate records. i want to load the duplicate records in to one table and remaing are loaded in to another target table?for this which transformations i need to use?

You need two transformation

1) Look up transformation : which will look on the target1 table which is having non duplicated data.

2) Router transformation which will route duplicated data to another taget2 table.

[gaurav.chahal]

use routertransformation with two target tables

[sambawipro]

Hi

By using expression transformation we can load the duplcate records into anothere target table simply we can call error table.

Ex - I have 10 duplicate records , we can send the 1 record to valid table and remaining to error table.

Then we can add one new field in that we can mention the logic like first record shoul be 'Y' and remaining should be 'N' and then by use router transfermation we can send the valid record (Y) into val table and error records (N) into error table.


Cheers
Samba

[sarojibm]

first you use an aggregrate and group it by all the columns from source. then add one more column in aggregrator which actualy aggregrate the count of each group.then after that either using filter or router according to ur need..send the duplicate records where count is greater than 1.
Then in next step make tat table as lookup and sending all the duplicates records to final table.