How do you remove duplicate records from a file and capture the rejected records in a separate file.

Questions by infinity

Showing Answers 1 - 13 of 13 Answers

Dharmendra

  • Aug 3rd, 2007
 

In parallel DS, we can use Duplicate removal Stage to avoid the deplicate records.
where as in server DS, we have to use lookup

  Was this answer useful?  Yes

Fanguna

  • Nov 19th, 2007
 

Other than remove duplicate stage ,we can also use aggregator stage to count the number of records exist for the key columns.If more than one record exist for the key column,then they are considered as duplicate records and using transformer we can set a stage variable as 'COUNT' and check if 'COUNT>1'.If so,using a constraint, reject that duplicate records into reject file.

  Was this answer useful?  Yes

nagoosk

  • May 29th, 2008
 

sort the records using the key field.In sort stage put "key change column = true".Then zero will be assigned to the duplicate records.then put a condition as which is record is zero then send it to reject link

  Was this answer useful?  Yes

Give your answer:

If you think the above answer is not correct, Please select a reason and add your answer below.

 

Related Answered Questions

 

Related Open Questions