Delete Duplicates Using Transformer

1.Without using stage variable how can we delete the duplicates using Transformer?
2.If we will remove duplicates using Transformer then minimum how many stage variable required for this?

Questions by Aloka   answers by Aloka

Showing Answers 1 - 12 of 12 Answers


  • Aug 23rd, 2016

1. Without Stage variable we can use link partitioning method use Hash Partitioning click the check box perform sort and click the unique option.
2. Three stage variables are required one for storing the current record, one for storing the previous record and third one for the counter. The key field should be sorted before doing this operation.

  Was this answer useful?  Yes

nikhil thomas

  • Nov 1st, 2016

Transformer Stage with Partition as HASH and Click on the perform Sort with Unique

  Was this answer useful?  Yes


  • Jan 11th, 2017

Just follow this
Source --> (link sort on key field) Transformer (user LastRowInGroup() function) --> Target

  Was this answer useful?  Yes


  • Mar 27th, 2017

Take two stage variable
ST1 := if ST2 <> inputcolumn then 1 else 0
ST2 := inputcolumn
take the output of ST1 to a column "Check" and in constraints write a condition , check =1. this will filter the records with value 0

  Was this answer useful?  Yes


  • May 26th, 2017

In the transformer STAGE PROPERTIES --> select the INPUT table --> Under that select PARTITIONING --> select HASH --> Select the I/P column and select the Sort and Unique checkbox.
Compile and Run

  Was this answer useful?  Yes


  • Jul 10th, 2021

The prerequisite for this is data should be partitioned by a key and sorted.
Later, follow the below steps:
S2 = if input.column = S1 then 0 else 1
S1 = input.column
In constraint : S2=1
And, you will get the expected result.

  Was this answer useful?  Yes

Give your answer:

If you think the above answer is not correct, Please select a reason and add your answer below.


Related Answered Questions


Related Open Questions