Thread: remove duplicate values in a source using Informatica

How to remove duplicate values in a source using Informatica?
Which transformations to use?
what is the query?

NOTE : [This question was asked by dharma]

Hi
There are 2 ways to do this and both of them are efficient.
Method 1: Sorter -Filter.
Send all the data to a sorter and , sort by all feilds that u want to remove duplicacy from . note that in the preoperties tab, select Unique .
This will select and send forward only Unique Data .

Method 2; Use an Aggregator
Use AGG Transformation and group by the keys /feilds that u want to remove duplicacy from.

Cheers
Manisha

Originally Posted by JobHelper

How to remove duplicate values in a source using Informatica?
Which transformations to use?
what is the query?

NOTE : [This question was asked by dharma]

Hi, I have some idea about that. Pls, reply whether these are correct or not
1. We can remove them at staging area, by using SQL before entering into informatica.
2. By using aggregator transformation we can remove by using group by port.

Originally Posted by JobHelper

How to remove duplicate values in a source using Informatica?
Which transformations to use?
what is the query?

NOTE : [This question was asked by dharma]

if soure is relational then we can used source qualifier transformation,
if source is flat file then we can used joiner (by using option sort) or by using Aggregator (by using Group by port) or look up Transformation.


Narasimha Reddy

How to remove duplicate rows using router and filter?

You can remove duplicates by using either select Distinct or Order by clause at the end of the SQL over ride in source qualifier

Hi
If it is a relational file u can select SELECT DISTINCT option in the source qualifier itself and can use filter transformation and specify the condition.

To Just remove duplicate values u can use distinct in Source qualifier transformation.
Or by using sorter transformation .