Sharmila

• Oct 20th, 2006

 

I thought we could identify dupilcates by using rank transformation.

suresh

• Oct 23rd, 2006

 

can u explain what are the steps for identifying the duplicates with the help of rank transformation.

phani

• Oct 23rd, 2006

 

can u explain detail?

Ramesh

• Oct 26th, 2006

 

Hi,

      You can write SQL override in the source qualifier (to eliminate duplicates). For that we can use distinct keyword.

For example : consider a table dept(dept_no, dept_name) and having duplicate records in that. then write this following query in the Source Qualifier sql over ride.

1)select distinct(deptno),deptname from dept_test;

2)select avg(deptno),deptname from dept_test
group by deptname;

if you want to have only duplicate records, then write the following query in the Source Qualifier SQL Override,

select distinct(deptno),deptname from dept_test a where deptno in(
select deptno from dept_test b
group by deptno
having count(1)>1)

shivakumar

• Oct 30th, 2006

 

i think we cant select duplicates  from rank transformation,if it is possible means explain how to do it,

n k rajkumar

• Oct 30th, 2006

 

We can get the duplicate records by using the rank transformation.

sravan

• Nov 3rd, 2006

 

we can aso use sorter transfermation.seect distinct one check box.

Anindya1281

 Profile Answers by Anindya1281 

• May 10th, 2010

 

This can be done using SQL query in case of relational sources
variables in an expression in case of reltn and flat file srcs

In case of Infa 8.x we have an inbuilt function called MD5 which can be used to find duplicates
src

id name
1 ABC
2 PQR
1 XYZ

src - sorter(based on id) -->exp --> filter --> tgt
in exp:

id
name
dup_chk = IIF(prev_id = id,1,0)
prev_id = id

Filter in all records where dup_chk = 1

Thanks
Anindya