rajivkumar23us

 Profile Answers by rajivkumar23us Questions by rajivkumar23us

• Mar 13th, 2009

 

Entire Partition.

1 Million rows aren't much to handle for DS engine, since Lookup Stage requires key columns to be in the same partition, with million rows from each side, Entire Partition can be used.

Correct me if I am wrong.

anujk_arora

 Profile Answers by anujk_arora 

• Mar 19th, 2009

 

I believe it depends upon the system configuration as the data in the reference link is loaded into the main Memory while processing. If the total consumption of the memory passing through the reference link is above the total capacity of the machine, in that case we should look for alternatives. It could be hash partitioning on reference link along with the main link. Having said this it all depends upon the logic. So, i believe, nothing is ideal for the lookup stage paritioning although entire is a default technique but should be changed if the situation demands.

gopalanrr

 Profile Answers by gopalanrr 

• Mar 26th, 2009

 

When the reference link is too high in terms of records fetched, you should be performing a join instead of performing a lookup

Kumar Shourav

 Profile Answers by Kumar Shourav 

• Oct 24th, 2009

 

Better to use JOIN or MERGE stage in place of Lookup. If input data is low in compare to reference data then Lookup is recommendable. Assumption is 1:100 or more.