GeekInterview.com
  I am new, Sign me up!
 
GeekInterview.com  >  Interview Questions  >  Data Warehousing  >  DataStage
Go To First  |  Previous Question  |  Next Question 
 DataStage  |  Question 381 of 390    Print  
Lookup Stage Partitioning
With a Lookup stage the input stage has 1million records and reference link has 1million records. What is the best Partition method and why?


  
Total Answers and Comments: 4 Last Update: October 25, 2009     Asked by: rajivkumar23us 
  
 Sponsored Links

 
 Best Rated Answer

No best answer available. Please pick the good answer available or submit your answer.
March 13, 2009 13:22:09   #1  
rajivkumar23us Member Since: March 2009   Contribution: 3    

RE: Lookup Stage Partitioning
Entire Partition.

1 Million rows aren't much to handle for DS engine since Lookup Stage requires key columns to be in the same partition with million rows from each side Entire Partition can be used.

Correct me if I am wrong.

 
Is this answer useful? Yes | No
March 19, 2009 09:00:17   #2  
anujk_arora Member Since: March 2009   Contribution: 1    

RE: Lookup Stage Partitioning
I believe it depends upon the system configuration as the data in the reference link is loaded into the main Memory while processing. If the total consumption of the memory passing through the reference link is above the total capacity of the machine in that case we should look for alternatives. It could be hash partitioning on reference link along with the main link. Having said this it all depends upon the logic. So i believe nothing is ideal for the lookup stage paritioning although entire is a default technique but should be changed if the situation demands.
 
Is this answer useful? Yes | No
March 26, 2009 21:27:41   #3  
gopalanrr Member Since: March 2009   Contribution: 1    

RE: Lookup Stage Partitioning

When the reference link is too high in terms of records fetched you should be performing a join instead of performing a lookup


 
Is this answer useful? Yes | No
October 24, 2009 11:11:34   #4  
Kumar Shourav Member Since: October 2009   Contribution: 1    

RE: Lookup Stage Partitioning

Better to use JOIN or MERGE stage in place of Lookup. If input data is low in compare to reference data then Lookup is recommendable. Assumption is 1:100 or more.


 
Is this answer useful? Yes | No


 
Go To Top


 Sponsored Links

 
About Us -  Privacy Policy -  Terms and Conditions -  Contact -  Ask Question -  Propose Category -  Site Updates 

Copyright © 2005 - 2009 GeekInterview.com. All Rights Reserved

Page copy protected against web site content infringement by Copyscape