GeekInterview.com
  I am new, Sign me up!
 
GeekInterview.com  >  Interview Questions  >  Data Warehousing  >  Abinitio
Go To First  |  Previous Question  |  Next Question 
 Abinitio  |  Question 186 of 196    Print  
Join on partitioned flow
If I have 2 files containing field file1(A,B,C) and file2(A,B,D), if we partition both the files on key A using partition by key and pass the output to join component, if the join key is (A,B) will it join or not and WHY?


  
Total Answers and Comments: 7 Last Update: July 17, 2009     Asked by: abinitio17 
  
 Sponsored Links

 
 Best Rated Answer

No best answer available. Please pick the good answer available or submit your answer.
July 01, 2008 17:00:52   #1  
Puneet123 Member Since: July 2008   Contribution: 1    

RE: Join on partitioned flow
Partition component divides the data into different partitions depending upon the key. Join component expect data to be in a ordered flow if "Input must be sorted" is checked.
In this case join will not going fail but it will not give the correct output.

 
Is this answer useful? Yes | No
July 24, 2008 07:10:10   #2  
sixto.dsilva Member Since: March 2008   Contribution: 59    

RE: Join on partitioned flow

Key is always important in Join component else you many not get the desired result.In abinitio everything is key based if the key is wrong everything can go wrong but the graph will run successfully. sometime you many not get the result atall.


 
Is this answer useful? Yes | No
August 13, 2008 05:41:59   #3  
srinivas.rao.etl Member Since: August 2008   Contribution: 2    

RE: Join on partitioned flow

.dbc : Database Connectivity - In input table specify db version host location user name password etc.

.cfg : Server Connectivity


 
Is this answer useful? Yes | No
January 16, 2009 06:13:15   #4  
Nayak_AbIntio Member Since: January 2009   Contribution: 3    

RE: Join on partitioned flow
I believe Join component expect data to be in a ordered flow if you select Input must be sorted as checked so that the input to JOIN will be a ordered set of data.
Then I believe the join results would be as expected.

Anyone pls comment if thinks with this the expected output wont be there and if so why?

 
Is this answer useful? Yes | No
January 27, 2009 06:30:43   #5  
anujaja Member Since: July 2007   Contribution: 6    

RE: Join on partitioned flow
Yes you can join and the can get the desired result
 
Is this answer useful? Yes | No
March 25, 2009 11:04:36   #6  
Subhra Dhar Member Since: March 2009   Contribution: 3    

RE: Join on partitioned flow
I do not think the join output would be correct. The partition key fields for the two input streams should be same as the join key fields in the join component otherwise the data from stream 1 would be partitioned in a different way than data from stream 2 and won't find all matches in the join component.
 
Is this answer useful? Yes | No
July 16, 2009 15:16:29   #7  
vss34 Member Since: July 2009   Contribution: 1    

RE: Join on partitioned flow
The partition key and join key do NOT have to be the exact same. In order to join properly you just have to make sure the records being compared are in the same partition.

So if the partition key is broader than the join key (which it is in this case since the partition key is just field A and the join key is A and B) then the join will work fine as long as you sort the data after the partition or make it an in-memory join. For example all records on both datasets with a value of 1 for field A will be placed in the same partition regardless of the value of field B. So then values for field A B as (1 X) where X is any value on both datasets will join up correctly since they will be in the same partition.

If the partition key is narrower than the join key (for example the partition key is A and B and the join key is just A) then the join will most likely not work correctly since you cannot guarantee the hashing algorithm of partition by key will place the proper records in the same partition.

 
Is this answer useful? Yes | No


 
Go To Top


 Sponsored Links

 
About Us -  Privacy Policy -  Terms and Conditions -  Contact -  Ask Question -  Propose Category -  Site Updates 

Copyright © 2005 - 2009 GeekInterview.com. All Rights Reserved

Page copy protected against web site content infringement by Copyscape