GeekInterview.com
   Home |  Tech FAQ  |   Interview Questions |  Placement Papers |  Tech Articles |  Learn |  Freelance Projects |  Online Testing |  Geeks Talk |  Job Postings |  Knowledge Base | Site Search |  Add/Ask Question

GeekInterview.com  >  Interview Questions  >  Data Warehousing  >  Abinitio
Go To First  |  Previous Question  |  Next Question 
 Abinitio  |  Question 134 of 193    Print  
what is data skew? how can you eliminate data skew while i am using partitiion by key?

  
Total Answers and Comments: 3 Last Update: May 15, 2008     Asked by: prabhupurna 
  
 Sponsored Links

 
 Best Rated Answer

No best answer available. Please pick the good answer available or submit your answer.
September 22, 2006 16:59:14   #1  
donbuckme        

RE: what is data skew? how can you eliminate data skew...

Skew is basically the data is not evently distributed across partitions.  An uneven distribution degrades the performance of the overall execution as CPUs are sitting idle and waiting for some other partitions to finish their job with larger volumes.

AbInitio partitioning the key by hash.  Choose the key that has the most distinct values will incrase the chances of data being partitioned evently.


 
Is this answer useful? Yes | No
September 22, 2006 17:07:37   #2  
dontbuckme Member Since: September 2006   Contribution: 8    

RE: what is data skew? how can you eliminate data skew...

data not evenly partitioned.   PBE uses hash function. choose the key that has the most distinct values might improve the data being distributed evenly.


 
Is this answer useful? Yes | No
May 15, 2008 02:40:11   #3  
hsevna Member Since: May 2008   Contribution: 1    

RE: what is data skew? how can you eliminate data skew while i am using partitiion by key?

The skew of a data or flow partition is the amount by which its size deviates from the average partition size, expressed as a percentage of the largest partition:

Skew of data =(partition size - avg.partition size)*100/(size of largest partition)


 
Is this answer useful? Yes | No


 
Go To Top


 Sponsored Links

 




About Us  |   Privacy Policy  |   Terms and Conditions  |   Contact  |   Site Map  |   Add Question  |   Propose Category  |   RSS Feeds  |   Articles Sitemap  |   Site Updates  |   Add Resource

Copyright © 2005 - 2008 GeekInterview.com. All Rights Reserved
Page copy protected against web site content infringement by Copyscape