Change Partition to Auto in Join stage

What will happen if we will change the partition to Auto in Join stage?

Questions by Aloka   answers by Aloka

Showing Answers 1 - 4 of 4 Answers


  • Jun 30th, 2016

If we change the partition it will reduce the performance of job because it will verify all partition techniques. so we prefer hash partition for join stage.


  • Jun 30th, 2016

Hi Aloka,
Join stage we used to Hash key partition by default. If you changed it auto then you may get job performance issue.
Thanks !

  Was this answer useful?  Yes

Thanks Suresh for your reply !
But when Hash partition used..It will also verify for all the partition.
It will be more helpful if you will elaborate more on this.

  Was this answer useful?  Yes


  • Feb 17th, 2017

We usually go for join stage when the source input data is huge!!! If we select Hash partitioning only then it will improve the performance of job because Hash partitioning ensures that the data with similar key column values stay on the same partition, and therefore theyre processed by same node..this reduces memory requirements because then lesser no. of records need to stay in memory at any one time!!! optimizing performance of our job...

  Was this answer useful?  Yes

Give your answer:

If you think the above answer is not correct, Please select a reason and add your answer below.


Related Answered Questions


Related Open Questions