I have a serial graph with below sequence: i/p -> sort(key1)->rollup(key1)->sort(key2)->rollup(key2)->normalise->o/p. How do we improvise this graph?

gouse

• Nov 20th, 2015

By reducing number of sort components

raja

• Feb 10th, 2015

It depends on your input data.If you specify some of your input data will be more helpful. My suggestion is instead of using two roll up and two sort keys try to use it only one rollup with key1 and key2

Answer Question  Select Best Answer

I have a master file with user personal details and around 50k users. Each day a transaction file with about 100million records come in and we need to update transaction file with user details present in master file. What is a good approach for this?

kalyanvadlamudi2kn

 Profile Answers by kalyanvadlamudi2kn Questions by kalyanvadlamudi2kn

• Jan 8th, 2014

1)I partition the data first 2)Remove unwanted fields if applicable 3)Followed by sorting on the matching key in first data if it can fit in memory without data spill as it small and other i wil...

sriram

• Dec 11th, 2013

Join with old master and new master file. you will get matched record those are not going to update. unused records are new records ,which are loaded into table.

Answer Question  Select Best Answer