Kiran SS

• Jul 6th, 2007

 

We should always use small file ( i.e., file with less no. of records ) as lookup. The reason is - This file will be kept in main memory ( RAM ) from the starting to ending of the script/graph run. Hence less the file size, more performance from server.

annegrace1

 Profile Answers by annegrace1 Questions by annegrace1

• Apr 8th, 2009

 

Lookup file should be always small. If the data is growing every day, then the performance will become poor and its not wise to use bigger file as lookup. It spoils the lookup concept.

abmaahi

 Profile Answers by abmaahi Questions by abmaahi

• Oct 27th, 2010

 

less data files

nagarjuna

• Jul 21st, 2011

 

When we want to combine large amount of data with very small amount of data we can use lookup

rakesh

• Aug 3rd, 2011

 

lookup file should always small,as it will kept in main memory. So, to improve performance of server..the file with less no. of records should be used as a lookup.

madhu922.m

 Profile Answers by madhu922.m Questions by madhu922.m

• Dec 21st, 2011

 

Lookup file contain a small data records. Lookup file is not connected to other graph components. Graph has lookup file the processing of graph is quick and more efficient. The lookup file data is kept in main memory.

siva sankar pulivarthi

 Profile Answers by siva sankar pulivarthi 

• Dec 27th, 2011

 

Lookup File should be of small size or less number records.

This is because, when the graph is invoked, all the data in the lookup file file will be loaded into memory..This means..if the data in the lookup file is too large memory consumption by the lookup file will be too large. If the lookup file is small, it consumes less memory. If the memory used by lookup file is large, memory available for the other components of the graph will be decreased. This results all other components will have less memory and while they performing some transformations like sort,join etc.. they cant do in-memory operations, instead they need to fetch the data from harddisk to memory and perform operations, hence the process will be delayed.

So, for this reason, we have to use a optimum size of lookup file(Not too large).

Hope this answer helps you.

------------
Siva Sankar Pulivarthi

Ram

• Jan 9th, 2012

 

When we say lookup, it can be in serial or it can in MFS and generally based on the application logic we decide what kind of lookup it needs to be. And again we need to think on the natire of the lookup, wheater it is staic then we can with the Serail lookup which can be of less data and if the lookup is dynamic then it is suggested to go for the MFS lookup and the size of the MFS lookup will be huge.
Regards,
Ram