Memory requirement of graph

How can I calculate the total memory requirement of a graph?

Questions by abinitio17

Showing Answers 1 - 6 of 6 Answers

dhanoo_p

  • Jun 23rd, 2008
 

You can roughly calculate memory requirement as:

1. Each partition of a component uses:
     ~ 7 MB + max-core (if any)

2. Add size of lookup files used in phase (if multiple components use same lookup, only count it once)

3. Multiply by degree of parallelism. Add up all components in a phase; that is how much memory is used in that phase.

4. (Total memory requirement of a graph) > (the largest-memory phase in the graph).

robmuza

  • Jul 10th, 2008
 

When we did some benchmarking we found that it was closer to ~8MB + MAX_CORE (if any) per component and add in the file sizes.

Basically, I agree with the existing answer.

Just remember, don't use "in-memory" components if you think the total memory will exceed your available memory.  This will severely degrade performance.

  Was this answer useful?  Yes

Give your answer:

If you think the above answer is not correct, Please select a reason and add your answer below.

 

Related Answered Questions

 

Related Open Questions