MAHESH MANAM

• Aug 16th, 2012

 

It is a automated process of building a data warehouse which involves taking the data from disparate source systems, converting them into a consistent form that can be loaded into the warehouse and performing quality checks while building the data warehouse.
It typically accounts for 70-80% of the effort in a data warehousing initiative.

If you store data in data warehouse directly it is very difficult to retrieve data from that because various types of database are going to store in warehouse.

So for better performance we are going to store data in staging in common format.

Mahendra Rajpoot

 Profile Answers by Mahendra Rajpoot 

• Sep 17th, 2014

 

Staging area may be required if you have any of the following scenarios:

Delta Loading: Your data is read incrementally from the source and you need an intermediate storage where incremental set of your data can be stored temporarily for transformation purpose
Transformation need: You need to perform data cleansing, validation etc. before consuming the data in the warehouse
De-coupling: Your processing takes lot of time and you do not want to remain connected to your source system (presumably the source system is being constantly used by the actual business users) during the entire time of your processing and, hence, prefer to just read the data from source system in one-go, disconnect from the source and then continue processing the data at your "own side"
Debugging purpose: You need not go back to your source all the time and you can troubleshoot issues (if any) from staging area alone
Failure Recovery: Source system may be transitory and the state of the data may be changing. If you encounter any upstream failure, you may not be in a position to re-extract your data as source has changed by that time. Having a local copy helps