What are the major differences between datastage 8.1 and datastage 8.5?
How do we know the no of processors in a job? Is there any specific calculation for this.
What are these terms used for in datastage - downstream and upstream?
upstream means data channel from source system to ETL.
there are verity of source system (RDBMS, Flat Files, XML, ODBC).this data is integrated by ETL.this is input to ETL.
same as upstream.
Downstream means data generated from ETL.
Could you plz site examples for upstream and downstream applications
Fetching last row from a particular column of sequential file
How to fetch the last row from a particular column.. Input file may be sequential file...
There are 2 ways to fetch the last row of any file if the number of records in the file is not known before run time:
1) Use the tail stage and run the tail stage in sequential mode
2) Use the lastrow() function in the transformer stage
Develop a job source seq file--> Transformer--> output stageIn the transformer write a stage variable as rowcount with the following derivationGoto DSfunctions click on DSGetLinkI...
What is the purpose of the copy stage:
What is the uses of the copy stage with out copies input link to output link datasetsand it can have any purposes pls send me with example
Besides used for making copies of the input, copy stage is also helpful for achieving 1) dropping columns between stages 2) to change column names 3) if the scenario is to end the job flow directly f...
copy stage purpose is to avoid the unwanted columns in output table means in a table u have 10 columns but u want 5 columns then you use copy stage to get which columns you want
Datastage job scenario question
Input file a contains12345678910input file b contains6789101112131415output file x contains12345output file y contains678910output file z contains1112131415how can we do in this in a single ds job in px ?....Could you please give me the logic to implement ???
Add an extra column colA and colB to the files A and B respectively. Let the value for colsA be a for all the rows in file A and the value for colB be b in file B(using the column generator stage).Now...
You take two seq files and club the files in to funnel stage as continuous funnel and take sort stage and filter the duplicates and the n transformer stage write the constraints over there and give 3 outputs
That's all its very simple method !!!
How to handle date convertions in datastage? Convert a mm/dd/yyyy format to yyyy-dd-mm?
We use a) "iconv" function - internal convertion. B) "oconv" function - external convertion. Function to convert mm/dd/yyyy format to yyyy-dd-mm is oconv(iconv(filedname,"d/mdy[2,2,4]"),"d-mdy[2,2,4]")
DateToString(date,"%yyyy:%mm:%dd")
Using datetostring function in transformer stage
example
DateToString(inputcolumnname(ex Dob),"%yyyy/%mm/%dd")
Oconv(Iconv(Filedname,"D/MDY[2,2,4]"),"D-YDM[4,2,2]")
JJ, you are wrong.
Apt_ Dump_Score
By using this only we can find.
1 conductor process
3 Section leader process
3 player process ( modify and filter are combined)
1 source sequential file
1 target sequential file
Total : 9
What is the size of your database ?
Please answer this question
your database size is the sum of all datafiles,tempfiles,redologs. so check the sum from dba_data_files,dba_temp_files and v$logfile
It Depends about the size,Example My Development Database size is 6TB and UAT Server is 12 TB and Production is ~25TB.
In datastage 8.1 what is the limit of the file size? Is there limit from # of rows and # of field perspective for file extract to be fed in profile stage
In 8.0/8.1 it uses parser which requires the entire XML document to be loaded into memory, so that youre limited by the amount of available memory. In 8.5 it uses a really clever streaming app
I have 3 jobs a,b & c, which are dependent each other, I want to run a & C jobs daily and b job run only on Sunday. How can I do it?
set the cron entry
This can be done in Director -->Add to Scheduler.
Source table name a a b b b c c d in source table data like this but I want traget table like this name count a 1 a 2 b 1 b 2 b 3 c 1 c 2 d 1 pls any one one solve this........
1)Sequential file->Take Sequential File and Import The flat file which consist of (AABBBCCD)data. 2)Transform Stage-->Take Transformer Stage and create two Stage variables In StageVariable1 Write t...
first count the rows using transformer stage (stage variables)stgvar ===dslink source,,,stgvar 1===count the rows
stgvar 3====concatenate stgvar stgvar1 you will get the result
Sequential file with duplicate records
A sequential file has 8 records with one column, below are the values in the column separated by space,1 1 2 2 3 4 5 6in a parallel job after reading the sequential file 2 more sequential files should be created, one with duplicate records and the other without duplicates.File 1 records separated by...
Its very simple: 1. Introduce a sort stage very next to sequential file, 2. Select a property (key change column) in sort stage and you can assign 0-Unique or 1- duplicate or viceversa as you wish. ...
After source sequential we can use sort stage with dump_key in which 0 is assigned to duplicate record and 1 is assigned to non duplicate record after sort stage we can use transformer stage in whic...
How to connect two stages which do not have any common columns between them?
Use transformer or generate extra rows and using column generator and make same number of columns in 2 inputs and join.
We can connect the two job activity stages without any column.
If are given a list of .Txt files and asked to read only the first 3 files using seq file stage. How will you do it?
In sequential file we can take a single file by using the file as Specified file.
But we can take the more than one file use File Pattern with different file names.
Metadata must be same.
HI,
In sequential file we can take a single file byusing the file as Specifyed file.
But we can take the more than one file use File Pattern with different file names.
Metadata must be same.
If you have numerical+characters data in the source, how will you load only character data to the target? Which functions will you use in transformer stage?
example : raje123ndh456ar
now it is converted to character string and we can load only character.Code
convert(0123456789,,raje123ndh456ar) = rajendhar
use alpha function.
Exp-->alpha(123vinay567)
Ans-->vinay
i thing this is your requeriment..
Why copy stage is better than transformer?
Incase of only dropping columns(without any transformations or business rules), we can go for copy stage instead of transformer.But can anyone tell me exactly why copy stage is better in performance than transformer?
Transformer is a heavier stage as apart from the orchestrate we do have the C++ operators for various derivations. Right from compilation to execution a Transformer will always take more time than a copy stage which is a simple straight forward passive(if we can call) stage
Copy stage is a passive stage and Transformer is Active and it involves the process usage and copy does not . So its good to go with copy since it takes less time then transformer to propagate the columns over the link
Conditional load of records in datastage
Source has sequential file stage in 10 records and move to transformer stage it has one output link 2 records and reject link has 5 records ? But I want remaining 3 records how to capture
You can choose O.W option in constraints, so remaining records you will get.
In the transformer Stage Constraints we can define the constraints where this records can get to which link so that the missing records can get catched.
Which partition is best in performance level?
Partition by Round Robin
Hash
Which partition followed by joiner,merge. Lookup,remove duplicate
Key partition is required and data should be sorted before all these stage while using