A flat file contains 200 records. I want to load first 50
records at first time running the job, second 50 records at
second time running and so on, how u can develop this job?
1st Way:
1. Add 'row number' column in Seq File stage, so that each
record has a number associated with it.
2. Add a job param with which we can provide the number of
record from where we want to run the job. We can pass this
either using Sequence Start LOOP(List type variables-
50,100,150,200) or by shell script.
3. In the tfm, use a stage variable to run only from the
record number till 50 records by counting each record.
2nd way:
Design the job like this:
1. Add 'row number' column in Seq File stage, so that each
record has a number associated with it.
2. Use filter stage and write the conditions like this:
a. row number column<=50(in 1st link to load the records
in target file/database)
b. row number column>50 (in 2nd link to load the records
in the file with the same name as input file name, in
overwrite mode)
So, first time when your job runs first 50 records will be
loaded in the target and same time the input file records
are overwritten with records next first 50 records i.e. 51
to 200.
2nd time when your job runs first 50 records(i.e. 51-100)
will be loaded in the target and same time the input file
records are overwritten with records next first 50 records
i.e. 101 to 200.
And so on, all 50-50 records will be loaded in each run to
the target
| Is This Answer Correct ? | 8 Yes | 1 No |
How to exclude first and last lines while reading data into a sequential file(having some 1000 records).I guess probably by using unix filter option but not sure which to use
why dataset ?
sed,awk,head
i have seq file that contents 10 million records load to target any data base.. in that case it takes lot of time for loading..how do performance tuning in that situation...?
A flatfile contains 200 records.I want to load first 50 records at first time running the job,second 50 records at second time running and so on,how u can develop the job?pls give the steps?
Can you explain engine tier in information server?
Thanks to all people who are posting their comments...
there are two schemas x and y are there. some data is in x schema. i want to use that in y schema..how can i use? please give some possibilities
What is the use of datastage director?
I have a source like file it have Number of records and i want to load without first and last records in target?Datastage?
What is the use of datastage designer?
How to convert RGB Value to Hexadecimal values in datastage?