Out of 4 mill records only 3 mill records are loaded to
target
and then job aborted. How to load only those 1 mill(not
loaded records) for next run.
This job is not sequential job, it is stand alone parallel
job.What are the possibilities available in datastage8.1?
Answers were Sorted based on User's Feedback
Answer / prasanna
Use already loaded records TGT as reference lookup,and use
lookup stage,with DROP option..and design the job.u ll get
req answer. refer
.
.
sour.......>lookup......>tgt
| Is This Answer Correct ? | 7 Yes | 4 No |
Answer / nish
there are plenty of options available.
just carefully study the scenario.
Source: 4 Mil records (doesn't change)
Target: 3 Million already loaded
Option1:
you just need to identify those 1Million pool them and then load to target.
This is clearly a case for Change Data Capture (CDC) stage.
use the Source As Before Table and Target as After.
Write those 1 Mil records based on change_code() (Deleted) to a file.
Move the contents of this file to target.
Option 2: This scenario also hints at updating the target with merge stage.use the DROP option to gather into a file and then update the target.
Option 3: Look Up being a performance concern should be your alternative.
| Is This Answer Correct ? | 2 Yes | 0 No |
Answer / venkatesh
first load the data using with surrogate stage
after that use filter stage and take that key of
surrogateand in lookup maintain the Drop option
| Is This Answer Correct ? | 2 Yes | 8 No |
How to read multiple files using a single datastage job if files have the same metadata?
How to initialize environment variables?
i have a scenario in which i/p columns areID,salary with 1,1000 2,2000 and 3,3000 i need an extra column in the o/p named avg(salary)how can i get it?
how to get sum of sal based on dept_no and then sum of all sal irrespective of dept_no in same sql. output:- 10, 200(sum of sal for dept_no 10), 5000(sum of all sal)
what is data mapping
Why we need datasets ratherthan sequential files?
Hi guys, please design job for this, MY INPUT IS COMPANY,LOCATION IBM,CHENNAI IBM,HYDRABAD IBM,PUNE IBM,BANGLOORE TCS,CHENNAI TCS,MUMBAI TCS,BANGLOORE WIPRO,HYDRABAD WIPRO,CHENNAI HSBC,PUNE MY OUTPUT IS COMPANY,LOCATION,COUNT IBM,chennai,hydrabad,pune,banglore,4 TCS,chennai,mumbai,bangloore,3 WIPRO,hydrabad,chennai,2 HSBC,pune,1 Thanks
convert yyyy mm dd to dd mm yyyy?
what is difference between migration project and integration project? can anyone explain with the example Thanks in advance
eno ename esal acct1 acct2 amount1 amount2 100 suresh 10000 sbi1 sbi2 1000 2000 this is our sourse data i would loke to disply like this eno ename esal acct amount 100 suresh 10000 sbi1 1000 100 suresh 10000 sbi2 2000
options available in sequence job to run,validate?
Hi all, can u explain header and trailer records in file? If a file has header and trailer /master-detail records how will you read it in datastage?