1. Categories >> Software >> Data Warehouse >> Data Stage
  2. Suggest New Category

Out of 4 mill records only 3 mill records are loaded to
target
and then job aborted. How to load only those 1 mill(not
loaded records) for next run.
This job is not sequential job, it is stand alone parallel
job.What are the possibilities available in datastage8.1?

Question Posted / nish

Answer Posted / nish

there are plenty of options available.
just carefully study the scenario.

Source: 4 Mil records (doesn't change)
Target: 3 Million already loaded
Option1:
you just need to identify those 1Million pool them and then load to target.

This is clearly a case for Change Data Capture (CDC) stage.
use the Source As Before Table and Target as After.
Write those 1 Mil records based on change_code() (Deleted) to a file.
Move the contents of this file to target.

Option 2: This scenario also hints at updating the target with merge stage.use the DROP option to gather into a file and then update the target.

Option 3: Look Up being a performance concern should be your alternative.

Is This Answer Correct ?    2 Yes 0 No



Post New Answer       View All Answers


Please Help Members By Posting Answers For Below Questions

How can we improve the performance in datastage?

654

What all are the different way to run a job?

642

How do you generate sequence number in datastage?

625

How do u convert the columns to rows in datastage?

694

what is stage is used for below Input columns: dept|mgr|employee|salary Output columns: mgr|count of employee per mgr|avg salary per dept note: each dept has one mgr and each mgr has many employees

664




Could anyone give brief explanation bout datastage admin

1973

i WANTED TO USE THE RANGE LOOKUP SCENARIO IN DATASTAGE 7.5.2 SRVER JOB.i HAVE A DATE FIELD IN SOURCE AND I SHOULD MATCH IT WITH A FIELD IN LOOKUP FILE.BUT,THE FIELDS SHOULD MATCH EVEN THOUGH THERE IS SOME RANGE.CAN SOMEONE TELL ME HOW CAN I DO THAT. THANKS

3315

What is the differentiate between data file and descriptor file?

863

Distinguish between informatica & datastage. Which one would you choose and why?

585

What are the repository tables in datastage?

828

What is the difference between the local container and shared container?

651

how to delete one row in target dataset

2588

What can we do with datastage director?

675

How do you import and export data into datastage?

703

What are the stages in datastage?

639