How to remove duplicates in transformer stage? in parallel
mode

Answer Posted / praveen sarva

STEP 1) TRANSFORMER STAGE PROPERTIES--> ADVANCED -->
EXECUTION MODE ---> PARLLEL

STEP 2) TRANSFORMER STAGE PROPERTIES --> INPUT -->
PARTITIONING--> PARTITION TYPE --> HASH ---> ENABLE SORT ---
> ENABLE UNIQUE

Simple u will get non duplicate records....

Is This Answer Correct ?    11 Yes 0 No



Post New Answer       View All Answers


Please Help Members By Posting Answers For Below Questions

Is it possible to implement parallelism in Mainframe Jobs ? If Yes how ? If no why ?

1779


How do you run datastage job from the command line?

596


What are the different kinds of views available in a datastage director?

600


what is 'reconsideration error' and how can i respond to this error and how to debug this

2117


On which Dimension Table you implemented SCD Type in your Project

1577






Describe link sort?

642


EXPLAIN SCD

960


Field,NVL,INDEX,REPLACE,TRANSLATE,COLESC

963


Hi guys, Please design a job for dis requirement with derivation(solution). my source table like dis. emp_no qualification 1 a 1 c 2 a 3 c 3 b To loaded to target like dis emp_no qualification 1 b 2 b 2 c 3 a my requirement is every employer have three qualifications i.e a,b and c. what qualification missed in source table that will be move to target systems. Hope u got it the requirement. Right Thanks.

2428


DB2 connector> transformer > sequential file Data will be exported into a csv format in a sequential file. This file will be send in a email using a sequence job. Problem here is, how to avoid sending a blank csv file? When I ran the job there are chances that it might return zero records but in the sequence job csv file is going blank. how can I avoid this? thanks

1031


Can you explain tagbatch restructure operator?

699


How do you import and export data into datastage?

699


How do you remove duplicate values in datastage?

625


How a source file is populated?

613


What are the partitioning techniques available in link partitioner?

1753