how many types of remove the duplicate records?

Answers were Sorted based on User's Feedback



how many types of remove the duplicate records?..

Answer / prabhu rathnam

1. remove duplicate stage
2. sort stage
3. copy stage

Is This Answer Correct ?    8 Yes 0 No

how many types of remove the duplicate records?..

Answer / rohit babar

There are 3 ways by which we can remove the duplicate records
1. By using remove duplicate stage. We have control here on which record we want to remove it's first or last. There is option record to retain in remove duplicate stage where we select first or last option.

2. By using sort stage. there is option allow duplicate in sort stage. We set this option true when we want duplicate records & If we want unique records we set it to the false

3. By in-line sorting. In Partition tab of any stage when we select key base partition tech. then perform sort option will enable if we check this option next 2 option will enable which is stable & unique if we select stable that means we allow duplicate records & if we select unique that means we remove duplicate records and getting unique records

Is This Answer Correct ?    5 Yes 0 No

how many types of remove the duplicate records?..

Answer / soumya

1. Sort the Data using a key column then Use 3 transformer stage Variable
SV1 = Col1
SV2 = If SV1 = SV3 then 'DUP' else 'UNQ'
SV3 (Initial value = 0)
In the constraint SV2 = 'DUP' or SV2 = 'UNQ'

2. Aggregater stage group by key col (col1) count the column
then a filter stage use count col > 1.  

Is This Answer Correct ?    1 Yes 2 No

Post New Answer

More Data Stage Interview Questions

how to run jon in unix back round process what is command use in runing a job?

2 Answers   HSBC,


what is normalization and denormalization

2 Answers   CTS,


how many types of sorting the data in data stage?

1 Answers   BoA, IBM,


what is A Datastage?

2 Answers  


how to change left and right links in join stage?

4 Answers   Cap Gemini, IBM,






Does datastage support slowly changing dimensions ?

0 Answers  


can any 1 put a difference between datastage 7.5 and 8.0 all concepts plz r mail 2 me anu4u09@gmail.com

4 Answers   IBM,


Hi, what is use of Macros,functions and Routines..? At what situation you are used. If you know the answer please explain it. Thanks.

0 Answers  


Why we need datasets ratherthan sequential files?

5 Answers  


Explain the ChangeApply stage?

11 Answers  


what is operator combinality in datastage?

1 Answers   TCS,


how to validate the jobs

2 Answers   IBM,


Categories