how many types of remove the duplicate records?
- 3 Answers
- 10085 Views
- Caterpillar, I also Faced
- E-Mail Answers
Answers were Sorted based on User's Feedback
Answer / prabhu rathnam
1. remove duplicate stage
2. sort stage
3. copy stage
| Is This Answer Correct ? | 8 Yes | 0 No |
Answer / rohit babar
There are 3 ways by which we can remove the duplicate records
1. By using remove duplicate stage. We have control here on which record we want to remove it's first or last. There is option record to retain in remove duplicate stage where we select first or last option.
2. By using sort stage. there is option allow duplicate in sort stage. We set this option true when we want duplicate records & If we want unique records we set it to the false
3. By in-line sorting. In Partition tab of any stage when we select key base partition tech. then perform sort option will enable if we check this option next 2 option will enable which is stable & unique if we select stable that means we allow duplicate records & if we select unique that means we remove duplicate records and getting unique records
| Is This Answer Correct ? | 5 Yes | 0 No |
Answer / soumya
1. Sort the Data using a key column then Use 3 transformer stage Variable
SV1 = Col1
SV2 = If SV1 = SV3 then 'DUP' else 'UNQ'
SV3 (Initial value = 0)
In the constraint SV2 = 'DUP' or SV2 = 'UNQ'
2. Aggregater stage group by key col (col1) count the column
then a filter stage use count col > 1.
| Is This Answer Correct ? | 1 Yes | 2 No |
1.what is stagearea?what is stage variable? 2.this is my source source:id, name target:id, name 100, murty 100,madan we have three duplicate records for the id column,how can we getthe source record? 100,madan 100,saran
j1,j2,j3 jobs r runing iwant to run j1,j3 after j2 how to do this using sequencer? 2)j1,j2,j3 jobs r runing iwant to run j1,j2,j3 how to do this?
Which is the best institute in Hyderabad to learn OBIEE and DataStage?Please tell me faculty name also
How can we improve performance of data stage jobs?
How can u execute the sql query through unix? What is the Primary key for Dimension table? what is the primary key for Fact table?
What is audit table?
Lookup constraints
In Informatica,for the table I can find coreesponding dependent mappings.Likewise can I find the dependent jobs with all the information by using the table name
hi, 1)totally how many jobs created in ur project 2)what is ur datamart size & Dwh size 3)How secure ur project 4)could u plz tell be about testing process after complete the job will run sucessfully or not in Datastage
what is stage is used for below Input columns: dept|mgr|employee|salary Output columns: mgr|count of employee per mgr|avg salary per dept note: each dept has one mgr and each mgr has many employees
What is orabulk stage?
i have seq file that contents 10 million records load to target any data base.. in that case it takes lot of time for loading..how do performance tuning in that situation...?
Teradata 
