How to remove duplicates in transformer stage? in parallel
mode
- 6 Answers
- 15196 Views
- Syntel, TCS, I also Faced
- E-Mail Answers
Answers were Sorted based on User's Feedback
Answer / kiran
partition the data by key and sort the data and click on
unique value. This will automatically delete duplicate
data.
| Is This Answer Correct ? | 20 Yes | 3 No |
Answer / praveen sarva
STEP 1) TRANSFORMER STAGE PROPERTIES--> ADVANCED -->
EXECUTION MODE ---> PARLLEL
STEP 2) TRANSFORMER STAGE PROPERTIES --> INPUT -->
PARTITIONING--> PARTITION TYPE --> HASH ---> ENABLE SORT ---
> ENABLE UNIQUE
Simple u will get non duplicate records....
| Is This Answer Correct ? | 11 Yes | 0 No |
Answer / kiran
i am not sure who marked my answer as wrong. Can you please
be responsible enough to state why its wrong?
| Is This Answer Correct ? | 1 Yes | 0 No |
Answer / satya
run u r job in sequencial mode and sort the source data
then play with stage variable's in Transformer.
because in parallel mode data is partioned .
| Is This Answer Correct ? | 1 Yes | 1 No |
Answer / prasad
Take 2 Stage variables in transformer stage
sV1 =Column_Name
sV2 =if Column_Name=sV1 Then 0 Else 1
put it constraint sV2=1 (only will get unique records)
if u want duplicates sV2=0
| Is This Answer Correct ? | 0 Yes | 1 No |
Answer / santhosh
go to transformer stage properties->input->define any kind of partition over there and enable perform sort check box....
n also define the particular column need to be sorted..
it gives the sorted column out view...
| Is This Answer Correct ? | 1 Yes | 6 No |
Explain entity, attribute and relationship in datastage?
A signal has a wavelength of 1 micro min in air.how far can the front of the wave travel during 1000periods?
How to remove blank spaces from data?
How can you find out whether datastage process is running or not in unix?
what is push and pull technique??? I want to two seq files using push technique import in my desktop what i will do????
I have source like "abcdefghijklmnopqrs", how to change 10th character from j to z in datastage?
How to implement complex jobs in data stage?
How to convert alpha Numeric values to alpha using functions?
How to LOG 'unmatched Master' records and 'Reject Updates' in log files using MERGE stage?
Define oconv () and iconv () functions in datastage?
Can aggregator and transformer stages use to sort the data? How ?
what is flow of project?
Teradata 
