Answer / vinod upputuri

First use the seq filestage then use the copy stage after
the copy stage one link is going to agregator stage there we
will calculate the count of records after the agregator
stage use filter stage for finding uniq records then those
records are connected to the lookup as a reference link. the
main link for lookup is comming from copy stage. Then
matching records are uniq else duplicate.

Agre--->filter
| |
Seq--->copy---->Lookup---->uniq
|
|
duplicate.

In agregator use count rows.

Answer / subhash

According his logic: from aggregator stage: the output is
this: 251,2
456,2
951,1
985,1
the main data is:
251
251
456
456
951
985

If you join these two links then the output will be:
251,2
251,2
456,2
456,2
951,1
985,1

Then your are specifying that count=1 then you get the
unique records. means YOU get: 951,985


in another link count<>1 means YOU get: 251,
251,
456,
456

this is our desired out put, our out put.
the logic explained by Shiva is correct.

Answer / siva

src-->copy-->agg
| |
| |
join stage--filter--uniq
|
|
dup
this is senario ans

plz do this u get correct answer

thanks
siva

Answer / binayak mohapatra

seq-agg-copy-filt1-tgt1
Again from that agg-filt2-tgt2
In agg group=empid
        count=rowcount
        outputcount=empid
In agg column  create another column like count_empid and map it.
in filt1 count_empid>1 and map into tag 1
then  you can take input from another copy source then filter2 put a condction like count_empid=1 and map them into tgt 2

Answer / vinod upputuri

Appreciate your Answer but it was wrong..because if you are using Agregator stage it will give only unique records with record count. so how can u print all the duplicate records.


in the above senario:

input to agregator stage: 251,251,456,456,951,985
i used count rows.
so, the o/p is 251,2
456,2
951,1
985,1
if U use the Filter stage then what is the output:
O/P1:
251
456

O/P2:
951
985

Unfortunately this is not the answer we are required...

This is not as simple as U think...i implemented it before posting!!!!!

Answer / pnvnreddy

Hi Shiva ur right

---------Agg -tar1
----------/ \ -/
seq---copy--join---filter---tar2

in Agg
Agg type=count

in filter
count=1-----tar1
count>1-----tar2

Answer / shar

Hi guys

Here is the answer.

Seq-->copy--->Agg,--> Join ----->Filter -----> DS1
----------> -----> DS2

Explanation:

From seq stg to copy and
from copy send one link to aggregator for counting of rows
and other to join stage for Clubbing.
At join use input column to club. From join to filter and at
filter give conditions at where clause count = 1 output link
-0 and count =2 output link -1.
Finally use Datasets as targets DS1 and DS2.

Ok thats it.

Answer / msbharathi

- > Unique
Sq File - > Aggregate - > Transformer |
- > Duplicate

1. Read data in seq file stage.
2. In Aggregate stage group by EID and count the number of rows in Count column
3. In transformer stage create loop condition as @ITERATION<= DSLink6.Count and derivation as EID = EMPNO.
4. In unique output tab use constraint count<=1 and in duplicate output tab use constraint count>1

Answer / vinod upputuri

Unfortunately, I didn't got the logic behind the YOUR answer.

According your logic: from aggregator stage: the output is this: 251,2
456,2
951,1
985,1
the main data is:
251
251
456
456
951
985

If you join these two links then:
251
251
456
456
951
985
251,2
456,2
951,1
985,1

Then your are specifying that count=1 then you get the unique records.

in another link count<>1 means YOU get 251
456

Unfortunately this is not our desired out put, our out put is: 251
251
456
456

SO, YOUR LOGIC IS NOT ACCEPTABLE. TRY TO THINK FOR DESIRED OUTPUT !!!!

in order to get the desired out put i using the lookup-stage
in lookup i am going to lookup the out put from filter stage which is 951,985 with main data there i will specify the condition as reject:
so YOU will get the desired output!!!!!

If you can prove the above logic is wrong send me the mail!!!!

Answer / sarath

seq---->aggr--->filter---- ds1
|
-->ds2

where we use config file as parameter ?

1 Answers   IBM,

What is the function in Transformer stage can act as Rank Function in oracle

1 Answers   Reliance,

Drop duplicate records ... SOURCE LIKE .......... ID flag1 flag2 100 N Y 100 N N 100 Y N 101 Y Y 101 N Y 102 Y N 103 N N 104 Y Y 105 N N 106 N Y 102 N Y 105 Y Y in above file if any id having both the flags as "N" then that corresponding id records should be dropped, in above case o/p should be as ID flag1 flag2 101 Y Y 101 N Y 102 Y N 102 N Y 104 Y Y 106 N Y Steps to do : 1) Identified the id’s that got duplicated (both the flag values having vales “N”) 2) Look up with these id’s to existing id’s to drop .

2 Answers  

Differentiate between Join, Merge and Lookup stage?

0 Answers  

How do you load 10 different sources with 10 different layouts to 10 different tables?

1 Answers   TIAA CREF,

What is the use of hoursfromtime() function in transformer stage in datastage?

0 Answers  

HOW WILL YOU IMPLEMENT SURROGATE KEY IN SCD BY USING SURR_KEY GENERATOR,THE VALUE OF S_KEY SHOULD NOT REPEAT EVEN IF THE JOB IS COMPILED REPEATEDELY?

4 Answers   L&T,

what is diff b/w datastage 8.1,8.5,8.7?

1 Answers   IBM,

Star Schema vs. Snow Flake Schema. What is the best model for DW? Give example for both the models performance wise?

3 Answers   IBM, TCS,

What are transforms and what is the differenece between routines and transforms?

0 Answers  

What is the use of surrogate key stage?

2 Answers  

How to write a left outer join condition by using Transformer stage in server jobs? Could any one help me?

1 Answers