1.How to read multiple file from sequential stage.
2.If a file doesn't arrive or doesn't exists in sequential
stage how do u handle this.
3.What do you do before taking data from source to staging
area.
4. I have a remove duplicate stage and transformer
stage.what will u do for optimizeing the performance of the
job.
Answers were Sorted based on User's Feedback
Answer / varma
Ans1 : We can use Read Method = “Specific Files” and give the full path of all the file one by one. Alternatively, you can use Read Method = “File Pattern” and specify the wild card.
Ans 2: We can control this using “Missing File Mode” Option. Values are “Ok” to skip the file and continue and “Error” to abort the job.
Ans3: It is important to make sure the meta data matches the record. It would be better to reject bad reject and collect them in “Reject Link”. This can be controlled by “Reject Mode” option. Values are “Continue”, “Fail” and “Output”. “Output” will collect the reject records in a reject link.
| Is This Answer Correct ? | 3 Yes | 0 No |
Answer / jayanthi
1)read method should be set as file pattern(for reading
single file it should be specificfile)
| Is This Answer Correct ? | 2 Yes | 0 No |
Answer / ashok
4.in case of only for duplicates removing i will use remove
duplicate stage.but in case of more logics applied i will
go to transformer.bcoz remove duplicate stage give better
performance compare to transformer
| Is This Answer Correct ? | 1 Yes | 2 No |
Answer / nagaraz
1.READ METHODE SHOULD BE TAKE SPECIFIC FILE U CAN TAKE
MULTIPLE FILE
| Is This Answer Correct ? | 0 Yes | 1 No |
Hi friends,Two input files, wants to validate only if the reference data has '0' otherwise no validation should be done..how to do this??
In my project source data comes from MAINFRAME in files.so,This time data is coming as a binary file...I know for binary data we use Complex flat file stage..I have used it also..but on 'view data' data is not coming correctly..as it in MAINFRAME.give me some ideas..
how to change left and right links in join stage?
How can we improve performance of data stage jobs?
SEQUENTIAL FILE I HAVE ONE RECORD,I WANT 100 RECORDS IN TARGET?HOW CAN WE DO THAT?PLS EXPLAIN ME AND WHAT STAGES ARE THERE?WHAT LOGIC?
guys pls tell me where we use sequence jobs exactly in realtime proj explain pls with example.
how can i abort the job if i get more than 100 errors in job log?
how will u design file watch jobs?
How many partitions does a node can have? Some body explain me clearly about the partitions and the nodes?
What is the difference between lookup and sparse lookup?
Why we need datasets ratherthan sequential files?
how to load meta data