How to read multiple files at different directories using a
single Sequential File stage in DS Enterprise Edition
- 6 Answers
- 24014 Views
- TCS, I also Faced
- E-Mail Answers
Answers were Sorted based on User's Feedback
If all files having same Metadata,
Select Read method as 'specific file(s)'
then u Can add as many files as you want to run by selecting 'file' property from the 'available properties to add'.
It locks like below:
File= /dsg/work/temp/Employee1.txt
File= /dsg/work/Employee2.txt
File= /dsg/Employee3.txt
Read Method= Specific file(s)
| Is This Answer Correct ? | 14 Yes | 0 No |
Answer / venkata ramana
To read given number of files, one can use follow below
In the sequential file stage, Use
Read Method =File Pattern
File Pattern=`ls /* | head -<Number of files to be read>`
For example, I had 5 files, with similar pattern (as
date*.txt), in working folder say "/data/working/". If we
want to read three files out of these five files, one can
use following command:
`ls /data/working/date*.txt | head -3`
Just modified Venkat's Answer based on tested job.
| Is This Answer Correct ? | 0 Yes | 0 No |
Answer / satyam
sequential file stage in that option is available you can observe that, ok i will send how to process later ok
| Is This Answer Correct ? | 4 Yes | 5 No |
Answer / pooja
To read multiple files in a sequential file follow the
below process..
1. Enable "Allow Multiple Instances" in the Job Properties.
2. Add a job parameter in the sequential file stage where
the input file needs to be defined.
3. Compile and execute the job.
4. During execution, a value needs to be passed to the job
parameter..here pass the input file path placed in
different directories.
5. Execute the job for all the multiple files using the
same job by passing the input file path for each run.
6. An invocation id needs to be provided for each run. This
id indicates the no. of times the job has been executed.
Can be observed in the log.
| Is This Answer Correct ? | 5 Yes | 6 No |
Answer / venkat
To read given number of files, one can use follow below
steps:
1. In the sequential file stage, Use Read Method as
Specific File(s).
2. Now in the file text field, Put below command.
`ls /* | head -<Number of files to be read>`
This will read given number of files.
For example, I had 5 files, with similar pattern (as
date*.txt), in working folder say "/data/working/". If we
want to read three files out of these five files, one can
use following command:
`ls /data/working/date*.txt | head -3`
Hope this helps.
venkat
09940692102
| Is This Answer Correct ? | 0 Yes | 7 No |
what are the types of nodes
in a job of 20 one job is very slow due to that entire job is slow how can u find out which job is slow?
difference between function and procedure...
What is the process of killing a job in datastage?
I/P --- ID Value 1 AB 2 ABC 3 ADE 4 A O/p --- ID Value 1 A 1 B 2 A 2 B 2 C 3 A 3 D 3 E 4 A
Explain the ChangeApply stage?
What is the difference between datastage and datastage tx?
What is difference between 8.1 , 8.5 and 9.1 ?
Hi guys, Design job sequence, we have 3 sources, in that 1st source in abort then only run the remaining sources.. How please design the job. Thanks.
source 1123445 I WANT OUTPUT AS DUPLICATES TO TARGET1 LIKE TARGET1 1144 NON-DUPLICATES TO TARGET2 TARGET2 235
Hai..,This Kiran . how to immlement SCD's through sequential file.please explain briefly.if nt ask me i will explain ,what i know.
create a job that splits the data in the Jobs.txt file into four output files. You will direct the data to the different output files using constraints. • Job name: JobLevels • Source file: Jobs.txt • Target file 1: LowLevelJobs.txt − min_lvl between 0 and 25 inclusive. − Same column types and headings as Jobs.txt. − Include column names in the first line of the output file. − Job description column should be preceded by the string “Job Title:” and embedded within square brackets. For example, if the job description is “Designer”, the derived value is: “Job Title: [Designer]”. • Target file 2: MidLevelJobs.txt − min_lvl between 26 and 100 inclusive. − Same format and derivations as Target file 1. • Target file 3: HighLevelJobs.txt − min_lvl between 101 and 500 inclusive. − Same format and derivations as Target file 1. • Rejects file: JobRejects.txt − min_lvl is out of range, i.e., below 0 or above 500. − This file has only two columns: job_id and reject_desc. − reject_desc is a variable-length text field, maximum length 100. It should contain a string of the form: “Level out of range: <min_lvl>”, where <min_lvl> is the value in the min_lvl field. My Question is how do you write the stage variable for reject rows.
Teradata 
