1. The fundamental concept of the Orchestrate
framework is the Data Set. Data Sets are the inputs and
outputs of Orchestrate operators.
2. As a concept a Data Set is like a database table,
in so far as it is a collection of identically-defined
rows. It is the only structure on which Orchestrate
operators operate. Each operator( i.e., stage) accepts
input from one Data Set and sends its output to another
3. A Data Set exists on all the processing nodes
defined for the job that is currently processing it. That
subset of rows in a Data Set that are located on a single
processing node is referred to as a "partition" of the Data
Set. Technically, a partition is a subset of the rows in a
Data Set (or File Set) earmarked for processing on the same
4. A control file is associated with each data set.
The control file contains the record schema that defines
the row structure (effectively its column definitions).
5. Within a Data Set data are stored in internal, or
1. It allows you to read data from or write data to a
2. The stage can have a single input link, a single
output link and a single reject link.
3. It only executes in parallel mode.
4. The data files and the file that lists them are
called a file set. This capability is useful because some
operating systems impose a 2 GB limit on the size of a file
and you need to distribute files among nodes to prevent
5. Only advantage of using fileset over a sequential
file is "it preserves partitioning scheme"
A dataset is a file/stage where the data can be read
directly by the DataStage, whereas a file set needs to be
converted into DataStage readable format (which happens
In simple words the data from the DataSet can be read
faster than from FileSet.
1) dataset in native format so it can view the data only internally(datastage) where as fileset is in binary format so data can be view in any where which is convert from binary to human understandable language.
2) dataset dont support reject link where as fileset support reject link.
3) dataset is copy operator fileset is import and export operator.
I have a scenario like
Deptno=10---->First record and last record
Deptno=20---->First record and last record
Deptno=30---->First record and last record
I want those first and last records from each department in
a single target. How to do this in DataStage, any one can
Thanks in advance.
eno ename esal acct1 acct2 amount1 amount2
100 suresh 10000 sbi1 sbi2 1000 2000
this is our sourse data
i would loke to disply like this
eno ename esal acct amount
100 suresh 10000 sbi1 1000
100 suresh 10000 sbi2 2000
in one scenario source flat file like
divide each 5 numbers as one column i.e
here i need
field1 field2 field3 field4
00122 00155 00562 00568
00256 00236 00145 00896
00123 00456 00789 00258
00147 00456 00258 00256
plz help me....