What is the difference between lookup and sparse lookup?



What is the difference between lookup and sparse lookup?..

Answer / subhash

Normal lookup will provide data for an in-memory look up
whereas Sparse lookup will access the database directly.
Normal might provide poor performance when the reference
data is huge as it has to load large data into memory.
In such scenarios a join stage may work better(It will put
the data onto an internal dataset file for each link,
and then do the join based on the data provided). If your
stream/source data is small then a sparse lookup is
suggested
as a row is retrieved from the database for each input row
instead of the full reference data being brought into
memory.
Sparse lookup sends individual SQL statement for every
incoming row (If stream/source data is huge you can imagine
the number
of times it has to hit DB and hence the down side on
performance). It can be used when you want to get the next
sequence
number from your database (Again expensive overhead on your
job as noted before). Also note that sparse lookup is only
available for DB2 and Oracle. Normal Lookup stage can have
multiple reference link but Sparse can only have one
reference
link. When normal is used it is a good practice to choose
less volume data as reference data.

Is This Answer Correct ?    15 Yes 1 No

Post New Answer

More Data Stage Interview Questions

can any 1 put a difference between datastage 7.5 and 8.0 all concepts plz r mail 2 me anu4u09@gmail.com

4 Answers   IBM,


Hi All, Could you please let me know whether Datastage server and px supports oredb? OREDB:It's a Oracle Retail Embedded Database.Previousely It was called as Acumate data base. It's a multidimensional database. Please help me on this issue ASAP. Thanks in advance Ashok

1 Answers   Accenture,


What are the types of views in datastage director?

0 Answers  


Hi dude, i/p eno ename date edate deptno 1 x 10thjuly 30july A 2 y 10th aug 30 aug B 1 x 1aug ------ A 3 z 1jan ------ c 2 y 1sep ------ B o/p is eno ename date edate deptno 1 x 1aug ------ A 2 y 1sep ------ B 3 z 1jan ------ c Here edate is System generates we don't know that value. Please design the job by using SCD stage and Sql query. If any body know this answer please tel me. Tanks.

1 Answers   Amdocs,


what is mapping lookup

1 Answers  






input Name Salay Sam 10 Sam 30 Sam 20 Ram 40 Ram 50 Output should be Name Salary Count Sam 10 3 Ram 40 2 get min salary and count of name . OutPut 2 Name Seq Sam 1 Sam 2 Sam 3 Ram 1 Ram 2

1 Answers   HCL,


What are the types of containers and how to create them?

0 Answers  


data stores in which location while using data set stage as the target?

1 Answers   IBM, TCS,


A signal has a wavelength of 1 micro min in air.how far can the front of the wave travel during 1000periods?

0 Answers  


What is the command line function to import and export the ds jobs?

0 Answers  


Hi Vijay here For Four CPU's how many nodes will required?

4 Answers   TCS,


How to read the length of word in unix?

0 Answers   CTS,


Categories