what are the best practices to extract data from flat file source which are bigger than 100 mb memory?
- 1 Answers
- 2432 Views
- Wipro, I also Faced
- E-Mail Answers
Answer / Vikash Kumar Anand
When dealing with large files (>100 MB), it's essential to use efficient methods for reading and processing data. Some best practices include:
1. Using chunking or streaming techniques to process the file in smaller parts instead of loading the entire file into memory.
2. Utilizing database management systems (DBMS) to store and manage large datasets, such as Hadoop's HDFS (Hadoop Distributed File System).
3. Leveraging ETL (Extract, Transform, Load) tools like Informatica PowerCenter for handling large data volumes.
| Is This Answer Correct ? | 0 Yes | 0 No |
What is lookup transformation?
without update strategy how to insert &update?
I am Unable to load the FixedWith FlatFile Into The Target.What Is The Reason. PLZ Help Me..
If session fails after loading 10000 records in the target,how can we load 10001 th record when we run the session in the nexttime?
How to list Top 10 salary, without using Rank Transmission?
source table have 3 records? and it is sucessfully loaded into target. and 4more records is added in to source .that means 7 records now in source. we have to load the remaining 4 records into the same trgt table with maintian top 3 records. how ?can any one give me the data flow of this logic plz?
What is the sequence generator transformation in informatica?
What are the types of maping in Getting Started Wizard?
Whats there in global repository
What is the advantage of persistent cache? When it should be used.
In real time scenario where can we use mapping parameters and variables?
In joiner, how to load Master table in Target?
Teradata 
