How can remove duplicates in a file using UNIX?
Answers were Sorted based on User's Feedback
Answer / j madhava rao
$sort -u filename will sorts the data and removes the duplicates lines from the output
| Is This Answer Correct ? | 17 Yes | 1 No |
Answer / vishwababu
You can use UNIQ to remove duplicates frm a file but it can
consider only same consecutive records.
Ex: if u hve records as
1 vishwa
1 Shruti
2 Ravi
3 Naveen
1 vishwa
and if u use UNIQ cmd, then you wil get vishwa, Ravi, Naveen
and vishwa.
So perform sort before using UNIQ so tat to avoid duplicates
| Is This Answer Correct ? | 7 Yes | 7 No |
Answer / prasad
Uniq -u File_name
It will remove duplicates records.
| Is This Answer Correct ? | 0 Yes | 2 No |
Suppose take my name "Reddyvaraprasad" From source it is coming Reddy vara prasad Need to get As Reddyvara prasad/
what is use of SDR function?
What is the Difference between Change capture stage and Difference Stage ? What are its significance individually ?
How can we perform the 2nd time extraction of client database without accepting the data which is already loaded in first time extraction?
Name the different types of Lookups in Datastage?
Hi, I am getting data from sequential file like below column1 A&B C&D A&E X&Y&Z v&x But I need the target column as like below column1 A and B C and D A and E X and Y and Z v and x
how to transfer file from one system to another system in unix? which cmd to be use?
What is the default padding character?
what is the difference between 7.1,7.5.2,8.1 versions in datastage?
i want job aborted after some records are loaded into output by using only sequential stage and dataset
A flatfile contains 200 records.I want to load first 50 records at first time running the job,second 50 records at second time running and so on,how u can develop the job?pls give the steps?
Why do we use link partitioner and link collector in datastage?