I have a case where i got a results file with the following pattern:
path:pattern found for example
./user/home/file1:this is a game in other words when i searched for some string i got the file and the line it found it.
Problem is sometimes i have multiple cases in the same file so i would like to remove the duplicates files (the cases would be different so it's not possible).
Any help or ideas are appreciated :)
End results is to turn this:
/user/home/desktop/file1:this is a game /user/home/desktop/file1:what kind of game /user/home/desktop/file1:fast action game into just the first results found without losing all the rest of the data in the file.
Update1:
So the actual file looks like this:
/user/home/desktop/file1:this is a game /user/home/desktop/file1:what kind of game /user/home/desktop/file1:fast action game /user/home/desktop/file2:a game /user/home/desktop/file3:of game /user/home/desktop/file4:fast game i'm looking to get rid of the multiple occurrences in the same file so it should look like this:
/user/home/desktop/file1:this is a game /user/home/desktop/file2:a game /user/home/desktop/file3:of game /user/home/desktop/file4:fast game 3 Answers
You could use sort -u:
grep pattern files | sort -t: -u -k1,1 -t:- use : as the delimiter-k1,1- sort based on the first field only-u- removed duplicates (based on the first field)
This will retain just one occurrence of files, removing any duplicates.
For your example, this is the output you get:
/user/home/desktop/file1:this is a game In case you are looking for multiple distinct matches with a file, then:
grep pattern files | sort -u Are you aware of the multiplicity switch in grep? This is an excerpt from the manpage:
-m NUM, --max-count=NUM Stop reading a file after NUM matching lines. If the input is standard input from a regular file, and NUM matching lines are output, grep ensures that the standard input is positioned to just after the last matching line before exiting, regardless of the presence of trailing context lines. This enables a calling process to resume a search. When grep stops after NUM matching lines, it outputs any trailing context lines. When the -c or --count option is also used, grep does not output a count greater than NUM. When the -v or --invert-match option is also used, grep stops after outputting NUM non-matching lines. So, using grep -m 1 "pattern" files you can limit the amount of results per file to one.
In case raw file names are sufficient, one can use grep pattern -l, where -l option is documented as "print only names of FILEs with selected lines". But it turns out that each file is printed only once, even when multiple lines are matched inside.
The resulting output in your case would be:
/user/home/desktop/file2 /user/home/desktop/file3 /user/home/desktop/file4 
