如果想將檔案內重複的資料刪除, 在 Linux 下可以用 sort 及 uniq 指令完成, 例如以下 example.txt 檔案內容是:
# cat example.txt
this is a testing.
abc def
abc def
this is a testing.
this is a testing.
abc def
abc def
this is a testing.
先用 sort 將檔案內容排序, 再用 uniq 刪除重複的資料:
# sort example.txt | uniq
abc def
this is a testing.
abc def
this is a testing.
如果想將過濾了的內容儲存, 可以這樣:
# sort example.txt | uniq > example02.txt
# cat example02.txt
abc def
this is a testing.
# cat example02.txt
abc def
this is a testing.