[NTLUG:Discuss] Combining data from multiple files

[Kenneth Loafman]

Michael P wrote:
>>Two hypothetical questions:  (If number 2 is too difficult, I'll just go
>>with #1 and fix it further from there.)
>>
>>#1.  I have two text files that have similar info but want to combine
>>the two, with no duplicate entries.  Is there a single command for that?
>>
>>#2. I have three text files that have similar info but want to combine
>>the three into one, with no duplicate entries.  Is there a single
>>command for that?
>>
>>(They each just have a series of 3 or 4 line entries but neither of them
>>are all inclusive.  I want one file to have all entries but no
>>duplicates.)
>>
> 
> 
> I'm a little too sleepy to work up the full command but what you are
> looking for is the command "sort" or maybe a combination of cat, sort, and
> uniq.


Lets take two lists, A & B, same format:

To eliminate duplicates:
     cat A B | sort | uniq -u

To eliminate singletons:
     cat A B | sort | uniq -d

To find singletons in in A only:
     cat A B B | sort | uniq -u

Similarly for any other set operation...

...Ken