[NTLUG:Discuss] Why would a command stop working?

[Bob Byron]

Rick,

Just to make sure you are the entries are unique, you could just try 
surrounding each line with /'s (slashes).  That would help to insure
that they aren't different due to a space at the end.

A command like this:
cat file4 | sed -e 's/.*/\/&\//'

That would help you to visually inspect the file to insure the lines
truly are or are not unique.

Bob

----- Original Message ----- 
From: "Rick Matthews" <Rick at Matthews.net>
To: <discuss at ntlug.org>
Sent: Sunday, December 30, 2001 10:29 PM
Subject: RE: [NTLUG:Discuss] Why would a command stop working?


> ----- Val W. Harris wrote: -----
> 
> > Did the processes generating the lines in file1 change?
> >     If so, perhaps there's a stray blank at the end of a line.
> 
> I don't control the processes that generate file1. I apologize for not
> being more specific before. I receive 2 files from external sources in a
> common format, and I maintain a third file locally (in the same format).
> The script currently combines them as follows:
> 
> cat file1 file2 file3 | sort | uniq > file4
> 
> As Steve Baker questioned earlier, my test this evening was with over 3
> meg of data. I cut it down to about 50 lines and 'cat file1 | sort |
> uniq > file2'  worked properly.
> 
> Apparently one of my input files contains some garbage. These should be
> straight text files and they are being sorted by the full line (no
> options used with sort or uniq). How can I validate the format of the
> input files prior to processing? (I need to check to see if there is a
> grep option to select only text lines...)?
> 
> > Are you sorting entire lines or just one field of the line?
> 
> Entire lines.
> 
> > Is there some reason your script doesn't just do?:
> >       sort -u file1 > file2
> 
> Learning curve, and multiple input files. Shoot, I was tickled pink when
> I found that I could use the cat|sort|uniq method! I've since learned of
> the unique option in sort (merge, too), in fact, I also tested with
> the -u option earlier. Using the 3.5 meg file the following two command
> lines produced identical results, with the output file still containing
> duplicates:
> 
> cat file1 | sort | uniq > file2
> sort -u file1 > file2
> 
> Thanks for your help!
> 
> Rick Matthews
> 
> 
> _______________________________________________
> http://www.ntlug.org/mailman/listinfo/discuss