Expert: Bill DeShawn - 10/26/2009

Question
Hi bill,

Nice to speak to you again.
How are you?

I have a  question about lisp again.

i need a little code thats open a text file
read it and filter out double line's


An example is below


"1","Current layer","User selects","User selects","0","0"
"2","Current layer","User selects","User selects","0","0"
"3","Current layer","User selects","User selects","0","0"
"4","Current layer","User selects","User selects","0","0"
"A CAP","Current layer","User selects","User selects","0","0"
"A CAP","Current layer","User selects","User selects","0","0"
"A CAP","Current layer","User selects","User selects","0","0"
"A CAP","Current layer","User selects","User selects","0","0"

The line strating with A CAP must be deleted until 1 of them is left.

In this example there are 4 double lines
In real this number can bed any number you can think of.

I hope that i provided enough info.
If you need to know more let me know.

Greetings
Don

Answer
I got some help on this one:
"stevor" submitted the following example code:

(defun file_l (fn /  fh L rl )  (if (setq Fh (open fn "r"))
  (while (setq rl (read-line Fh)) (setq L (cons rl L)) ) )
 (if fh (close fh) (progn (princ "\n no filefound: ")(prin1 fn)))
 (if L (reverse L)) )

(defun L_file (L fn / e fh1 ) (if (setq Fh1 (open fn "w"))
  (progn (foreach e L (write-line e Fh1)) (close fh1) fn)) )


Kent Cooper offered the following advice:
Are the double lines always *adjacent* to each other?

If so, even in a very large file, you could avoid wrestling with what would become a very large list in T.Willey's suggestion, by taking this approach:

Open a new file for writing, to become the list with the duplications removed.
Read the first line, write it to the new file, and save its content to a local variable.
Read each line successively, and only if its content does *not* match that variable, write it to the new file *and* save its content to that same variable, to be compared to the next line.

Whether having two files open is more overhead than building up a big list of unique lines, I don't know.  One relevant question would be whether you want to have *both* the original and the reduced versions in the end.  T.Willey's suggestion could be made to do write the resulting list of lines to a new file, or overwrite the original if you don't need both; with my suggestion, you would need to delete the original afterwards.

--
Kent Cooper

Gert responded:
i guess
the (vl-string-sort.... eliminates dublicates, so read in the file, make
a large list, sort it, write out every element of the sorted list

Then Kent said:
I like it [except that it should be (vl-sort), rather than (vl-string-sort)] -- I hadn't thought of its eliminating duplicates.  The question arises, though, whether the OP would want the resulting list in the same order as the original one.  If so, possibly some combination of (vl-sort-i) to get a list of index numbers and (vl-sort) could do that, with some further list manipulation.

I'm still trying to think of a way to do it without making a potentially large list of text lines that AutoLISP would need to "carry."  If the duplicate lines are always adjacent, my earlier suggestion would eliminate all the "heavy lifting," with all the bulk of information in external files, and Lisp would only need to have two text lines in mind at any time [the variable containing the previous line, and the latest line to compare it to].  But if they're not necessarily adjacent, there may be no way to do it without making such a list.

--
Kent Cooper

Alan Henderson wrote:
here is a quick and dirty program, no error checks or testing
(defun C:RWTF (/ F1 F2 RL Ltext)
 (setq F1 (open "FileIn.txt" "r"))
 (setq F2 (open "FileOut.txt" "a"))
 (while (setq RL (read-line F1))
   (if (not (member RL Ltext))
     (setq Ltext (append Ltext (list RL)))
   )
 )
 (foreach Ttext Ltext
   (write-line Ttext F2)
 )
 (close F1)
 (close F2)
 (princ)
)

Hopefully these samples and discussions helped you.

Keep in touch
Bill DeShawn
http://my.sterling.net/~bdeshawn