Remove duplicate lines

Cleaning up lists

Lists collected from spreadsheets, logs, exports and copy-paste often repeat entries: the same email from two sign-up forms, the same URL in two crawls, the same keyword in several research files. This tool keeps the first occurrence of each line and removes later copies, without reordering anything, so the list stays in the order you gave it.

Ignore case treats Apple and apple as the same line. Trim spaces ignores spaces at the start and end of lines, a common invisible difference after copying from spreadsheets. Remove empty lines also drops blank lines.

Worked example

ana@example.com
bob@example.com
Ana@example.com
bob@example.com

With ignore case on, the result is ana@example.com and bob@example.com: 4 lines in, 2 out, 2 removed. Without it, Ana@example.com would be kept as a different line.

Sorted output

To sort as well, run the result through the sort lines tool.

The page reads your text line by line (Windows and Unix line endings alike), builds a comparison key for each line by trimming and/or lowercasing it as the options say, and keeps a line only the first time its key appears. The kept lines are output in their original order, and the table shows how many lines went in, came out and were removed. Everything runs in your browser.