Z

Remove Duplicate Lines

Strip duplicate lines from a list, keeping the first occurrence. Optional case-insensitive and trim modes.

Runs in your browser — files never leave your device

Result (0 lines)

How it works

Paste a list and every repeated line disappears instantly, keeping the first occurrence of each and leaving the rest of the order untouched. The Result box shows a live count of the lines that remain, and Copy grabs the cleaned list. Nothing is sorted or rewritten — removing repeats is the only change.

By default a duplicate means an exact, character-for-character repeat of a whole line — case and whitespace included, so apple, Apple and apple with a trailing space are three different lines. The two checkboxes widen that definition. Trim each line strips leading and trailing whitespace before comparing (and the output lines are emitted trimmed). Ignore case compares lowercased copies, while the surviving line keeps the casing of its first occurrence.

A concrete example: pasting the four lines apple, Apple, apple with a trailing space, and apple again leaves three lines by default — apple, Apple and the trailing-space variant — because only the exact repeat was removed. Turn on Trim each line and the list collapses to apple and Apple. Add Ignore case and a single apple remains.

This is the standard first pass when cleaning real-world data: mailing lists merged from several exports, keyword lists for SEO or ads, IDs copied out of a CSV column, hostnames from an inventory, or repetitive log lines you want reduced to the distinct messages before reading them. Deduplicating also keeps version-controlled lists such as .gitignore files short and their diffs readable.

Details that matter: blank lines are lines too, so several of them collapse into one blank rather than vanishing entirely. Comparison is literal — it will not catch two lines that differ by an invisible character between words, such as a non-breaking space where the other line has a normal one. Matching uses a hash set in a single pass, so even very large lists dedupe as fast as you can paste them.

Frequently asked questions

What exactly counts as a duplicate?
By default, an exact whole-line repeat — every character must match, including spaces and capitalization. A line with one trailing space is a different line, and apple is not a duplicate of Apple. Turn on Trim each line and Ignore case to loosen the comparison.
Which copy is kept, and does the order change?
The first occurrence of each line survives and every later repeat is dropped. Surviving lines stay in their original order — the tool never sorts. If you want a sorted unique list, run the result through the Sort Lines tool afterwards.
With Ignore case on, which casing wins?
The casing of the first occurrence. If your list reads iPhone, IPHONE, iphone, the output is just iPhone — the later two compare equal after lowercasing, so they are removed and the first spelling is what remains.
Does Trim each line change the output text?
Yes — lines are trimmed before comparison and the trimmed versions are what appears in the output. With the option off, whitespace is preserved exactly, and lines that differ only in indentation or trailing spaces count as distinct.
Are blank lines removed?
They are deduplicated like any other line: the first empty line is kept and later empty lines are removed. A list with many blank separators therefore comes out with a single blank — the tool never strips every blank line outright.
How large a list can it handle?
Each line is checked against a hash set in a single pass, so the work grows linearly with the number of lines and results appear as you type, even for lists tens of thousands of lines long. Mixed CRLF, CR and LF line endings are all handled; output uses LF.
Is my text uploaded?
No. Deduplication runs entirely in your browser as you type — nothing is sent to a server, and the same logic powers the ZoolTools mobile app via shared test vectors.

Related tools