Remove Duplicate Lines
Remove duplicate lines from text while keeping the order.
Use on any page (bookmarklet)
Want Remove Duplicate Lines without leaving the page you're on? Drag the button below to your bookmarks bar, then click it on any website to open Remove Duplicate Lines right there — it runs entirely in your browser.
Use responsibly: a bookmarklet runs on whatever page you click it on. Avoid sensitive sites such as online banking, payment, or healthcare pages — you run it at your own risk. Everything is processed locally and no data is sent anywhere. See our Terms.
- Show your bookmarks bar if it's hidden — Ctrl+Shift+B (⌘+Shift+B on Mac).
- Drag the button above onto the bookmarks bar.
- Open any website and click the bookmark — the Remove Duplicate Lines panel appears in the top-right corner. Use ✛ to move it between corners, or ›/‹ to tuck it against the edge and pull it back out.
- Click the bookmark again (or the ✕) to close it.
Note: a few sites with strict security policies may block bookmarklets.
Examples
Remove repeated entries
Input
apple banana apple cherry banana
Output
apple banana cherry
Ignore case
Input
Apple apple APPLE
Output
Apple
About this tool
This free online tool removes duplicate lines from any text or list while keeping the first occurrence and the original order. Paste a list of emails, URLs, keywords, or any line-separated data and get back a clean, de-duplicated version. Everything runs in your browser with no upload, so your data stays private.
How to use
- Paste your lines into the input box (or click Sample to try it).
- Optionally turn on Ignore case or Trim lines to control how duplicates are matched.
- Copy the de-duplicated result from the output box.
Why order-preserving matters
The classic way to deduplicate a list is to sort it and collapse neighbouring matches. That works, but it silently destroys whatever order your data carried — and for ranked search results, a priority backlog, or a chronological log, the order was the information. This tool keeps the first occurrence exactly where it was and removes only the later repeats, so it is safe to run on data whose sequence means something.
When identical-looking lines don't match
The most common support question about any deduplication tool is why two obviously identical lines both survived. The answer is essentially always an invisible difference:
- Trailing spaces — routine in anything pasted from a spreadsheet or PDF. Turn on Trim lines.
- Case differences — Apple and apple are distinct strings until you turn on Ignore case.
- Non-breaking spaces — U+00A0 arrives with text copied from web pages and looks exactly like a space without being one. Replace it first using find and replace.
Merging two lists
To union two lists — two subscriber exports, two keyword sets, yesterday and today's results — paste one after the other and deduplicate the combined text. Entries present in both collapse into one, keeping the position from whichever list you pasted first. That single step replaces a spreadsheet formula and works on lists far larger than are comfortable to scroll.
Common uses
Cleaning email or subscriber lists before an import, de-duplicating keyword research pulled from several sources, tidying a CSV column, collapsing repeated log entries to see distinct errors, and building a unique set from two overlapping exports. Pair it with remove empty lines to drop the gaps and sort lines if you want the result alphabetised afterwards.
Frequently asked questions
Does it keep the original order?
Yes. The first occurrence of each line stays exactly where it was and only later repeats are dropped. This matters more than it sounds: the common alternative — sorting a list and collapsing neighbours — destroys any ranking your data carried. Order-preserving deduplication is safe to run on search results, priority lists, and chronological logs.
Why are two lines that look identical not being matched?
Almost always because of something invisible. Trailing spaces are the usual cause, which 'Trim lines' fixes. Beyond that: a non-breaking space (U+00A0) copied from a web page looks exactly like a normal space but isn't one, and text from different sources can use different line endings. If two lines look the same on screen but survive deduplication, one of them holds a character you can't see.
Can it ignore case?
Yes. 'Ignore case' treats Apple and apple as the same line and keeps the first one encountered — so the survivor's capitalisation depends on the order of your input, not on which form is more common. For email addresses this is usually correct, since the domain part is case-insensitive.
What does 'Trim lines' do?
It ignores leading and trailing whitespace when comparing, so ' apple' and 'apple' count as the same line, and it trims the lines that survive. Enable it for anything pasted out of a spreadsheet or PDF, where invisible trailing spaces are routine and would otherwise defeat matching entirely.
Should I sort before or after removing duplicates?
After, if you want a sorted unique list — deduplicate first to preserve which occurrence wins, then sort. Sorting first also works arithmetically but throws away the original order, so you lose the ability to keep 'the first one that appeared'. If order doesn't matter, either sequence gives the same set.
Does it remove blank lines too?
Only duplicate blank lines — the first one is kept, since it's a line like any other. To strip all blank lines, run the remove empty lines tool instead, either before or after this one.
How do I merge two lists into one unique set?
Paste both lists one after the other into the input and deduplicate the combined text. Entries appearing in both collapse to a single copy, with the position from whichever list you pasted first. This is the quickest way to union two subscriber exports or keyword sets without a spreadsheet.
Is my text uploaded anywhere?
No. Everything runs in your browser with no upload, so your data never leaves your device — which matters when the list is customer emails or internal records.
Learn more
Related tools
Regex Tester
Test regular expressions with live match highlighting, groups, and replace.
Word Counter
Count words, characters, sentences, and reading time.
Case Converter
Convert text between upper, lower, title, camel, and snake case.
Character Counter
Count characters and stay under tweet, SMS, and meta tag limits — live.