Remove Duplicate Lines

Keep one of each line and drop the repeats.

FREE - NO SIGN-UP - RUNS ON YOUR DEVICE

Compared in your tab, never uploaded

  • 2counts returned
  • Sortoptional
  • Trimoptional
  • 0uploads
Specification

Remove Duplicate Lines at a glance

Input
One line per item
Output
Unique lines, and two counts
Method
Local comparison, case optional
Options
Sorting and trimming, optional
Price
Free
Account required
None
Where processing happens
On your device
How to

How to use Remove Duplicate Lines

  1. 1

    Paste the list

    Paste one item per line.

  2. 2

    Set the options

    Choose trimming, case and sorting.

  3. 3

    Take the result

    Copy the lines, check the counts.

Questions

Frequently asked questions

How do I know it removed the right ones?

Two counts come back with the list: how many lines were removed and how many remain. If the numbers surprise you, trimming or case matching is usually the reason.

Are Apple and apple the same line?

Only if case normalization is switched on. Left off, capitals make two lines different, which is what you want for things like codes and passwords.

What about spaces at the end of a line?

A trailing space makes two otherwise identical lines different. Turn trimming on and surrounding spaces are ignored before the comparison.

Is the list uploaded?

No. The comparison runs in this browser tab. Nothing is sent to a processing server and nothing is stored in an account.

Background

Lists that dedupe cleanly

Paste the list, one item to a line. Repeats come out and the unique lines stay. You are told how many were removed and how many remain, so the result is checkable.

  • Export the list one item to a line first; a comma-separated blob has to be split before repeats can be found.
  • Trimming catches the invisible difference a stray space makes, which is the usual reason a duplicate survives.
  • Leave case matching off when capitals carry meaning, as they do in codes and identifiers.
  • Read the two counts against what you expected. A drop far bigger than you guessed usually means the wrong option is on.

Full specification

Specification of the Remove Duplicate Lines tool
InputOne line per item, with cleanup choices.
OutputThe unique lines, plus the removed and remaining counts.
MethodA local set comparison, with case normalization available.
OptionsSort the result, and trim surrounding spaces before the comparison.
PriceFree. Supported by clearly labelled ads outside the working area.
Account requiredNo.
Where processing happensIn your browser tab. Your input is never uploaded.
Next step

Keep going with the same files

All text tools: Text tools on SPOTTOOLS