Remove duplicate lines
Remove duplicate lines. Case-sensitive by default, keeping the first occurrence; optional ignore-case, trim-before-compare, Unicode normalisation and drop-blank switches, plus an ordered multi-step pipeline.
Inputs and parameters
input— Input text (one item per line)steps— Step order (comma-separated; blank = this op only)ignoreCase— Ignore case (keep first occurrence only)trimForDedup— Trim line ends before comparingdropBlank— Drop blank linesnormalize— Unicode normalisationlineEnding— Output line endingincludeBom— Include UTF-8 BOM on download (legacy Notepad)previewOnly— Preview only (never overwrite input)
Real example
a
A
a
B
A → default
→ a
A
B (a / A / a keeps a and A by default; "ignore case" keeps only a)Limits and failure handling
Whole-line comparison only: no fuzzy matching, no AI rewriting, no subtitle timecodes. Unicode is not normalised by default, so e+combining-acute (NFD) and é (NFC) are distinct lines — opt in to normalisation to merge them. Limits: 2 MiB / 100 000 lines, rejected before processing and cancellable. Combining marks and ZWJ emoji are preserved by default.
FAQ
Which occurrence is kept?
Always the FIRST occurrence, with its original line text; comparison keys (lowercase, trimmed, normalised) never rewrite the output.
Are blank lines removed?
Not by default; repeated consecutive blanks collapse to one. Check "drop blank lines" to remove all of them.