--- url: https://findutils.com/guides/csv-dedupe title: "Remove Duplicate Rows From a CSV by Email, ID or Any Column" description: "Dedupe a mailing list or any CSV export by the columns you choose, keep the first or last copy, and get the removed rows as their own file. Runs in your browser." category: converters content_type: guide guide_type: subtopic cluster: data-conversion locale: en read_time: 7 status: published author: "olgunozoktas" published_at: 2026-09-26T12:00:00Z excerpt: "A newsletter export lists the same subscriber three times: once as signed up, once in capitals from a form, once from a second import. Removing duplicate rows by the email column, with case and stray spaces ignored, leaves one row per person and a separate file showing exactly what went." tag_ids: ["csv", "duplicates", "data-cleaning", "spreadsheet"] tags: ["CSV", "Duplicates", "Data Cleaning", "Spreadsheet"] primary_keyword: "remove duplicate rows from csv" secondary_keywords: ["dedupe mailing list by email", "csv remove duplicates by column", "deduplicate csv online", "keep last duplicate csv", "remove duplicate emails from csv"] tool_tag: "csv-dedupe" related_tool: "csv-dedupe" related_tools: ["csv-dedupe", "csv-diff", "csv-merger", "csv-viewer", "duplicate-line-remover", "dedupe-bookmarks"] og_image: "/images/content/guides/csv-dedupe-cover-20260926.webp" image_alt: "A conveyor carrying a row of blank white cards past a glass funnel that lifts duplicate cards into a teal tray." updated_at: "2026-09-26T12:00:00Z" --- To remove duplicate rows from a CSV, decide which columns make two rows the same record, such as `email`, then keep one row per value of those columns. FindUtils [CSV Dedupe](/convert/csv-dedupe/) does this in your browser: pick the key columns (or none, to compare whole rows), keep the first or the last copy, and download the cleaned file plus a second file holding every removed row. The kept rows stay in their original order, and the cells are written back exactly as they were. The file is not uploaded. This guide covers choosing the key, the matching options, what happens to blank keys, and the kind of duplicate the tool does not try to find. ## Why Exports Collect Duplicates CRMs, newsletter tools, form builders and shops rarely stop a record from being added twice: - **The same person signs up twice**, from two forms or two campaigns. - **Case and spacing differ.** `Ada@northwind.io` and ` ada@northwind.io ` are the same address to a mail server, but not to an exact comparison. - **An import runs twice** and appends every row again. - **An append-only log** holds one row per update, and you want the latest per customer. Sending to that list means the same person gets the same message more than once. ## How to Dedupe a CSV ### Step 1: Load the file Open [CSV Dedupe](/convert/csv-dedupe/) and drop a `.csv`, `.tsv` or `.txt` file on the Input area, choose one, or paste the text. The delimiter is detected from the first line; set it under File format if the guess is wrong. ### Step 2: Pick the key columns Pick the columns that decide a duplicate, for example `email`. Pick several for a composite key, such as `first_name` + `last_name` + `postcode`. Leave all unpicked to remove only rows that repeat in every cell. ### Step 3: Choose which copy to keep Under When a row repeats, keep, choose First row to keep the earliest occurrence of each key or Last row to keep the latest. Then set Trim whitespace and Ignore case. ### Step 4: Check and download The status line shows how many rows were read, kept and removed. Check the Kept rows and Removed duplicates tabs, then download the cleaned file and, if you want a record, the duplicates file. ## A Worked Example This list has one person twice under the same email: ```csv email,name ada@x.io,Ada bob@x.io,Bob ada@x.io,Ada Lovelace ``` With `email` as the key and the first copy kept, the cleaned file is: ```csv email,name ada@x.io,Ada bob@x.io,Bob ``` With the last copy kept, the cleaned file keeps the newer name, and Bob stays in front because kept rows never change order: ```csv email,name bob@x.io,Bob ada@x.io,Ada Lovelace ``` The duplicates file for that run holds the removed row under the same header: `ada@x.io,Ada`. ## How Trim and Ignore Case Change the Result Trim whitespace is on by default and Ignore case is off. Here is one column, `email`, with three spellings of one address: ```csv email Ada@x.io ada@x.io ADA@X.IO ``` | Trim whitespace | Ignore case | Rows kept | Why | |---|---|---|---| | On | Off (default) | 3 | The letters differ in case | | Off | On | 2 | ` ada@x.io ` keeps its spaces, so it differs from the other two | | On | On | 1 | All three reduce to `ada@x.io`; the first, `Ada@x.io`, is kept | The kept row is written exactly as it was, capitals and spaces included. The options change only the comparison. For email addresses, Ignore case is usually safe. For names, IDs and codes it usually is not: `AB-1` and `ab-1` may be different products. ## What Happens to Rows With an Empty Key? By default, a row whose key cells are all empty is kept as unique. Ten subscribers with no email address are ten different people, not one. Choose Collapse into one under Blank keys to deduplicate empty keys like any other value, which keeps one of them. A cell of only spaces counts as empty. This applies to key columns only. With no key picked, whole rows are compared, and two completely identical rows are duplicates whatever they contain. ## Exact Matching Only: What It Will Not Find CSV Dedupe removes rows whose keys match exactly after the optional trimming and case folding. It does not guess: | Row A | Row B | Duplicate? | |---|---|---| | `ada@x.io` | `ADA@x.io` | Only with Ignore case on | | `ada@x.io` | `ada@x.io ` | Yes with Trim on (the default) | | `Jon Smith` | `John Smith` | No: different text | | `+44 20 1234` | `020 1234` | No: phone formats are not normalised | | `ada@x.io` | `ada+news@x.io` | No: different addresses | Deciding that two differently written records belong to one person is fuzzy matching, and the tool does not attempt it. Rows it leaves alone stay in the cleaned file for you to review. ## CSV Dedupe vs Spreadsheet Remove Duplicates | Feature | CSV Dedupe | Spreadsheet "Remove duplicates" | |---|---|---| | Choose key columns | Yes | Yes | | Keep the last copy | Yes | No: the first occurrence is kept | | Separate file of removed rows | Yes | No: rows are deleted in place | | Ignore spaces and case | Checkboxes | Needs helper columns or formulas | | Values changed on the way in | Never: the text is written back as read | Opening a CSV can drop leading zeros or turn values into dates | | Fuzzy matching | No | No | ## Common Mistakes **Keying on a name column.** Two people can share a name. Use email or an ID, or combine name with a second column. **Keeping the first copy of an append-only log.** In a log where later rows are newer, keep the last copy. **Deduping one export when the duplicates span two.** Combine the files first with [CSV Merger](/convert/csv-merger/), then dedupe the merged file. **Using it on a plain list.** For one value per line with no header, [Duplicate Line Remover](/text/duplicate-line-remover/) is simpler. ## Use It From Code The same logic is the `csv_dedupe` tool on the FindUtils REST API and MCP server. Text sent there is processed on the server rather than in your browser. ```bash curl -X POST https://api.findutils.com/api/tools/csv-dedupe/execute \ -H "Content-Type: application/json" \ -d '{"csv":"email,name\nada@x.io,Ada\nbob@x.io,Bob\nADA@x.io,Ada L\n","keys":["email"],"keep":"last","ignore_case":true}' ``` Other arguments: `trim`, `blank_keys` (`distinct` or `collapse`), `delimiter` and `header`. The response holds `cleaned_csv`, `duplicates_csv`, the counts and a preview of up to 50 removed rows. Input can be up to about 10 MB. See the [API reference](/api/csv-dedupe/) and the [MCP reference](/mcp/csv-dedupe/). ## Tools Used in This Guide | Tool | Use | |---|---| | [CSV Dedupe](/convert/csv-dedupe/) | Remove duplicate rows by key columns or whole rows | | [CSV Merger](/convert/csv-merger/) | Combine several CSV files before deduplicating | | [CSV Diff](/convert/csv-diff/) | Compare the cleaned file with the original by key | | [CSV Viewer](/convert/csv-viewer/) | Read a large CSV as a table | | [Duplicate Line Remover](/text/duplicate-line-remover/) | Remove repeated lines from a plain list | | [Dedupe Bookmarks](/convert/dedupe-bookmarks/) | The same job for browser bookmarks | ## FAQ ### How do I remove duplicate emails from a CSV mailing list? Load the file in CSV Dedupe, pick the `email` column as the key, turn on Ignore case, and download the cleaned file. Trim whitespace is already on, so addresses with stray spaces match too. ### Which copy of a duplicate is kept? The first occurrence by default, or the latest when you choose Last row. Either way, the kept rows stay in their original file order. ### Does it change my data? No. Cells are written back exactly as they were read, including spaces and letter case. Only which rows are kept changes. ### Can it find near-duplicates like Jon and John? No. Matching is exact after the optional trimming and case folding. Different spellings, typos and reformatted phone numbers are different values. ### Is my file uploaded? On the page, no. The file is read and processed in your browser tab. ## Next Steps Clean your export with [CSV Dedupe](/convert/csv-dedupe/). To see exactly which rows changed between the original and the cleaned file, compare them with [CSV Diff](/convert/csv-diff/).