Duplicate Content Checker

There is no duplicate content penalty

This is the single most persistent myth in SEO and it has been denied by Google repeatedly for well over a decade. Publishing two similar pages does not trigger a sanction. Nothing gets demoted for being a copy.

What actually happens is duller and, in its own way, worse, because nothing tells you it is happening.

• Consolidation. Google picks one URL from the set and indexes that one. The others get filtered out of results. The URL it picks is frequently not the one you would have picked, and any links pointing at the discarded versions do a lot less for you than they would have if they all pointed at one page.
• Wasted crawl. On a big site, thousands of near-identical URLs — faceted navigation, printer versions, session parameters — burn the crawl budget that should be going to pages you actually want fetched. On a fifty-page site this does not matter at all.
• Split signals. Two pages half-ranking for the same query instead of one page ranking properly. This is the version that costs real money, and it is usually self-inflicted rather than a technical accident.

The fix is almost never deletion. It is a canonical tag pointing the duplicates at the version you want, or a 301 if the duplicate has no reason to exist, or — for two pages that are genuinely about different things but read the same — a rewrite so they actually are different.

How the overlap is found

Both texts are split into words, punctuation stripped and case folded. Every window of consecutive words — five by default — becomes a shingle, and the shingle sets are compared. That is the standard approach to near-duplicate detection and it is robust to a reordered paragraph in a way that a plain string comparison is not.

The score is the Jaccard index of the two shingle sets: shingles in both, divided by shingles in either. Containment is reported separately because Jaccard punishes a length difference — a 200-word passage copied verbatim into a 2,000-word article scores low on Jaccard and very high on containment, and for a plagiarism question containment is the number you want.

Matching shingles are then used as seeds and extended word by word for as long as both texts agree, which recovers the longest shared run. That is why a 40-word identical paragraph shows up as one highlighted passage rather than 36 overlapping five-word ones.

This tool or the similarity checker?

Use this one when the question is "will these two pages compete in search" — two service pages for neighbouring towns, a category page and its filtered variant, an article you syndicated. The output you want is the highlighted passages, because those are the parts to rewrite.

Use the text similarity checker when the question is "how far did this draft move from that one" — checking a writer's rewrite, comparing two versions of a page, confirming a translation round-trip changed something. It scores the whole text three different ways instead of hunting for copied runs.