The Quick And The Dead: Examining Fifteen Years Of Links

November - 2021

I'm migrating my site to a new framework (Next.js) and content storage system (MDX files in my Grimoire). I wrote some scripts to find dead links. Figured I'd use the opportunity to pull some stats as well.

The Breakdown

  • Site Page: 1367
  • External Links: 4117

Links Per Year

YearLinksLiveDead% Dead
200572541825%
2006139994029%
2007106792725%
20081581292918%
20091691303923%
201165491625%
20126051915%
20131791483117%
201491771415%
201572551723%
2016151417%
2017880-
201917170-
20202,3852,2601255%
20215815156611%
Total4,1173,68543212%

In 2020, I started automatically pulling in the links I visit each day which is where there's such a bump. There are actually a bunch more, but I've taken them out for time being.

I'm surprised by the number of dead links in 2021, and, really 2020. Want to look at that more to figure out if some's up with my methodology or if those numbers are really legit.

Status Codes

CodeStatusBefore RedirectsAfter Redirects
N/AUnreachable6983
200OK2,7743,685
204No Content1010
301Moved Permanently880-
302Found139-
303See Other5-
307Temporary Redirect12-
308Permanent Redirect24-
400Bad Request2323
401Unauthorized11
403Forbidden8096
404Not Found90198
410Gone24
451Unavailable For Legal Reasons-1
500Internal Server Error36
503Service Unavailable59
525SSL Handshake Failed-1

Other Things To Look At

  • Add the number of posts per year
  • Add number of pages with links per year
  • List the number of distinct domains showing the top ten
  • Show number of original http and https links pear year

Wrapping Up

The scripts I used to find, scrub, and pull the link details are here

And, yeah. It's actually sixteen years of links, but fifteen sounds better in the title.