vanderwal.net

Eight Rounds of Link Clean-up

I completed this round of links clean-up of my links page, which was the eighth pass. I went from 943 links on the page to 866 after removals. I have added a few new links from my usual laissez-faire finding methods.

The process honed into the Python link checker outputing a markdown file with sections for: Broken; Blocked or access denied; Timed out; SSL / certificate errors; Connection errors; Working, but redirected. I have the script put the related links that have been flagged into tables in these sections.

After the fifth or sixth pass I added an additional column to the tables in the markdown output note for “Action Decisions” where I put my annotation for: Keep; Delete; new URL (with the link); and update (for the redirected links section). After this eighth pass I added a small table at the top of the markdown output from the Python script link checker with each of these four decisions at the top. I had a couple of days between the sixth run of the check and the seventh and eighth (the last check was the post final clean-up check). Going through what remained on the seventh pass (mostly the 90 redirections) I swore I had more decision action types I had used, so having the list in the note I use for my manual assessment seemed really helpful.

This Task was a Sad Task

I’ve run a version of this links page for about 30 years. It has largely been an act of accretion. There have been many links that have gone dead and things no longer living on the web (or no longer living period). My link checker has done a decent job until about three or four years ago when my local version of Python didn’t mesh well with the older BeautifulSoup package. A couple of years ago when I got them in alignment there were quite a few false positives. But the task also had many links that had gone away.

This current update to approach (see Links Weren’t Linking :: Off the Top :: vanderwal.net) drastically improved link checking, but also pointed to a fragile state of the web. Not only the cause of massively increased friction to run a link checker (a couple of orders of magnitude increase in crawler and bot traffic that is non-organic and non-productive). This traffic isn’t out of malice, but ignorance, poor thinking, and poor planning (a common aggregation that is a central consumable to build from is better than thousands of rogue entities).

I have many “archive” links that I grey out where the site is mostly functioning still, but not been updated in years. Many of these I found in this current run are now completely gone.

There are many personal sites and blogs gone. But, also many subject focussed sites in urban planning & cities focussed that are gone, along with many in the Food & Coffee & Travel section, Productivity / PKM / Systems, and the Enterprise / KM / Learning sections. So many good resources, mostly from individuals who are crafting, honing, and adding incredibly good and smart insights that are now gone.

Some of the redirects had gone from personal insights to machine-generated content with factual errors in abundance or ad driven content, which are pretty much indistinguishable from the former.

I’m Looking to Replace Lost Sources with Good Human Content

I started blogging in 2001 to put things out into the world to have conversation and share things from my personal world to my work world. I found blogging is an utterly insanely good way of finding people of like minds and interests. I found the fellow curious travelers who wanted to learn more and understand the small to the vast interconnections of systems, cultures, and diversities of understanding in this wonderful world of “other possibilities” that surrounds us. I write and share only what I personally create out of my mind to understand what I think and find cracks that inspire more curiosity. The nodes on the web that are blogs worked through good search, which is now mostly gone as Google seems to have lost interest and there isn’t really a viable replacement for that widening gap.

I’m deeply looking for replacements in the sections where I lost sites that provided good solid insights and sparked not only understanding but fact-based inspirations to understand more and incite more exploration.

Many of the people and voices that over the years drifted off from posting to their own sites I deeply miss. Some have drifted to silos that are (have become) horrible places for sharing information as they are really difficult to do anything positive with that information and what has been shared. Decent town square sites / services and plazas for people to digitally gather and share and interact have deteriorated and/or been manufactured by those running them to be problematic spaces by intent. Some I don’t go to at all, some I wander through, and some I wish their leadership cared (or had an inkling of understanding) to get back to something that was much better.

My Links Page is Largely in Decent Shape

My links pages has long been my loose tethering to others and other web spots that had information, people, and ideas of interest. It is for thin strands of interest as a weak attractor in a Model of Attraction sense. I can click a link and attract something to my screen of interest. People and their thoughts are drawn closer. These links are now mostly working after cleaning up more than 400 of them.