Scarcity in an Ocean of Knowledge

I wrote down the three things I wanted to work on today; one being a blog post on how I automated adding content to the blog that you’re currently reading. I was on a high after assembling what I thought was a simple but useful infrastructure, and I wanted to share it with the world. But then I decided to check for novelty in an attempt to not be redundant. It was then I realized the same infrastructure was assembled and documented by many individuals, some of which included a template for it on github.

I’m not writing the post anymore.

That moment highlighted one of the gripes of the information age. Searching the internet is difficult. Its even harder for the one searching to find what they’re looking for when they aren’t sure themselves. Stack overflow is a great example of this, where almost everyday you find new questions already asked and answered multiple times on the same site. Senior users and mods often hide their frustrations with this under cheeky remarks about googling it themselves. Problem is, novice developers don’t know what to search for. Search engines aren’t librarians who can help you narrow down on what exactly you’re looking for. They give you posts that exactly match what you searched for, or match some of the words in your search.

So, what then?

In all honesty, I don’t know. There are people out there who recognize this issue and are working on solutions (CROKAGE for example), but I myself don’t know the solution. Maybe its in machine learning and AI. Maybe we need internet librarians. Who knows? We’ll see.