Newsletter #17 - Next Generation Personal Knowledge Management

Sébastien Dubois / July 23, 2021

16 min read

Welcome to the 17th edition of DeveloPassion's newsletter.

A few more people have joined recently. I'm really grateful that you're interested in what I'm sharing over here!

As usual, I'd like to ask all of you to help me out a bit. If you find this newsletter interesting, then please do take a bit of time to share the link on social media: https://mailchi.mp/fb661753d54a/developassion-newsletter. With your help, others will also get a chance to discover and enjoy reading it. As an added benefit, it'll also motivate me to continue the experience!

Don't forget that you can also follow me on Twitter.

Dev Concepts News

As you probably know if you've been following me, Dev Concepts, is a very important project for me. I would love to deliver the books faster, but crafting words takes time; it cannot be rushed, and it's okay. I'm enjoying the experience, and that's what matters for now!

The more I spend time on this project (300+ hours so far!), the more I'm convinced that what I'm creating is unique. I still haven't found any other book/series that covers all of the subjects that I'm going to include in Dev Concepts. I'm confident that it should end up being a success, even if takes me a while to fully deliver on the promise.

The hardest part for me is marketing, which I basically have zero experience with. I'm currently going through a copywriting course, which is really interesting. This will help me to create a better landing page in the coming weeks, and help people better understand what the series is all about. I've also been discussing and exchanging ideas with other content creators. Some gave me great insights into how to better approach marketing & sales. One of the main takeaways there for me is the fact that I will most probably increase the prices once the whole series is completed.

I'm actively looking for more reviewers. If you're interested, then please reach out to me (by replying to this mail or sending me a DM on Twitter!). I'll give you free access to the books/drafts in exchange for honest feedback & reviews that I can share publicly.

I'm writing Dev Concepts in public, publishing weekly status updates on IndieHackers, so check it out if you want to follow the project more closely: https://www.indiehackers.com/product/dev-concepts. You can already get the first volume as well as the second one; those two are already available. If you can't afford it, then send me a DM on Twitter, and I'll give you a free copy. I'm also interested in giving free access to the content in exchange for honest feedback, so please don't hesitate to reach out to me ;-)

Building the next generation of Knowledge Management systems

Since I've put my meeting management SaaS startup project on hold, I've been thinking a lot about "the next step". I'm not the kind of person who likes doing nothing. I'm a maker, and I need to keep building new things. It's tons of fun, enlightening, and full of joy for me. During the last few years, I've realized that I no longer want to be an employee or a freelancer. I want to build products.

Choosing what to build is really tough, and quite a lot of research is necessary to find actual problems worth solving for others. At this point, I have explored a number of ideas, and have one that shines much brighter than others: personal knowledge management. I still have to do my homework though...

I need to validate that people really face the same problems I have and care enough to be willing to pay for a solution. This step is crucial. Without proper validation, I'd just be building something for fun, which is not what I want to be doing right now. I want to be able to work on this problem for a decade or even longer, and the best way to do so is to build a business around it. I want to build something I truly care about and that lets me live the way I want; a so-called "lifestyle business".

While thinking about what to do next, I realized that learning, acquiring knowledge, and creating/sharing content are the three key pillars that drive me and give me energy every day. I truly have a passion for learning and sharing knowledge.

My interest in knowledge management goes way back. I've spent many years of my life consuming content, reading/watching, taking notes, learning new things, organizing content & knowledge, structuring ideas/information (both personally and professionally), and creating new content. For instance, when I started working in IT infrastructure, I was frustrated by the lack of documentation and quickly proposed to introduce Wikis. When I left that position, I had written over a thousand pages on the wiki, detailing the infrastructures we put in place, their configuration, their relationships, the processes, the security model, projects, ideas, changelogs, etc. The Wikis quickly became popular. Many teams started contributing and relied heavily on those. Knowledge bases are both powerful and empowering.

Writing is not something new for me. I've been blogging for more than ten years now, and wrote a lot throughout my life; it is my medium of choice. Recently I took the plunge and started writing non-fiction books. It was the next logical step for me. I always wanted to do it but kept pushing the idea back, until I was proposed a book deal. Finally doing it made me realize that it's something that I truly enjoy, and could dedicate my entire life to (no wonder that I ended up writing 800+ pages :p). I won't stop writing anytime soon, that's for sure! But writing books requires being able to juggle with a ton of information, making sense of it, organizing and providing it in a way that makes sense to others.

Importantly, one needs a system to curate content, acquire knowledge, structure information, and derive new insights and ideas.

Even if I'm a very organized individual, my personal knowledge management system is still surprisingly weak. For the longest time, my "system" was really basic; I didn't give it much thought. I collected literally hundreds (800+ !) of RSS feeds with Google Reader (and later Feedly) about all sorts of subjects that I enjoyed reading and learning about (technology, programming, Web development, writing, entrepreneurship, startups, leadership, management, etc). Those were blogs of people I admired such as Jeff Atwood & Joel Spolsky (creators of Stackoverflow), the creators of 37Signals/Basecamp, Steve Yegge, Henrik Kniberg, Jurgen Appelo, Chet Haase, Romain Guy, and countless others whose ideas inspired and influenced me heavily.

I used to print dozens of articles each day (no kidding, ouch!), and I read those during my train rides (~2 hours per day). I did this literally for years. I would take an article out of my bag, read it, try to make sense of the ideas, the code, the schemas, etc. I scribbled things down, added annotations, highlighted passages, drew behind the pages, etc. This was my way of "ingesting" knowledge. This helped me tremendously over the course of my career. Going through those articles helped me learn about a ton of ideas and concepts that I would otherwise not have been exposed to as part of my day-to-day job. I also read tons of books. Next to that, I bookmarked a gazillion pages that piqued my interest: videos, blog posts, essays, research papers, Web designs, source code, projects, demos, etc. At some point, I had over 10K bookmarks. Totally crazy. Of course, things got even worse over time, until it became completely unmanageable. A real content overdose.

While that basic system did help me a lot, it lacked a key element: capturing knowledge. I failed to keep my notes, my insights, my ideas. Once I read an article, I threw away the pages along with my handwritten notes or removed the bookmark. I did keep notes, but those are scattered over many notebooks. Ultimately, I only kept whatever my brain could retain. I did not track what I read either. What a waste!

Over the years, I've expanded my system. It has slowly evolved to help me retain more information. First I used mind maps, but quickly hit the limits of what I could do with those. There were too many, those were offline and not synchronized. I've also used many wiki systems like Confluence, MediaWiki, DokuWiki, Evernote, and other hierarchical content management systems. Those gave me a place to keep knowledge outside of my brain, which was a big relief. There were tons of things I found interesting and wanted to keep, but couldn't bother to memorize.

Wikis helped me a lot, but they have major limitations. A first problem is that content is organized hierarchically. With those, ideas can only live in one place at a time. You create a page, somewhere in the tree, and it stays there. You can move it around, and link to it, but it can only ever be at a single location at a time. And that is actually a big problem; it creates friction. Each time you want to extend your knowledge base, you need to decide where the new information belongs. A second problem with those is that the "unit" of knowledge is the page. Of course, a page can be very short and focus on a single idea, but having a million pages in a tree structure quickly becomes unmanageable; it doesn't make much sense. Another issue is that content cannot be reused easily. Wikis focus on being able to link between pages. It's sometimes possible to embed other pages, but not a panacea. Once you have a large body of knowledge, you need to be able to reuse small passages here and there, instead of having to duplicate information. Information duplication is a nightmare for knowledge management.

Blogging became an important way for me to capture knowledge for myself while sharing it with others. It was a clear win-win. But blogging is not perfect either. Most of the content that I've written and published as blog posts lives in silos that are completely disconnected from the rest of my personal knowledge base. I could maintain copies, but then I would have to synchronize everything manually. This is boring, and it's actually a bigger issue than that. Recently I've started cross-posting my articles on different platforms: my blog, Medium, DEV.to and Hashode. Doing so is important for me because it exposes my content to a wider audience, but I end up having to maintain many separates copies of the same content. Each time I stumble upon a mistake, I need to fix it multiple times, which is dull work.

Lately, I've been using Notion for knowledge management and decided to ditch my old wikis. Notion feels vastly superior to my previous favorite wiki system, Atlassian Confluence. Notion has the concept of a page like most wikis do, but pages contain multiple blocks instead of a single blob of information. Block can be different things (e.g., a paragraph of text, a todo list, a table, a bullet list, etc). And to me, this idea is pure gold. The Angular framework made the term "transclusion" more popular than ever. Transclusion is actually a wonderful idea. Using transclusion, we can project existing content at different locations, without having to duplicate it. Recently, Notion also introduced synced blocks, which allows keeping synchronized copies of blocks. Changing the content in one location immediately updates all the copies. Another incredibly powerful feature! Another neat capability of Notion is to publish content easily. Many products now publish their documentation through Notion given how easy it is to do. Although, I wouldn't be able to achieve my goals using Notion alone. While Notion is a really great product, it's still hierarchical in nature and thus suffers from the same limitations as other hierarchical systems. Information is still too coarse-grained for personal knowledge management. Also, I can't easily cross-post content I have in notion to other platforms.

Next to Notion, I've been exploring graph-based knowledge management systems like Obsidian, Roam Research, and Athens. Those feel much closer to what I need to create and structure my second brain (a cool term coined by Tiago Forte). Those tools are great because they provide a lot more freedom for storing/organizing knowledge and ideas. To me, they have two key features. First, they don't impose a hierarchical structure (although you may create one if you wish, which can still be useful), and second, they not only allow linking between pages but also between blocks. In addition, links are bi-directional; if you link note A with note B, then you can see on note B that A links to it, which is awesome. Those tools also expose a graph view, which allows visualizing the relationships between different notes. Finally, just like Notion, Roam Research also supports synced blocks, allowing to transclude content at different locations.

So where am I going with this story? From the looks of it, you could think "Okay, so now you have Obsidian/Roam, those are powerful; case closed!". Well, not really. Obsidian and Roam focus purely on knowledge management (which is great). But they're actually missing key elements to support my "dream" system: content/ideas curation, knowledge extraction/ingestion, and content publication.

To this date, I keep using bookmarks, mails, and tools such as Readwise/Instapaper/Pocket to capture content from various locations: articles, newsletters, podcasts, Youtube videos, handwritten notes, voice memos, etc. But no single tool allows me to "capture everything" in one place, and process everything in that same place. That's precisely what I'm missing! I'm forced to use different tools, each with a different UI/UX & philosophy. When I'm on the road and have an idea, I use a voice memo. If I'm not driving then I send myself a mail (seriously :p). When I surf the Web and find something interesting, then I either bookmark the page, mail myself a paragraph, or capture content in Readwise, Instapaper or Pocket. It's a real mess!

It is quite sad to think that we're in 2021, producing more content than ever before, and we're completely unequipped to make the best of it. As a heavy consumer, I feel that it's really important to be able to "capture" the essence of the content I consume and to be able to keep track of what I've read/watched before. So far, I haven't found anything that helps me to do so. Readwise & al focus on helping with the capture part, which is, of course, important, but not sufficient. Those tools don't help at all to organize, prioritize and extract useful information out of captured content.

And this is the starting point for what I'm thinking about building: an integrated solution that helps me to capture content and ideas from various locations, preprocessing it before "ingesting" it, extracting useful knowledge, and integrating it into my second brain, linking the new content with my existing knowledge base. And, ultimately, using my knowledge base as the single source of truth for deriving, managing, and sharing my own content from a single location.

I imagine a system that would help me with each step of the process. For instance, making it easy for me to find articles that I've captured on a subject I'm busy researching. A system that would help me focus while going through an article and extracting bits of knowledge as well as interesting ideas to further explore. I imagine a system that would automatically preprocess content that I've captured, summarizing it, tagging it, creating clusters, and allowing me to prioritize my discoveries, avoiding reading content pseudo-randomly. I imagine a system that would allow me to quickly find back everything I looked at in the past week, helping me to easily share interesting content with my audience. I imagine a system that would allow me to push my content on other platforms and keep it up to date automatically. I imagine a system that would allow me to derive new ideas from existing ones, helping me to research subjects and learn new things.

Dreaming BIG is a ton of fun, but dreams take time to become reality. Everything needs to start small, with baby steps. And my first step is to validate that this is a dream worth turning into reality. I'll tell you more about this as I move forward!

I'd be thrilled to have your opinion about this. How do you capture content? How do you prioritize what to read/watch/learn? How do you take notes? Are these problems that you're also facing? How painful is this for you? What is your system like?

A call for help

Dev Concepts is an important project for me. I have a passion for IT and software development and want to share it with as many people as I can. I strongly believe that these books are valuable, but people need to know that they exist before they can decide for themselves.

So please go ahead and help me to let people know that it exists:

It won't cost you much time, and it's invaluable to me.

That's all folks!

Make sure to follow me on Twitter (https://twitter.com/dSebastien) if you want to hear about cool things around software development and technology in general.

Also, I'm really interested to hear what you think of this newsletter. If you have remarks/ideas/articles to share, then please don't hesitate to send me a DM on Twitter. You can also simply reply to this e-mail.

If you find this newsletter interesting, then please share the link around: https://mailchi.mp/fb661753d54a/developassion-newsletter

See you next time!

PS: check out my Dev Concepts books, join the Software Crafters community, the Personal Knowledge Management community, and come say hi on Twitter!
Discuss on TwitterEdit on GitHub