Why I Use Note Taking Apps
My memory sucks. Partly that’s age, but if I’m honest, it’s never been that good. For years now, whenever I learn something useful, I make sure I put it in a searchable note taking system so that I can find it again later. Sometimes, I even blog about it, and read my own blog later to remind myself of the details 😀 But I always make sure my notes repository has a mention of it & link, so I don’t have to remember whether it’s on my blog, in one of my browser tabs or favourites, or in a 3rd party web saving system like Pocket or whatever. I want to search one place, and find what I need, if I’ve encountered it before.
I used Evernote back in the day, then migrated to OneNote later, mostly because it was free (especially if you have an Office sub anyway) and Evernote annoyed me somehow. Did they change their subscription model or something? See, I didn’t make a note of that, so I can’t remember 😄
I use these tools for writing my own notes, and for taking local copies of other people’s articles - both to make them easier for me to find again, and to protect against link rot. At the risk of over-stressing it, the key is I should always have one place to search for things I’ve already curated.
What Pushed Me Over The Edge
OneNote isn’t great. It’s a pain in the arse to format code snippets, it insists on using OneDrive, its diagramming tools are kind of useless (they could have chosen to put Visio in there, but nooo), the formatting frequently goes wonky for no reason, and the best you can say for search is it’s “OK”. It did the job enough that I kept using it because all my notes were already there, but I wouldn’t say I liked it very much.
For taking copies of web pages, I used a Web Clipper - Evernote probably pioneered this, and OneNote had one too. I use ‘had’ in the past tense, because when I went to use it a week ago, I received this notification:
I use Firefox, because Google is horribly compromised by their ad funding model, and cannot be trusted to do right with Chrome, and Microsoft Edge inherits Google’s bad choices via Chromium, and adds their own on top. Today’s Firefox runs better than both of them anyway, with the added bonus of actually having principles.
To see Microsoft imply that Firefox wasn’t a “modern” browser, and that I should use Edge instead if I wanted to do basic fecking things with their just-about-adequate product, was the final straw. I’d stuck with OneNote mostly because of laziness, I’d have much preferred something better, and this was the nudge I needed.
My requirements
What I wanted from a note taking system was:
- Searchable database of multimedia notes
- Arbitrary nesting in sections
- Clip websites for local backup
- Sync across multiple machines, preferably self-hosted
- Shareable with others
- A database I own & control, that’s open & interoperable.
That last point is important. These are my notes, and I don’t want some company holding them hostage (any more). Preferably I wanted the storage format to be Markdown, because everything else I use is (this blog included), and it’s something that’s really easy to read even without a specific tool.
Why I Chose Obsidian
I put out a call on Mastodon, and Obsidian was by far the most popular suggestion. It perfectly fit almost all my requirements. A significant factor is that its “database” is really just a structured filesystem of markdown text files and images, with some extra side-loaded config and caching. I can’t tell you how nice it is to see all my notes stored like this - markdown is extremely familar and easy to use (code snippets actually work without fuss!), and I know that if in the future I need to migrate this data again, it’ll be much easier because of this format.
While other suggestions like Notion and UpNote look interesting, they’re based on proprietary formats and really, that only has downsides for notes.
The Obsidian app is also very nice, with an absolutely huge collection of community plugins that extend it even further. It’s a mature ecosystem but not one I feel locked into.
One thing to bear in mind is that the client is not open source. I would have preferred that, but the main OSS alternative LogSeq seemed not quite as mature or as highly recommended (yet). The fact that all my data is simple Markdown and pretty instantly portable to other systems later makes me comfortable with using a non-OSS frontend though.
One feature I use in Obsidian is Wikilinks which is strictly not portable to plain Markdown. Wikilinks are much shorter internal links, and Obsidian auto-completes them, so when doing links between pages it’s much more convenient. But, these links only function with a client that’s capable of finding where they’re pointed, a plain text editor can’t follow them because they’re not direct. However, Wikilinks are a common concept, and e.g. LogSeq supports them, and at a push it’s not that hard to manually resolve them. So I’m not worred about data portability later even when using these.
The Conversion Process
So enough of all that, how did I convert all my old notes from OneNote into Obsidian?
It’s Just Markdown, Jim
In fact I didn’t need to convert my OneNote data “into Obsidian” as such, it just needed to become a set of folders with Markdown files and images in them. Obsidian is then just a layer of convenience on top of that.
I used the ConvertOneNote2MarkDown Powershell tool. You do need to install a couple of things first:
-
An older (7.0.x) version of Powershell. Yeah, I know - but MS removed the feature needed to call OneNote’s APIs in PS 7.1. 🙄 Luckily you can install it alongside your newer version just for this purpose
-
A copy of Pandoc, installed and on your path.
It’s not that onerous. Once you’ve installed both of these, you’ll be running the
tool by manually firing up that out of date 7.0.x version of pwsh.exe and
running ConvertOneNote2MarkDown-v2.ps1 within the shell. But first, let’s configure it.
Configuring ConvertOneNote2MarkDown
While you can specify all the arguments on the command line, I used a
config.ps1 file that I tweaked over time. I recommend taking a look at the
self documenting example version,
Here are the settings I modified, with comments explaining why. I left everything
else defaulted, so you can read the below and make these same changes to config.ps1
if you want to repeat my method:
# Obviously I set the destination. Don't worry, you can just move the files later
$notesdestpath = 'E:\Notes\Converted'
# To begin with, I changed this setting to the smallest notebook I had, as a test
# And then re-ran it each time with each notebook as I learned. It also meant it
# was easier to restart a smaller process in the case of an error
# You can leave this blank if you want to do them all at once, I just preferred a more controlled test
$targetNotebook = 'My Notebook'
# This setting makes the process run faster, why wouldn't you?
$usedocx = 2
# I preferred to risk blowing the file path limit and not truncate things so much
# The default is 32 which would truncate a lot of my page names.
$mdFileNameAndFolderNameMaxLength = 200
# I chose: 2: Separate 'media' folder for each folder in the hierarchy
# I prefer this to lumping all media in a single root location
$medialocation = 2
# This was a recommended setting for Obsidian and seemed to work quite well for me
$conversion = 'markdown-simple_tables-multiline_tables-grid_tables+pipe_tables-bracketed_spans+native_spans+startnum'
# I chose to change this to CRLF since I'm on Windows, you would leave it default on Linux/Mac
$newlineCharacter = 2
Do check the Known Issues on the conversion tool in case you have some edge cases I didn’t.
Failures
I had a few pages that would abort the conversion with this message:
Exception: Exception calling "Publish" with "4" argument(s): "The remote procedure call failed. (Exception from HRESULT: 0x800706BE)
It turned out that there was something a little corrupted in OneNote about these pages. To fix the problem all I needed to do was:
- In OneNote, make a new copy of the page via right-click “Move or Copy”
- Delete the original page
- Re-run the conversion
This magically fixed this problem. 🤷♂️
Post Conversion Bulk Clean-up
The conversion isn’t perfect. You end up with some spurious stuff which is luckily fairly easily removed. I did try to eliminate this stuff through config changes, but for whatever reason it eithe rdidn’t work, or made some other aspect worse, so these settings, plus a bit of bulk post-conversion cleanup, ended up as the best option.
I performed all of these clean-ups by opening the output folder in VS Code, and using the Replace In Files function in Regular Expression mode
- Remove post-image “{width=“3.8541666666666665in” height=“1.7708333333333333in”}” metadata:
- Find:
\{width=[^\}]+\} - Replace with: Nothing
- Find:
- Remove “Created/Modified” sections
- Find:
^Created: .*\n\nModified:.*\n\n---\n - Replace with: Nothing
- Find:
- Remove additionl blank lines
- Find:
\n\n\n - Replace with: Nothing
- Find:
After that, there may still be some things which need manual tidy up inside Obsidian itself. I had some tables with embedded code that needed the manual HTML removing, but it’s not something you can do automatically. I’ve just been tidying up as I come across them, it’s usually not a big deal.
If a page that you previously clipped from the web is particularly nasty looking, you’d probably be best to re-clip it using our new method described below.
Re-organisation
I had 5 separate notebooks in OneNote, but that was solely because OneNote only allows 2 levels of nesting (not strictly true, you can make sub-pages, but they’re unwieldy). Once converted to Obsidian, it’s all just folders and files so you can have any arbitrary level of nesting!
I combined my 5 notebooks into 2 Obsidian vaults, one shared and one not, through the simple magic of moving folders around.
An Obsidian “Vault” is just another name for the root folder in a hierarchy. Each vault can have a different config, theme etc. You can just copy files around between vaults, and into the vault from outside sources, it’s all good.
Web Clipping
As I mentioned earlier I like to keep offline copies of blog posts that I consider useful, both because it adds them to my unified search, and it protects against those blogs going offline and leaving me just with a broken link, and broken dreams.
While Evernote and OneNote had their own clippers, you don’t need an Obsidian-specific web clipper - all you need is a tool that can scrape a HTML page down to usable Markdown. I found Markdownload works great for this. You can just download a web page as Markdown + images, then move those files directly into your Obsidian tree.
Configuring Markdownload
There are some settings you’ll want to tweak in the Markdownload extension first. For simplicity, rather than just enumerate them, I’ll provide the settings I use as JSON below. You can import them in the Markdownload settings page, reached in Firefox via Extensions > Gear icon next to Markdownload > Options. At the bottom:
{
"headingStyle": "atx",
"hr": "---",
"bulletListMarker": "-",
"codeBlockStyle": "fenced",
"fence": "```",
"emDelimiter": "_",
"strongDelimiter": "**",
"linkStyle": "inlined",
"linkReferenceStyle": "full",
"imageStyle": "obsidian",
"imageRefStyle": "inlined",
"frontmatter": "Source: {baseURI}\n***\n",
"backmatter": "",
"title": "{pageTitle}",
"includeTemplate": true,
"saveAs": false,
"downloadImages": true,
"imagePrefix": "media/",
"mdClipsFolder": "Markdownload",
"disallowedChars": "[]#^",
"downloadMode": "downloadsApi",
"turndownEscape": true,
"contextMenus": true,
"obsidianIntegration": false,
"obsidianVault": "",
"obsidianFolder": ""
}
Of particular note in these settings:
- I enable Front Matter and have it just add the Source URL, so I know where this page came from originally
- I put all images in a
media/subfolder, matching the layout I used during conversion - I set the download folder to “Markdownload” so my markdown is separated from other downloads
Clipping a page with Markdownload
Once you’ve got this configured, all you do to clip a web page is:
-
Right-click the page, and select “Download Tab As Markdown”
-
Move the file & media folder from Downloads/Markdown to the location you want in Obsidian
I say Move specifically because it’s useful that your Markdownload folder is empty every time you clip a new page, so all the media is cleaned out.
That’s it! Sometimes you might need to tidy up some pages afterwards, but Markdownload does a pretty good job of stripping out unnecessary sidebars, headers/footers etc. If you need more control you could also switch Firefox into Reader View (F9) before downloading, although I’ve rarely found that necessary.
You can configure the Markdown clipper to spit things directly into Obsidian via the “Enable Obsidian Integration” option, if you also add the Advanced Obsidian URI plugin to Obsidian. I choose not to use that though, because I have multiple vaults and don’t want things to automatically be added to one of them. It’s so easy just to move the downloaded files where I want I prefer the simpler config, but YMMV.
Syncing
Because Obsidian’s data store is just a filesystem, there are lots of ways you can sync your Obsisdian notes; anything which can sync files will do (at least on desktop, more on that later).
I chose to use our own private Synology Drive, because it’s where we keep all our other documents, has backups configured, is already shared between us etc. It just works! Obsisian monitors the file system so any changes one person makes that’s synced to the other is immediately updated for them.
You can of course use OneDrive, iCloud Drive, Dropbox, Google Drive, whatever you want. Obsidian also provide an optional subscription service for hosting if you don’t have any of those, or need it for other reasons (see below).
iOS Limitations
The one downside is that the Obsidian iOS app only supports syncing via either iCloud Drive, or their own $8pm sync service. I prefer not to run iCloud on my Windows machine (it kinda sucks there) and I can’t really justify $8pm just for iOS support. I hardly ever use OneNote on iOS anyway, my notes are usually related to things I’m doing on a desktop anyway, so I choose to live without being able to use the iOS app. At a push, I can view/edit the markdown in plain text via the Synology Drive iOS app, but I doubt I will. If you need full-featured usage on iOS then you’d be better with one of the officially supported options.
Another way to sync to iOS is to use Git. Basically you store everything in Git and clone down to the local Obsidian storage location. It’s a faff though; you have to commit/push/pull through another app like Working Copy, and it means every change on desktop has to be committed / pushed / pulled too. There is a community plugin to semi-automate that, but it’s still so much less immediate than plain file syncing, especially if you share your vault. It didn’t seem worth the trouble given my use case.
Theming
The default Obsidian theme is fine, but it’s nice to be able to change it; there are loads of community themes.
I chose to use Prism, because it is highly configurable via the Style Settings Plugin.
I mention this because I didn’t look at Prism until I’d been through loads of others, each of which was almost to my liking but not quite, and to tweak them required forking and changing CSS etc. Prism didn’t catch my eye at first because its theme icon is a project logo rather than a preview, but after I tried it, I preferred it to all the others because of these config-based adjustment options. Recommended.
Plugins
I was told early on that “just wait until you see what the community plugins can do”, and they’re not wrong.
I haven’t played with many yet, but the top used ones are a good guide to what’s hot. I’ve installed the following so far:
- Editing Toolbar. By default Obsidian just expects you to know Markdown, which is fine for me, but my wife doesn’t use Markdown regularly like I do. This plugin adds a formatting toolbar to make that friendlier.
- Style Settings. As mentioned above, this provides configuration options for the Prism theme (among other things)
- Excalidraw. Remember how OneNote has crap diagramming because Microsoft want to nickel & dime people for Visio? This adds Excalidraw diagramming support right in Obsidian, and it’s great.
Plugins I haven’t tried yet but have my eye on:
- Kanban. Want to put Trello in Obsidian? That’s basically this. All our stuff is still in Trello right now, but I wonder whether this might be in my future.
- Tasks. A list-based task management system instead.
- Advanced Tables. Helper for working with tables in Markdown. Actually you know what, I’m going to install this one right after I’ve finished this post. 🙂
Conclusion
A lot of people recommended Obsidian to me, and I can see why. I wish I’d started using it a long time ago, but as they say, the second-best time to start doing anything is now 🙂
I especially recommend moving across if you’re on OneNote, which looks especially tatty once you see the contrast. I hope this guide helps you make the move; the good news is that even with having to figure this stuff out for myself, it still only took me about a day to do. With this info, it should hopefully take you a lot less. Happy note taking!