Moving away from localStorage

[RobRoyce]

It has become increasingly obvious that the localStorage method for storing Projects and Sources is not sustainable. Some of the reasons include:

• It is difficult and error prone to import/export data
• It is not immediately obvious how to create backups (although a method does exist)
• Electron imposes a hard limit of 25MB per entry (which I have already hit), making it impossible to add new Sources to Projects that have reached that limit.
• After a certain point, the data completely stops showing up in DevTools (probably a bug in Chromium, but still)

I have been contemplating how to move away from localStorage for a while, and I have finally settled on one. Starting in version 0.8.0, the app will store data in JSON files. By default, those files will be stored in the same directory in which Knowledge stores Autoscan files. e.g.:

• On macOS and Linux: ~/.Knowledge/
• On Windows: C:\Users\username\.Knowledge\

This will eliminate all of the issues laid out above, while maintaining the same format (JSON) and avoiding the need for self-hosted and/or external databases. Additionally, it provides a new way of synchronizing data between multiple devices (rsync, scp, network attached storage, etc.). Finally, this will provide an easy way to manage imported files. That is, if the user enables File Manager in Settings > Import for all imported files, the files will be stored in the same directory where the Source JSON file is stored.

Once 0.8.0 is released, Knowledge will update itself and begin the migration automatically. Along with this change, the update will bring a new splash screen on app startup to alert the user when new updates are available (see demo below). I do not plan to add a way to opt out of the migration, but I will if enough people request it between now and then. As far as I can tell, there is zero downside to this approach and the change will not be noticeable from the users perspective.

I anticipate the release of 0.8.0 within 2-4 weeks from the date of this post. Please leave any comments or feedback below.

Cheers!

Rob Royce