Indexed DB & Dexie JS for my Hybrid App
Requirements
For my app, which uses a reactjs stack + capacitor for deploying to mobile platforms (android and ios), I wanted a datastore that can persist data locally on:
- Browser
- Android app
- iOS app
Browser storage possibilities
Firstly, the browser is the most restrictive of the three platforms, in terms of supported ways of storing data locally. The different types of storage are:
- Cookies – not really for a whole database; more suited for tiny amounts of data.
- Local storage – not really for a whole database; more suited for limited data (limit 5 MB).
- Session storage – same as local storage, but only for one session, not persisted across sessions.
- IndexedDB – promising, it’s a key-value store (NoSQL) for arbitrary application data; finally this is my choice.
- Web SQL – deprecated and not supported by all browsers.
- Cache storage – too limited and not for persistence purposes.
- Sqlite3 and WASM (web assembly) – did some extensive trials with this, but in the end, this also uses IndexedDB under the hood – so I see no need to add this layer over IndexedDB (also because it is not an easy layer to interact with, and brings all kinds of complexities with it).
There are many excellent resources as listed in the Resources section, explaining these and the differences between them very well.
IndexedDB is the winner, the most suited for my app, given the constraints (3 platforms) and reasons above.
Accessing IndexedDB
IndexedDB has an API and is quite well documented. So I started writing code to use it directly. This worked quite well for a prototype. The code worked well in browser and on the android mobile app. Did a brief test on iOS, and it seems to work fine.
But soon after that, I needed querying capabilities, and the IndexedDB API is quite cumbersome to deal with directly.
Having learned how it works under the hood, I became comfortable to look for a wrapper that adds convenience of use.
There are quite a lot of options at this stage, but I considered two seriously:
PouchDB Trial
PouchDB is a client-side implementation of a document store, based on (or inspired by!) Apache CouchDB, open source, and seems quite robust. It essentially writes to IndexedDB on the browser, and falls back to local storage or something else if not available.
It was easy and the code is elegant: maybe has something to do with my slightly better understanding of asynchronous Promise based coding. I got excited by the possibility of syncing the data seamlessly to a backend CouchDB. It is super simple, literally one or 2 lines of code.
After a minimal trial where I got it working to write food logs to indexeddb, I decided to investigate other solutions (i.e., Dexie).
There are a few different reasons for my hesitation to go forward with PouchDB. While each single reason may not be enough to dismiss it, the whole set makes me wary:
- No separation of entities: Being a document store, it doesn’t provide a way to separate the different entities into separate object stores (as raw IndexedDb allows!). So in my case, the food_log, activity_log, etc. will all be stored as a document in the same space.
- Filter on entity type for all queries: This immediately means, I need to filter each entity for every type of query I need to do. This absolutely makes no sense to me (coming from a relational db world), and also for my usecase, this doesn’t seem to make semantic sense. I am sure it can be worked around, but should I have to work around this?!
- Install hack: After npm installing pouch db in my project, I had to add it to the public/index.html – and didn’t figure out a way of specifying the node_modules/… path – had to copy pouchdb.min.js to the /public folder. Could potentially fix this issue, but for now ignoring.
- Query syntax needs a plugin, and is new to learn: Looking beyond the basic put and get operations, if you want to query and filter, things get quickly complicated: I figured that I would be better off to create the _id value as a meaningful string, concatenating meaningful fields into a type of composite key. But if I want to query by, say, date and meal moment for some use cases, and just meal moment across many days for a different usecase, the primary index will only be useable for one of the cases.
- Secondary Indexes to the rescue, but…: You can create secondary indexes (see this article) – but given the first two points, I wondered if I truly wanted to use a document store and have to learn all this syntax.
- Sync to CouchDB: Though the sync is simple, there is no cloud player that offers a managed CouchDB instance. It will have to be paid solutions on their marketplaces (Bitnami, Websoft9 etc.)
- CouchDB Sync is only data sync: This means, even if I get a paid cloud couchdb, it will only do the data sync and no other serverside functionalities… for the product roadmap, I have in mind some serverside analysis features, and don’t like to have to add those separate from CouchDB – and have to copy the data from CouchDB to another datastore for analysis.
Given all these reasons, I am shelving PouchDB for now, and investigating Dexie.js.
Dexie.js
After a few weeks using Dexie.js, I really like it. It’s intuitive, doesn’t get the in way, has excellent documentation, and great features for dealing with IndexedDB. I don’t use Dexie cloud sync – just use it locally within my app and browser as a dependency.
Resources
Browser data storage options explained