Better Bun Support (bun:sqlite)

[Fawaz]

Hello! 👋

I'm currently using your awesome package to build an internal local-first app at my company. I'm using Bun & SvelteKit to build my project, and so I noticed that this package has experimental support for Bun. Now I'm still a beginner in this whole local-first cr-sqlite CRDT world. But as I understand it, any SQLite instance that works with cr-sqlite should be able to work with CRStore, bun:sqlite included.

Problem

When I try to run this file in Bun:

import { crr, primary } from "crstore";
import { boolean, object, string } from "superstruct";
import { database } from "crstore";
import Os from 'os';
import { Database } from 'bun:sqlite';

const macOS = Os.platform() === "darwin";

if (macOS) {
    /**
     * By default, macOS ships with Apple's proprietary build of SQLite
     * which doesn't support extensions. (therefore does not support cr-sqlite)
     * To use extensions, you'll need to install a vanilla build of SQLite.
     * Reference: https://bun.sh/docs/api/sqlite#loadextension
     */
    Database.setCustomSQLite("/opt/homebrew/opt/sqlite/lib/libsqlite3.dylib");
}

// Struct that represents the table
const todos = object({
  id: string(),
  title: string(),
  text: string(),
  completed: boolean(),
});
crr(todos); // Register table with conflict-free replicated relations
primary(todos, "id"); // Define a primary key (can be multi-column)

const schema = object({ todos });

const { close } = database(schema);

close();

I get this error:

53 |             return Promise.resolve({
54 |                 rows: stmt.all(parameters),
55 |             });
56 |         }
57 |         else {
58 |             const { changes, lastInsertRowid } = stmt.run(parameters);
                                                      ^
TypeError: Right side of assignment cannot be destructured
      at executeQuery (/Users/fawaz/Projects/crstore-bun-issue/node_modules/kysely/dist/esm/dialect/sqlite/sqlite-driver.js:58:50)
      at /Users/fawaz/Projects/crstore-bun-issue/node_modules/kysely/dist/esm/dialect/sqlite/sqlite-driver.js:35:15
      at rollbackTransaction (/Users/fawaz/Projects/crstore-bun-issue/node_modules/kysely/dist/esm/dialect/sqlite/sqlite-driver.js:34:31)
      at /Users/fawaz/Projects/crstore-bun-issue/node_modules/kysely/dist/esm/kysely.js:417:23

Minimal Reproduction: https://github.com/fawaz-alesayi/crstore-bun-issue

Why I think this happens

After a couple of minutes of investigation I noticed that you're using Kysely when creating the new tables defined by superstruct.

As I understand it, Kysely only supports bun:sqlite if you're using a custom adapter like these ones
https://github.com/dylanblokhuis/kysely-bun-sqlite
https://github.com/subframe7536/kysely-sqlite-tools/tree/master/packages/dialect-bun-worker

Solutions

Here are some ways I thought up of making Bun support better:

Solution 1: Allow users to provide their own Kysely adapters

This is the solution I'm using in my fork and it works great. The new API for the database function would be

import { object, string, type Infer } from "superstruct";

    const db = database(schema, {
      name: "test.db",
      kysely: ({ database }) => {
        return new Kysely<Schema<Infer<typeof schema>>>({
          dialect: new SqliteDialect({ database }), // Or BunDialect, LibSQL, whatever works with cr-sqlite
          plugins: [new JSONPlugin()],
        });
      },
    });

Solution 2: Conditional check to load appropriate Kysely Dialect

Would look something like this:

// https://github.com/Azarattum/CRStore/blob/main/src/lib/database/index.ts
async function init<T extends CRSchema>(
  file: string,
  schema: T,
  paths = defaultPaths,
) {
  type DB = Schema<T>;
  if (connections.has(file)) return connections.get(file) as Connection<DB>;

  const { database, browser } = await load(file, paths);
  const Dialect = browser ? CRDialect : process.isBun ? SqliteBunDialect : SqliteDialect;
  const kysely = new Kysely<DB>({
    dialect: new Dialect({ database }),
    plugins: [new JSONPlugin()],
  });

// ... rest of the code

I haven't had much time to dig into the actual error that is produced by Kysely and why it happens with Bun. but my guess is that there's some subtle API differences between bun:sqlite and better-sqlite3 that causes this error.

[Azarattum]

Hello, @fawaz-alesayi. Thank you for an awesome description of an issue and all your investigation on the problem.

The issue is indeed originates from bun:SQLite having a slightly different API. In particular, run function doesn't return any metadata unlike better-sqlite. In fact we cannot even differentiate whether a prepared statement returns any result or not (because reader property is not implemented in bun:sqlite). kysely-bun-sqlite works around this issue by treating every query as if it returned something (running them with .all()).

I have made a separate runtime entry in CRStore for bun which makes reader always be true in prepared statements therefore making them compatible with default SQLite adapter. That also brought some QOL changes, like now you don't have to do Database.setCustomSQLite nonsense to make it work on MacOS (CRStore does that for you). You can also overwrite that default path with the the same binding path option that you would use in node.

I've published 0.23.0 version with these changes. Although I was able to run CRStore's test suit successfully with bun (even without vitest), let me know if there are any other bun-related difficulties you encounter while building your app. So, that we can finally bring bun support out of the experimental stage.