groue / GRDB.swift

What did you do?

Tried to build GRDB on Linux
Linked with a SQLite distribution built without setting SQLITE_ENABLE_SNAPSHOT so that various database snapshot related APIs are not available.
Observed a linker error with unresolved symbols for functions like sqlite3_snapshot_get.

I understand that Linux isn't really supported. I think that I'm asking for something relatively small here, and might be able to send a PR.

What did you expect to happen?

To compile successfully without any undefined symbol references in GRDB code.

What happened instead?

GRDB referenced undefined symbols that weren't available from the SQLite library, because it was built without enabling SQLITE_ENABLE_SNAPSHOT.

Environment

GRDB flavor(s): GRDB
GRDB version: v6.24.2
Installation method: Manual
Xcode version: N/A
Swift version: 5.10
Platform(s) running GRDB: Linux
macOS version running Xcode: N/A

Demo Project

N/A

More Context

The relevant code is attempting to guard against this state, as an example, this condition is used in many places throughout the codebase: #if SQLITE_ENABLE_SNAPSHOT || (!GRDBCUSTOMSQLITE && !GRDBCIPHER && (compiler(>=5.7.1) || !(os(macOS) || targetEnvironment(macCatalyst))))

Some examples:

GRDB.swift/GRDB/Core/WALSnapshotTransaction.swift

Line 2 in 4b934fd

#if SQLITE_ENABLE_SNAPSHOT || (!GRDBCUSTOMSQLITE && !GRDBCIPHER && (compiler(>=5.7.1) || !(os(macOS) || targetEnvironment(macCatalyst))))
GRDB.swift/GRDB/Core/WALSnapshot.swift

Line 2 in 4b934fd

#if SQLITE_ENABLE_SNAPSHOT || (!GRDBCUSTOMSQLITE && !GRDBCIPHER && (compiler(>=5.7.1) || !(os(macOS) || targetEnvironment(macCatalyst))))
GRDB.swift/GRDB/Core/DatabasePool.swift

Line 648 in 4b934fd

#if SQLITE_ENABLE_SNAPSHOT || (!GRDBCUSTOMSQLITE && !GRDBCIPHER && (compiler(>=5.7.1) || !(os(macOS) || targetEnvironment(macCatalyst))))

When using SQLite built in the same project (but NOT GRDBCUSTOMSQLITE) or a pre-built distribution of SQLite on any platform other than macOS, while using compiler newer than 5.7.1, this condition will be true no matter whether that SQLite supports snapshot features. It would be nice to allow disabling these references when using SQLite without database snapshot support on platforms other than macOS.

I'm not positive what's the right behavior here, since most distributions of SQLite probably have database snapshots enabled. Maybe split it so that one of one of SQLITE_ENABLE_SNAPSHOT || GRDBCUSTOMSQLITE || GRDBCIPHER is required on Linux?

#if SQLITE_ENABLE_SNAPSHOT || (os(Linux) && (GRDBCUSTOMSQLITE || GRDBCIPHER)) || (!GRDBCUSTOMSQLITE && !GRDBCIPHER && (compiler(>=5.7.1) || !(os(macOS) || targetEnvironment(macCatalyst))))

That would mean when using a prebuilt SQLite distribution with snapshot support, SQLITE_ENABLE_SNAPSHOT would have to be set explicitly to enable the features in GRDB.

Alternatively, there could be a flag to opt out of the features when the developer knows their SQLite doesn't support snapshots which would be set explicitly as needed when building GRDB in environments like mine. I'm open to that too.

Unfortunately, just setting SQLITE_ENABLE_SNAPSHOT when building SQLite isn't an option for me, so it'd be nice if I could build GRDB while excluding the snapshots APIs.

Hello @dylansturg,

Welcome to the world of a library that promises to run with many SQLite versions!

As you may have noticed, Linux is not "officially" supported. It is not tested in the CI, and it is based on (very welcome) user contributions. I do my best to not break it.

That would mean when using a prebuilt SQLite distribution with snapshot support, SQLITE_ENABLE_SNAPSHOT would have to be set explicitly to enable the features in GRDB.

This would be acceptable to me. To use snapshots (highly recommended) on Linux, one would have to fork GRDB, and add SQLITE_ENABLE_SNAPSHOT to swiftSettings.

Alternatively, there could be a flag to opt out of the features when the developer knows their SQLite doesn't support snapshots which would be set explicitly as needed when building GRDB in environments like mine. I'm open to that too.

This would work, but would not be quite standard. SQLite users expect #if FLAG, not #if !(INVERSED_FLAG).

Unfortunately, just setting SQLITE_ENABLE_SNAPSHOT when building SQLite isn't an option for me [...].

I'm very sorry. You'll miss nice runtime optimizations, especially if you use database observation on a WAL database!

To sum up: I'll warmly welcome a pull request that modifies the condition so that 1. it does not change the current behavior on Apple platforms, 2. it enables snapshot support on Linux if and only if SQLITE_ENABLE_SNAPSHOT is defined.

The current expression is:

SQLITE_ENABLE_SNAPSHOT
||
(
  !GRDBCUSTOMSQLITE
  &&
  !GRDBCIPHER
  &&
  (
    compiler(>=5.7.1)
    ||
    !(os(macOS) || targetEnvironment(macCatalyst)
  ))
)

In pseudo code, it translates to:

let snapshotApisEnabled = if SQLITE_ENABLE_SNAPSHOT {
  // explicit flag: enable
  return true
} else {
  if GRDBCUSTOMSQLITE {
    // Not Apple SQLite: rely on SQLITE_ENABLE_SNAPSHOT
    return false
  }
  if GRDBCIPHER {
    // Not Apple SQLite: rely on SQLITE_ENABLE_SNAPSHOT
    return false
  }
  // From now on, assume we're on an Apple system
  if compiler(>=5.7.1) {
    // <sqlite.h> contains snapshot apis
    return true
  }
  if !(os(macOS) || targetEnvironment(macCatalyst) {
    // compiler < 5.7.1, macOS
    // <sqlite.h> does not contain snapshot apis :-/
    return false
  }
  // compiler < 5.7.1, iOS, tvOS, watchOS, etc
  // <sqlite.h> contains snapshot apis
  return true
}

This logic turns wrong on the line "From now on, assume we're on an Apple system", so I expect that the new test should be:

SQLITE_ENABLE_SNAPSHOT
||
(
  !GRDBCUSTOMSQLITE
  &&
  !GRDBCIPHER
  &&
  !os(Linux)
  &&
  (
    compiler(>=5.7.1)
    ||
    !(os(macOS) || targetEnvironment(macCatalyst)
  ))
)

This expression is awful, and we can't split it up because Swift does not support #define 😅

// That would be cool
#if SQLITE_ENABLE_SNAPSHOT || (APPLE_SQLITE && APPLE_SQLITE_ENABLE_SNAPSHOT)