feat: add durable provider cache
Add a private provider-neutral SQLite WAL cache with transactional migrations, retained raw payloads and tombstones, per-window opaque checkpoints, and reserved outbox/conflict tables. Verify migrations, permissions, identity invariants, concurrent access, atomic pull-page rollback, and future-schema rejection with deterministic tests. No OAuth, networking, secrets, or Graph synchronization is included yet.
This commit is contained in:
23
README.md
23
README.md
@@ -7,7 +7,9 @@ inherits its colors from your terminal theme.
|
|||||||
This repository currently contains the first local-calendar vertical slice:
|
This repository currently contains the first local-calendar vertical slice:
|
||||||
date/domain logic, guarded atomic iCalendar persistence, add/edit/delete forms,
|
date/domain logic, guarded atomic iCalendar persistence, add/edit/delete forms,
|
||||||
a responsive ANSI terminal UI with month and agenda views, explicit safe
|
a responsive ANSI terminal UI with month and agenda views, explicit safe
|
||||||
calendar import/export, tests, and Linux launch integration. See
|
calendar import/export, tests, and Linux launch integration. A provider-neutral
|
||||||
|
SQLite cache now provides the durable boundary required before
|
||||||
|
Office 365 or any other remote provider is connected. See
|
||||||
[the product specification](docs/PRODUCT.md),
|
[the product specification](docs/PRODUCT.md),
|
||||||
[architecture](docs/ARCHITECTURE.md), and [roadmap](docs/ROADMAP.md).
|
[architecture](docs/ARCHITECTURE.md), and [roadmap](docs/ROADMAP.md).
|
||||||
|
|
||||||
@@ -105,3 +107,22 @@ backup itself is kept so recovery can be repeated.
|
|||||||
|
|
||||||
For a popup calendar binding and version-specific Hyprland rules, see
|
For a popup calendar binding and version-specific Hyprland rules, see
|
||||||
[Hyprland integration](docs/HYPRLAND.md).
|
[Hyprland integration](docs/HYPRLAND.md).
|
||||||
|
|
||||||
|
## Remote-provider foundation
|
||||||
|
|
||||||
|
The remote-provider foundation is a private local SQLite database in WAL mode,
|
||||||
|
separate from the interactive `.ics` file workflow. Its schema is migrated by
|
||||||
|
version in all-or-nothing transactions. It preserves raw provider payloads,
|
||||||
|
inactive calendars, and tombstoned events so synchronization cannot silently
|
||||||
|
discard data that the current application does not understand. Each opaque
|
||||||
|
provider cursor is scoped to a calendar and synchronization window and is
|
||||||
|
committed atomically with the pulled page it represents. Normalized event
|
||||||
|
instances provide immutable snapshot input, while transactional outbox and
|
||||||
|
conflict records are reserved for later write synchronization.
|
||||||
|
|
||||||
|
This is a persistence prerequisite, not working Office 365 integration. Nocal
|
||||||
|
does not yet perform OAuth, make Graph requests, or synchronize a remote
|
||||||
|
calendar. OAuth tokens and other secrets will never be stored in SQLite; they
|
||||||
|
belong in the Freedesktop Secret Service. The next implementation step is an
|
||||||
|
injectable HTTP boundary with PKCE authentication and deterministic fake
|
||||||
|
transports, followed by read-only Microsoft Graph delta synchronization.
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ and terminal rendering replaceable.
|
|||||||
```text
|
```text
|
||||||
src/main.cpp
|
src/main.cpp
|
||||||
├── tui/ ANSI rendering, input decoding, terminal lifecycle
|
├── tui/ ANSI rendering, input decoding, terminal lifecycle
|
||||||
├── storage/ iCalendar file adapter (later: SQLite cache)
|
├── storage/ iCalendar file adapter and provider-neutral cache
|
||||||
└── domain/ Event, Calendar, civil dates, month layout and queries
|
└── domain/ Event, Calendar, civil dates, month layout and queries
|
||||||
|
|
||||||
future sync workers
|
future sync workers
|
||||||
@@ -118,13 +118,27 @@ semantically. Browsing remains available for unsupported files. This
|
|||||||
conservative boundary is more important than partial editing because a
|
conservative boundary is more important than partial editing because a
|
||||||
successful-looking edit must not erase unrelated calendar data.
|
successful-looking edit must not erase unrelated calendar data.
|
||||||
|
|
||||||
Provider synchronization will not write directly into this UI file. It will use
|
Provider synchronization does not write directly into this UI file. Its
|
||||||
a transaction-capable local cache and preserve remote ETags/sync tokens in a
|
prerequisite durable boundary is a provider-neutral SQLite database in WAL mode.
|
||||||
provider metadata table.
|
The database is a private local file, and schema changes run as versioned,
|
||||||
|
all-or-nothing migrations so a failed upgrade cannot leave a partially migrated
|
||||||
|
cache.
|
||||||
|
|
||||||
The planned cache schema has `calendars`, `events`, `event_instances`, and
|
The cache separates provider records from normalized projections. It retains
|
||||||
`sync_state`. Raw provider payloads are retained alongside normalized fields so
|
raw provider payloads alongside normalized fields so an older Nocal cannot
|
||||||
an older Nocal cannot destroy fields it does not understand.
|
destroy fields it does not understand. Inactive calendars and tombstoned events
|
||||||
|
remain recorded rather than disappearing from history. Normalized event
|
||||||
|
instances are materialized for immutable UI snapshots; provider payload details
|
||||||
|
do not cross into rendering or the domain model.
|
||||||
|
|
||||||
|
Incremental progress is scoped by provider account, calendar, and synchronization
|
||||||
|
window. A provider's opaque cursor is committed in the same transaction as each
|
||||||
|
pulled page, preventing a crash from advancing the cursor beyond durable data.
|
||||||
|
Transactional outbox and conflict records are reserved in the boundary for
|
||||||
|
future local writes and deterministic conflict handling. This cache does not
|
||||||
|
itself implement networking, authentication, or Microsoft Graph synchronization.
|
||||||
|
OAuth tokens and other secrets are explicitly excluded from SQLite and belong
|
||||||
|
in the Freedesktop Secret Service.
|
||||||
|
|
||||||
## Terminal backend
|
## Terminal backend
|
||||||
|
|
||||||
@@ -149,3 +163,10 @@ Service, never in the calendar database or command line.
|
|||||||
Conflict resolution is deterministic: unchanged side wins; otherwise retain
|
Conflict resolution is deterministic: unchanged side wins; otherwise retain
|
||||||
both versions and mark a conflict for the user. Network work happens outside the
|
both versions and mark a conflict for the user. Network work happens outside the
|
||||||
render loop and publishes immutable snapshots to it.
|
render loop and publishes immutable snapshots to it.
|
||||||
|
|
||||||
|
The first provider-facing step after the cache is verified is an injectable HTTP
|
||||||
|
boundary, browser-based authorization-code flow with PKCE, and fake transports
|
||||||
|
for deterministic authentication and failure tests. Read-only Microsoft Graph
|
||||||
|
calendar discovery and delta synchronization follows. Graph delta links remain
|
||||||
|
opaque and are persisted only through the calendar/window transaction described
|
||||||
|
above; remote writes and conflict resolution come later.
|
||||||
|
|||||||
@@ -112,6 +112,29 @@ values incompatible with the event's `DTSTART` time basis or `TZID`, detached
|
|||||||
read-only. Remaining local-data work includes detached overrides and rule
|
read-only. Remaining local-data work includes detached overrides and rule
|
||||||
editing.
|
editing.
|
||||||
|
|
||||||
|
## Remote calendar foundation
|
||||||
|
|
||||||
|
Remote calendars must cross a durable local boundary before they are exposed to
|
||||||
|
the UI. That boundary is a private, provider-neutral SQLite cache in WAL mode,
|
||||||
|
with versioned migrations that either commit completely or leave the prior
|
||||||
|
schema intact. It keeps raw provider payloads, inactive calendars, and
|
||||||
|
tombstoned events so refreshes and older clients do not erase information. Each
|
||||||
|
opaque incremental cursor belongs to a particular calendar and time window and
|
||||||
|
is committed atomically with every pulled page. Normalized event instances feed
|
||||||
|
immutable snapshots rather than exposing provider response types to the month
|
||||||
|
view.
|
||||||
|
|
||||||
|
The cache also reserves transactional outbox and conflict records for future
|
||||||
|
offline writes, but remote mutation behavior is not part of this phase. OAuth
|
||||||
|
tokens and other credentials never belong in the calendar database; account
|
||||||
|
secrets will be held by the Freedesktop Secret Service.
|
||||||
|
|
||||||
|
This foundation does not make Office 365 usable yet: Nocal currently performs no
|
||||||
|
OAuth flow, HTTP request, or Microsoft Graph synchronization. The next product
|
||||||
|
slice introduces injectable HTTP and PKCE authentication with fake transports,
|
||||||
|
then adds read-only Graph delta synchronization. Remote writes remain gated on
|
||||||
|
the durability and conflict paths being exercised end to end.
|
||||||
|
|
||||||
## Local file interchange
|
## Local file interchange
|
||||||
|
|
||||||
Import and export are explicit command-line workflows. They never contact the
|
Import and export are explicit command-line workflows. They never contact the
|
||||||
|
|||||||
@@ -24,12 +24,22 @@
|
|||||||
- Embedded VTIMEZONE support
|
- Embedded VTIMEZONE support
|
||||||
- Screen-reader-friendly linear view and full Unicode display-width handling
|
- Screen-reader-friendly linear view and full Unicode display-width handling
|
||||||
|
|
||||||
## 0.2 — durable cache and provider boundary
|
## 0.2 — durable cache and provider boundary — in progress
|
||||||
|
|
||||||
- SQLite WAL cache and migration framework
|
Completed cache foundation:
|
||||||
- Background job queue, immutable UI snapshots, observable sync status
|
|
||||||
- Secret Service credentials and OAuth callback helper
|
- Provider-neutral SQLite WAL cache in a private local file
|
||||||
- Generic `SyncProvider` contract plus a fake provider test suite
|
- Versioned, all-or-nothing schema migrations
|
||||||
|
- Raw provider payload retention, inactive calendars, and event tombstones
|
||||||
|
- Per-calendar/window opaque cursors committed atomically with pulled pages
|
||||||
|
- Normalized event instances for immutable snapshots
|
||||||
|
- Reserved transactional outbox and conflict records for later remote writes
|
||||||
|
|
||||||
|
Next provider-boundary slices:
|
||||||
|
|
||||||
|
- Injectable HTTP, PKCE authentication, and deterministic fake transports
|
||||||
|
- Secret Service credentials; OAuth secrets are never stored in SQLite
|
||||||
|
- Generic `SyncProvider` contract and observable sync status
|
||||||
|
|
||||||
## 0.3 — CalDAV
|
## 0.3 — CalDAV
|
||||||
|
|
||||||
@@ -40,9 +50,15 @@
|
|||||||
## 0.4 — hosted providers
|
## 0.4 — hosted providers
|
||||||
|
|
||||||
- Google Calendar adapter with incremental synchronization
|
- Google Calendar adapter with incremental synchronization
|
||||||
- Microsoft 365 adapter over Microsoft Graph delta queries
|
- Microsoft 365 read-only discovery and Graph delta synchronization, followed by
|
||||||
|
ETag-aware remote writes after the conflict boundary is proven
|
||||||
- Account/calendar management UI and per-calendar ANSI identity
|
- Account/calendar management UI and per-calendar ANSI identity
|
||||||
|
|
||||||
|
The current cache work is only the prerequisite durable boundary. No network,
|
||||||
|
OAuth, or Graph synchronization is implemented in that phase. After the cache
|
||||||
|
passes its verification gates, the immediate sequence is injectable HTTP plus
|
||||||
|
PKCE and fake transports, then read-only Microsoft Graph delta synchronization.
|
||||||
|
|
||||||
## 1.0 — distribution quality
|
## 1.0 — distribution quality
|
||||||
|
|
||||||
- Stable configuration/data format and documented recovery procedures
|
- Stable configuration/data format and documented recovery procedures
|
||||||
|
|||||||
122
include/nocal/storage/sync_cache.hpp
Normal file
122
include/nocal/storage/sync_cache.hpp
Normal file
@@ -0,0 +1,122 @@
|
|||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <cstdint>
|
||||||
|
#include <filesystem>
|
||||||
|
#include <memory>
|
||||||
|
#include <span>
|
||||||
|
#include <string>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
namespace nocal::storage {
|
||||||
|
|
||||||
|
struct CachedAccount {
|
||||||
|
std::string id;
|
||||||
|
std::string provider;
|
||||||
|
std::string remote_subject;
|
||||||
|
std::string display_name;
|
||||||
|
|
||||||
|
bool operator==(const CachedAccount&) const = default;
|
||||||
|
};
|
||||||
|
|
||||||
|
struct CachedCalendar {
|
||||||
|
std::string id;
|
||||||
|
std::string account_id;
|
||||||
|
std::string remote_id;
|
||||||
|
std::string name;
|
||||||
|
std::string color;
|
||||||
|
bool read_only{false};
|
||||||
|
bool active{true};
|
||||||
|
std::string raw_payload;
|
||||||
|
|
||||||
|
bool operator==(const CachedCalendar&) const = default;
|
||||||
|
};
|
||||||
|
|
||||||
|
struct CachedEvent {
|
||||||
|
std::string id;
|
||||||
|
std::string calendar_id;
|
||||||
|
std::string remote_id;
|
||||||
|
std::string uid;
|
||||||
|
std::string etag;
|
||||||
|
std::string change_key;
|
||||||
|
std::string raw_payload;
|
||||||
|
bool deleted{false};
|
||||||
|
|
||||||
|
bool operator==(const CachedEvent&) const = default;
|
||||||
|
};
|
||||||
|
|
||||||
|
struct CachedEventInstance {
|
||||||
|
std::string id;
|
||||||
|
std::string event_id;
|
||||||
|
std::int64_t start_epoch_microseconds{0};
|
||||||
|
std::int64_t end_epoch_microseconds{0};
|
||||||
|
bool all_day{false};
|
||||||
|
std::string title;
|
||||||
|
std::string location;
|
||||||
|
std::string description;
|
||||||
|
std::string time_zone;
|
||||||
|
|
||||||
|
bool operator==(const CachedEventInstance&) const = default;
|
||||||
|
};
|
||||||
|
|
||||||
|
struct SyncCheckpoint {
|
||||||
|
std::string calendar_id;
|
||||||
|
std::string window_start;
|
||||||
|
std::string window_end;
|
||||||
|
std::string cursor;
|
||||||
|
bool complete{false};
|
||||||
|
|
||||||
|
bool operator==(const SyncCheckpoint&) const = default;
|
||||||
|
};
|
||||||
|
|
||||||
|
struct PullPage {
|
||||||
|
std::vector<CachedEvent> events;
|
||||||
|
std::vector<CachedEventInstance> instances;
|
||||||
|
SyncCheckpoint checkpoint;
|
||||||
|
|
||||||
|
bool operator==(const PullPage&) const = default;
|
||||||
|
};
|
||||||
|
|
||||||
|
struct CacheSnapshot {
|
||||||
|
std::vector<CachedAccount> accounts;
|
||||||
|
std::vector<CachedCalendar> calendars;
|
||||||
|
std::vector<CachedEvent> events;
|
||||||
|
std::vector<CachedEventInstance> instances;
|
||||||
|
std::vector<SyncCheckpoint> checkpoints;
|
||||||
|
|
||||||
|
bool operator==(const CacheSnapshot&) const = default;
|
||||||
|
};
|
||||||
|
|
||||||
|
class SyncCache {
|
||||||
|
public:
|
||||||
|
static constexpr int current_schema_version = 1;
|
||||||
|
|
||||||
|
explicit SyncCache(std::filesystem::path path);
|
||||||
|
~SyncCache();
|
||||||
|
|
||||||
|
SyncCache(const SyncCache&) = delete;
|
||||||
|
SyncCache& operator=(const SyncCache&) = delete;
|
||||||
|
SyncCache(SyncCache&&) noexcept;
|
||||||
|
SyncCache& operator=(SyncCache&&) noexcept;
|
||||||
|
|
||||||
|
[[nodiscard]] const std::filesystem::path& path() const noexcept;
|
||||||
|
|
||||||
|
void upsert_account(const CachedAccount& account);
|
||||||
|
|
||||||
|
// Call only after a provider calendar listing has completed. Calendars
|
||||||
|
// omitted from the completed listing are retained but marked inactive.
|
||||||
|
void replace_calendars_after_complete_listing(
|
||||||
|
std::string account_id, std::span<const CachedCalendar> calendars);
|
||||||
|
|
||||||
|
// Applies one provider page and advances its opaque cursor in the same
|
||||||
|
// transaction. Instances for each changed event are replaced; tombstoned
|
||||||
|
// events retain their last raw payload and have no instances.
|
||||||
|
void apply_pull_page(const PullPage& page);
|
||||||
|
|
||||||
|
[[nodiscard]] CacheSnapshot snapshot() const;
|
||||||
|
|
||||||
|
private:
|
||||||
|
struct Impl;
|
||||||
|
std::unique_ptr<Impl> impl_;
|
||||||
|
};
|
||||||
|
|
||||||
|
} // namespace nocal::storage
|
||||||
28
meson.build
28
meson.build
@@ -12,40 +12,48 @@ project(
|
|||||||
)
|
)
|
||||||
|
|
||||||
inc = include_directories('include')
|
inc = include_directories('include')
|
||||||
|
sqlite = dependency('sqlite3')
|
||||||
|
threads = dependency('threads')
|
||||||
nocal_sources = files(
|
nocal_sources = files(
|
||||||
'src/domain/calendar_transfer.cpp',
|
'src/domain/calendar_transfer.cpp',
|
||||||
'src/domain/date.cpp',
|
'src/domain/date.cpp',
|
||||||
'src/domain/event_query.cpp',
|
'src/domain/event_query.cpp',
|
||||||
'src/domain/month_layout.cpp',
|
'src/domain/month_layout.cpp',
|
||||||
'src/storage/ics_store.cpp',
|
'src/storage/ics_store.cpp',
|
||||||
|
'src/storage/sync_cache.cpp',
|
||||||
'src/tui/Screen.cpp',
|
'src/tui/Screen.cpp',
|
||||||
'src/tui/EventEditor.cpp',
|
'src/tui/EventEditor.cpp',
|
||||||
'src/tui/Terminal.cpp',
|
'src/tui/Terminal.cpp',
|
||||||
'src/tui/TuiApp.cpp',
|
'src/tui/TuiApp.cpp',
|
||||||
)
|
)
|
||||||
nocal_lib = static_library('nocal-core', nocal_sources, include_directories: inc)
|
nocal_lib = static_library('nocal-core', nocal_sources, include_directories: inc,
|
||||||
|
dependencies: sqlite)
|
||||||
|
nocal_dep = declare_dependency(include_directories: inc, link_with: nocal_lib,
|
||||||
|
dependencies: sqlite)
|
||||||
|
|
||||||
nocal_executable = executable('nocal', 'src/main.cpp', include_directories: inc,
|
nocal_executable = executable('nocal', 'src/main.cpp',
|
||||||
link_with: nocal_lib, install: true)
|
dependencies: nocal_dep, install: true)
|
||||||
|
|
||||||
domain_tests = executable('domain-tests', 'tests/domain_tests.cpp',
|
domain_tests = executable('domain-tests', 'tests/domain_tests.cpp',
|
||||||
include_directories: inc, link_with: nocal_lib)
|
dependencies: nocal_dep)
|
||||||
test('domain', domain_tests)
|
test('domain', domain_tests)
|
||||||
calendar_transfer_tests = executable('calendar-transfer-tests',
|
calendar_transfer_tests = executable('calendar-transfer-tests',
|
||||||
'tests/calendar_transfer_tests.cpp', include_directories: inc,
|
'tests/calendar_transfer_tests.cpp', dependencies: nocal_dep)
|
||||||
link_with: nocal_lib)
|
|
||||||
test('calendar-transfer', calendar_transfer_tests)
|
test('calendar-transfer', calendar_transfer_tests)
|
||||||
ics_tests = executable('ics-tests', 'tests/ics_tests.cpp',
|
ics_tests = executable('ics-tests', 'tests/ics_tests.cpp',
|
||||||
include_directories: inc, link_with: nocal_lib)
|
dependencies: nocal_dep)
|
||||||
test('ics-storage', ics_tests)
|
test('ics-storage', ics_tests)
|
||||||
|
sync_cache_tests = executable('sync-cache-tests', 'tests/sync_cache_tests.cpp',
|
||||||
|
dependencies: [nocal_dep, threads])
|
||||||
|
test('sync-cache', sync_cache_tests)
|
||||||
tui_tests = executable('tui-tests', 'tests/tui_tests.cpp',
|
tui_tests = executable('tui-tests', 'tests/tui_tests.cpp',
|
||||||
include_directories: inc, link_with: nocal_lib)
|
dependencies: nocal_dep)
|
||||||
test('tui', tui_tests)
|
test('tui', tui_tests)
|
||||||
tui_focus_tests = executable('tui-focus-tests', 'tests/tui_focus_tests.cpp',
|
tui_focus_tests = executable('tui-focus-tests', 'tests/tui_focus_tests.cpp',
|
||||||
include_directories: inc, link_with: nocal_lib)
|
dependencies: nocal_dep)
|
||||||
test('tui-focus', tui_focus_tests)
|
test('tui-focus', tui_focus_tests)
|
||||||
editor_tests = executable('editor-tests', 'tests/editor_tests.cpp',
|
editor_tests = executable('editor-tests', 'tests/editor_tests.cpp',
|
||||||
include_directories: inc, link_with: nocal_lib)
|
dependencies: nocal_dep)
|
||||||
test('editor', editor_tests)
|
test('editor', editor_tests)
|
||||||
|
|
||||||
shell = find_program('sh')
|
shell = find_program('sh')
|
||||||
|
|||||||
@@ -9,5 +9,6 @@ pkgs.mkShell {
|
|||||||
|
|
||||||
buildInputs = with pkgs; [
|
buildInputs = with pkgs; [
|
||||||
gcc
|
gcc
|
||||||
|
sqlite
|
||||||
];
|
];
|
||||||
}
|
}
|
||||||
|
|||||||
738
src/storage/sync_cache.cpp
Normal file
738
src/storage/sync_cache.cpp
Normal file
@@ -0,0 +1,738 @@
|
|||||||
|
#include "nocal/storage/sync_cache.hpp"
|
||||||
|
|
||||||
|
#include <sqlite3.h>
|
||||||
|
|
||||||
|
#include <cerrno>
|
||||||
|
#include <cstring>
|
||||||
|
#include <filesystem>
|
||||||
|
#include <limits>
|
||||||
|
#include <mutex>
|
||||||
|
#include <stdexcept>
|
||||||
|
#include <string>
|
||||||
|
#include <string_view>
|
||||||
|
#include <unordered_set>
|
||||||
|
#include <utility>
|
||||||
|
|
||||||
|
#include <fcntl.h>
|
||||||
|
#include <sys/stat.h>
|
||||||
|
#include <unistd.h>
|
||||||
|
|
||||||
|
namespace nocal::storage {
|
||||||
|
namespace {
|
||||||
|
|
||||||
|
[[noreturn]] void throw_sqlite(sqlite3* database, std::string_view operation) {
|
||||||
|
const char* detail = database == nullptr ? nullptr : sqlite3_errmsg(database);
|
||||||
|
throw std::runtime_error(
|
||||||
|
"sync cache " + std::string(operation) + " failed"
|
||||||
|
+ (detail == nullptr ? std::string{} : ": " + std::string(detail)));
|
||||||
|
}
|
||||||
|
|
||||||
|
void execute(sqlite3* database, const char* sql, std::string_view operation) {
|
||||||
|
char* detail = nullptr;
|
||||||
|
const int result = sqlite3_exec(database, sql, nullptr, nullptr, &detail);
|
||||||
|
if (result == SQLITE_OK) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const std::string message = detail == nullptr ? sqlite3_errmsg(database) : detail;
|
||||||
|
sqlite3_free(detail);
|
||||||
|
throw std::runtime_error(
|
||||||
|
"sync cache " + std::string(operation) + " failed: " + message);
|
||||||
|
}
|
||||||
|
|
||||||
|
class Statement {
|
||||||
|
public:
|
||||||
|
Statement(sqlite3* database, const char* sql, std::string_view operation)
|
||||||
|
: database_(database), operation_(operation) {
|
||||||
|
if (sqlite3_prepare_v2(database_, sql, -1, &statement_, nullptr) != SQLITE_OK) {
|
||||||
|
throw_sqlite(database_, operation_);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
~Statement() { sqlite3_finalize(statement_); }
|
||||||
|
|
||||||
|
Statement(const Statement&) = delete;
|
||||||
|
Statement& operator=(const Statement&) = delete;
|
||||||
|
|
||||||
|
void bind_text(int index, std::string_view value) {
|
||||||
|
if (value.size() > static_cast<std::size_t>(std::numeric_limits<int>::max())
|
||||||
|
|| sqlite3_bind_text(statement_, index, value.data(), static_cast<int>(value.size()),
|
||||||
|
SQLITE_TRANSIENT)
|
||||||
|
!= SQLITE_OK) {
|
||||||
|
throw_sqlite(database_, operation_);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void bind_int64(int index, std::int64_t value) {
|
||||||
|
if (sqlite3_bind_int64(statement_, index, value) != SQLITE_OK) {
|
||||||
|
throw_sqlite(database_, operation_);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void bind_bool(int index, bool value) { bind_int64(index, value ? 1 : 0); }
|
||||||
|
|
||||||
|
[[nodiscard]] bool next() {
|
||||||
|
const int result = sqlite3_step(statement_);
|
||||||
|
if (result == SQLITE_ROW) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
if (result == SQLITE_DONE) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
throw_sqlite(database_, operation_);
|
||||||
|
}
|
||||||
|
|
||||||
|
void run() {
|
||||||
|
if (next()) {
|
||||||
|
throw std::runtime_error(
|
||||||
|
"sync cache " + std::string(operation_) + " returned an unexpected row");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
[[nodiscard]] std::string text_column(int index) const {
|
||||||
|
const unsigned char* value = sqlite3_column_text(statement_, index);
|
||||||
|
const int size = sqlite3_column_bytes(statement_, index);
|
||||||
|
if (value == nullptr || size == 0) {
|
||||||
|
return {};
|
||||||
|
}
|
||||||
|
return {reinterpret_cast<const char*>(value), static_cast<std::size_t>(size)};
|
||||||
|
}
|
||||||
|
|
||||||
|
[[nodiscard]] std::int64_t int64_column(int index) const {
|
||||||
|
return sqlite3_column_int64(statement_, index);
|
||||||
|
}
|
||||||
|
|
||||||
|
[[nodiscard]] bool bool_column(int index) const {
|
||||||
|
return sqlite3_column_int(statement_, index) != 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
private:
|
||||||
|
sqlite3* database_;
|
||||||
|
sqlite3_stmt* statement_{nullptr};
|
||||||
|
std::string operation_;
|
||||||
|
};
|
||||||
|
|
||||||
|
class Transaction {
|
||||||
|
public:
|
||||||
|
explicit Transaction(sqlite3* database) : database_(database) {
|
||||||
|
execute(database_, "BEGIN IMMEDIATE", "starting transaction");
|
||||||
|
}
|
||||||
|
|
||||||
|
~Transaction() {
|
||||||
|
if (!finished_) {
|
||||||
|
sqlite3_exec(database_, "ROLLBACK", nullptr, nullptr, nullptr);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Transaction(const Transaction&) = delete;
|
||||||
|
Transaction& operator=(const Transaction&) = delete;
|
||||||
|
|
||||||
|
void commit() {
|
||||||
|
execute(database_, "COMMIT", "committing transaction");
|
||||||
|
finished_ = true;
|
||||||
|
}
|
||||||
|
|
||||||
|
private:
|
||||||
|
sqlite3* database_;
|
||||||
|
bool finished_{false};
|
||||||
|
};
|
||||||
|
|
||||||
|
[[nodiscard]] std::string path_error(const std::filesystem::path& path, std::string_view reason) {
|
||||||
|
return "invalid sync cache path '" + path.string() + "': " + std::string(reason);
|
||||||
|
}
|
||||||
|
|
||||||
|
[[nodiscard]] bool prepare_database_file(const std::filesystem::path& path) {
|
||||||
|
if (path.empty() || !path.has_filename() || path.filename().empty()
|
||||||
|
|| path.filename() == "." || path.filename() == ".."
|
||||||
|
|| path.native().find('\0') != std::string::npos) {
|
||||||
|
throw std::invalid_argument(path_error(path, "a database filename is required"));
|
||||||
|
}
|
||||||
|
|
||||||
|
struct stat link_status {};
|
||||||
|
bool created = false;
|
||||||
|
int descriptor = -1;
|
||||||
|
if (::lstat(path.c_str(), &link_status) == 0) {
|
||||||
|
if (S_ISLNK(link_status.st_mode) || !S_ISREG(link_status.st_mode)) {
|
||||||
|
throw std::invalid_argument(path_error(path, "existing target must be a regular file"));
|
||||||
|
}
|
||||||
|
descriptor = ::open(path.c_str(), O_RDWR | O_CLOEXEC | O_NOFOLLOW);
|
||||||
|
} else if (errno == ENOENT) {
|
||||||
|
descriptor = ::open(path.c_str(),
|
||||||
|
O_RDWR | O_CREAT | O_EXCL | O_CLOEXEC | O_NOFOLLOW, S_IRUSR | S_IWUSR);
|
||||||
|
created = descriptor >= 0;
|
||||||
|
} else {
|
||||||
|
throw std::runtime_error(path_error(path, std::strerror(errno)));
|
||||||
|
}
|
||||||
|
if (descriptor < 0) {
|
||||||
|
throw std::runtime_error(path_error(path, std::strerror(errno)));
|
||||||
|
}
|
||||||
|
struct stat opened_status {};
|
||||||
|
if (::fstat(descriptor, &opened_status) != 0) {
|
||||||
|
const int error = errno;
|
||||||
|
::close(descriptor);
|
||||||
|
if (created) {
|
||||||
|
std::error_code ignored;
|
||||||
|
std::filesystem::remove(path, ignored);
|
||||||
|
}
|
||||||
|
throw std::runtime_error(path_error(path, std::strerror(error)));
|
||||||
|
}
|
||||||
|
if (!S_ISREG(opened_status.st_mode)) {
|
||||||
|
::close(descriptor);
|
||||||
|
if (created) {
|
||||||
|
std::error_code ignored;
|
||||||
|
std::filesystem::remove(path, ignored);
|
||||||
|
}
|
||||||
|
throw std::invalid_argument(path_error(path, "opened target is not a regular file"));
|
||||||
|
}
|
||||||
|
if (::fchmod(descriptor, S_IRUSR | S_IWUSR) != 0) {
|
||||||
|
const int error = errno;
|
||||||
|
::close(descriptor);
|
||||||
|
if (created) {
|
||||||
|
std::error_code ignored;
|
||||||
|
std::filesystem::remove(path, ignored);
|
||||||
|
}
|
||||||
|
throw std::runtime_error(path_error(path, std::strerror(error)));
|
||||||
|
}
|
||||||
|
if (::fstat(descriptor, &opened_status) != 0) {
|
||||||
|
const int error = errno;
|
||||||
|
::close(descriptor);
|
||||||
|
if (created) {
|
||||||
|
std::error_code ignored;
|
||||||
|
std::filesystem::remove(path, ignored);
|
||||||
|
}
|
||||||
|
throw std::runtime_error(path_error(path, std::strerror(error)));
|
||||||
|
}
|
||||||
|
if ((opened_status.st_mode & (S_IRWXU | S_IRWXG | S_IRWXO)) != (S_IRUSR | S_IWUSR)) {
|
||||||
|
::close(descriptor);
|
||||||
|
if (created) {
|
||||||
|
std::error_code ignored;
|
||||||
|
std::filesystem::remove(path, ignored);
|
||||||
|
}
|
||||||
|
throw std::runtime_error(path_error(path, "database permissions are not private"));
|
||||||
|
}
|
||||||
|
if (::close(descriptor) != 0) {
|
||||||
|
const int error = errno;
|
||||||
|
if (created) {
|
||||||
|
std::error_code ignored;
|
||||||
|
std::filesystem::remove(path, ignored);
|
||||||
|
}
|
||||||
|
throw std::runtime_error(path_error(path, std::strerror(error)));
|
||||||
|
}
|
||||||
|
return created;
|
||||||
|
}
|
||||||
|
|
||||||
|
[[nodiscard]] int user_version(sqlite3* database) {
|
||||||
|
Statement statement(database, "PRAGMA user_version", "reading schema version");
|
||||||
|
if (!statement.next()) {
|
||||||
|
throw std::runtime_error("sync cache schema version is unavailable");
|
||||||
|
}
|
||||||
|
const std::int64_t version = statement.int64_column(0);
|
||||||
|
if (version < 0 || version > std::numeric_limits<int>::max()) {
|
||||||
|
throw std::runtime_error("sync cache schema version is invalid");
|
||||||
|
}
|
||||||
|
return static_cast<int>(version);
|
||||||
|
}
|
||||||
|
|
||||||
|
void require_pragma_integer(
|
||||||
|
sqlite3* database, const char* sql, std::int64_t expected, std::string_view operation) {
|
||||||
|
Statement statement(database, sql, operation);
|
||||||
|
if (!statement.next() || statement.int64_column(0) != expected) {
|
||||||
|
throw std::runtime_error("sync cache " + std::string(operation) + " was not applied");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void require_pragma_text(
|
||||||
|
sqlite3* database, const char* sql, std::string_view expected, std::string_view operation) {
|
||||||
|
Statement statement(database, sql, operation);
|
||||||
|
if (!statement.next() || statement.text_column(0) != expected) {
|
||||||
|
throw std::runtime_error("sync cache " + std::string(operation) + " was not applied");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
constexpr const char* schema_v1 = R"sql(
|
||||||
|
CREATE TABLE accounts (
|
||||||
|
id TEXT PRIMARY KEY CHECK(length(id) > 0),
|
||||||
|
provider TEXT NOT NULL CHECK(length(provider) > 0),
|
||||||
|
remote_subject TEXT NOT NULL CHECK(length(remote_subject) > 0),
|
||||||
|
display_name TEXT NOT NULL,
|
||||||
|
UNIQUE(provider, remote_subject)
|
||||||
|
) STRICT;
|
||||||
|
CREATE TABLE calendars (
|
||||||
|
id TEXT PRIMARY KEY CHECK(length(id) > 0),
|
||||||
|
account_id TEXT NOT NULL REFERENCES accounts(id) ON UPDATE RESTRICT ON DELETE RESTRICT,
|
||||||
|
remote_id TEXT NOT NULL CHECK(length(remote_id) > 0),
|
||||||
|
name TEXT NOT NULL,
|
||||||
|
color TEXT NOT NULL,
|
||||||
|
read_only INTEGER NOT NULL CHECK(read_only IN (0, 1)),
|
||||||
|
active INTEGER NOT NULL CHECK(active IN (0, 1)),
|
||||||
|
raw_payload TEXT NOT NULL,
|
||||||
|
UNIQUE(account_id, remote_id)
|
||||||
|
) STRICT;
|
||||||
|
CREATE INDEX calendars_account_id_idx ON calendars(account_id);
|
||||||
|
CREATE TABLE events (
|
||||||
|
id TEXT PRIMARY KEY CHECK(length(id) > 0),
|
||||||
|
calendar_id TEXT NOT NULL REFERENCES calendars(id) ON UPDATE RESTRICT ON DELETE RESTRICT,
|
||||||
|
remote_id TEXT NOT NULL CHECK(length(remote_id) > 0),
|
||||||
|
uid TEXT NOT NULL,
|
||||||
|
etag TEXT NOT NULL,
|
||||||
|
change_key TEXT NOT NULL,
|
||||||
|
raw_payload TEXT NOT NULL,
|
||||||
|
deleted INTEGER NOT NULL CHECK(deleted IN (0, 1)),
|
||||||
|
UNIQUE(calendar_id, remote_id)
|
||||||
|
) STRICT;
|
||||||
|
CREATE INDEX events_calendar_id_idx ON events(calendar_id);
|
||||||
|
CREATE TABLE event_instances (
|
||||||
|
id TEXT PRIMARY KEY CHECK(length(id) > 0),
|
||||||
|
event_id TEXT NOT NULL REFERENCES events(id) ON UPDATE RESTRICT ON DELETE CASCADE,
|
||||||
|
start_epoch_microseconds INTEGER NOT NULL,
|
||||||
|
end_epoch_microseconds INTEGER NOT NULL,
|
||||||
|
all_day INTEGER NOT NULL CHECK(all_day IN (0, 1)),
|
||||||
|
title TEXT NOT NULL,
|
||||||
|
location TEXT NOT NULL,
|
||||||
|
description TEXT NOT NULL,
|
||||||
|
time_zone TEXT NOT NULL,
|
||||||
|
CHECK(start_epoch_microseconds <= end_epoch_microseconds),
|
||||||
|
CHECK(all_day = 0 OR start_epoch_microseconds < end_epoch_microseconds)
|
||||||
|
) STRICT;
|
||||||
|
CREATE INDEX event_instances_event_id_idx ON event_instances(event_id);
|
||||||
|
CREATE TABLE sync_state (
|
||||||
|
calendar_id TEXT NOT NULL REFERENCES calendars(id) ON UPDATE RESTRICT ON DELETE CASCADE,
|
||||||
|
window_start TEXT NOT NULL,
|
||||||
|
window_end TEXT NOT NULL,
|
||||||
|
cursor TEXT NOT NULL CHECK(length(cursor) > 0),
|
||||||
|
complete INTEGER NOT NULL CHECK(complete IN (0, 1)),
|
||||||
|
CHECK((length(window_start) = 0) = (length(window_end) = 0)),
|
||||||
|
PRIMARY KEY(calendar_id, window_start, window_end)
|
||||||
|
) STRICT;
|
||||||
|
CREATE TABLE outbox (
|
||||||
|
id TEXT PRIMARY KEY CHECK(length(id) > 0),
|
||||||
|
calendar_id TEXT NOT NULL REFERENCES calendars(id) ON UPDATE RESTRICT ON DELETE RESTRICT,
|
||||||
|
event_id TEXT NOT NULL REFERENCES events(id) ON UPDATE RESTRICT ON DELETE RESTRICT,
|
||||||
|
operation TEXT NOT NULL CHECK(length(operation) > 0),
|
||||||
|
payload TEXT NOT NULL,
|
||||||
|
base_etag TEXT NOT NULL,
|
||||||
|
idempotency_key TEXT NOT NULL UNIQUE CHECK(length(idempotency_key) > 0),
|
||||||
|
status TEXT NOT NULL CHECK(length(status) > 0),
|
||||||
|
attempt_count INTEGER NOT NULL DEFAULT 0 CHECK(attempt_count >= 0),
|
||||||
|
not_before_epoch_seconds INTEGER
|
||||||
|
) STRICT;
|
||||||
|
CREATE TABLE conflicts (
|
||||||
|
id TEXT PRIMARY KEY CHECK(length(id) > 0),
|
||||||
|
calendar_id TEXT NOT NULL REFERENCES calendars(id) ON UPDATE RESTRICT ON DELETE RESTRICT,
|
||||||
|
event_id TEXT NOT NULL REFERENCES events(id) ON UPDATE RESTRICT ON DELETE RESTRICT,
|
||||||
|
base_etag TEXT NOT NULL,
|
||||||
|
local_payload TEXT NOT NULL,
|
||||||
|
remote_payload TEXT NOT NULL,
|
||||||
|
status TEXT NOT NULL CHECK(length(status) > 0)
|
||||||
|
) STRICT;
|
||||||
|
)sql";
|
||||||
|
|
||||||
|
void validate_nonempty(std::string_view value, std::string_view field) {
|
||||||
|
if (value.empty()) {
|
||||||
|
throw std::invalid_argument(std::string(field) + " must not be empty");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void require_account(sqlite3* database, std::string_view account_id) {
|
||||||
|
Statement statement(database, "SELECT 1 FROM accounts WHERE id = ?", "checking account");
|
||||||
|
statement.bind_text(1, account_id);
|
||||||
|
if (!statement.next()) {
|
||||||
|
throw std::invalid_argument("sync cache account does not exist");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void ensure_account_identity(sqlite3* database, const CachedAccount& account) {
|
||||||
|
Statement statement(database,
|
||||||
|
"SELECT provider, remote_subject FROM accounts WHERE id = ?", "checking account identity");
|
||||||
|
statement.bind_text(1, account.id);
|
||||||
|
if (statement.next()
|
||||||
|
&& (statement.text_column(0) != account.provider
|
||||||
|
|| statement.text_column(1) != account.remote_subject)) {
|
||||||
|
throw std::invalid_argument("account identity cannot be reassigned");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void require_active_calendar(sqlite3* database, std::string_view calendar_id) {
|
||||||
|
Statement statement(
|
||||||
|
database, "SELECT active FROM calendars WHERE id = ?", "checking calendar");
|
||||||
|
statement.bind_text(1, calendar_id);
|
||||||
|
if (!statement.next()) {
|
||||||
|
throw std::invalid_argument("sync cache calendar does not exist");
|
||||||
|
}
|
||||||
|
if (!statement.bool_column(0)) {
|
||||||
|
throw std::invalid_argument("cannot apply a pull page to an inactive calendar");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void ensure_calendar_identity(sqlite3* database, const CachedCalendar& calendar) {
|
||||||
|
Statement statement(database,
|
||||||
|
"SELECT account_id, remote_id FROM calendars WHERE id = ?",
|
||||||
|
"checking calendar identity");
|
||||||
|
statement.bind_text(1, calendar.id);
|
||||||
|
if (statement.next()
|
||||||
|
&& (statement.text_column(0) != calendar.account_id
|
||||||
|
|| statement.text_column(1) != calendar.remote_id)) {
|
||||||
|
throw std::invalid_argument("calendar identity cannot be reassigned");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void ensure_event_identity(sqlite3* database, const CachedEvent& event) {
|
||||||
|
Statement statement(database,
|
||||||
|
"SELECT calendar_id, remote_id FROM events WHERE id = ?", "checking event identity");
|
||||||
|
statement.bind_text(1, event.id);
|
||||||
|
if (statement.next()
|
||||||
|
&& (statement.text_column(0) != event.calendar_id
|
||||||
|
|| statement.text_column(1) != event.remote_id)) {
|
||||||
|
throw std::invalid_argument("event identity cannot be reassigned");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace
|
||||||
|
|
||||||
|
struct SyncCache::Impl {
|
||||||
|
explicit Impl(std::filesystem::path cache_path) : path(std::move(cache_path)) {
|
||||||
|
const bool created = prepare_database_file(path);
|
||||||
|
sqlite3* opened = nullptr;
|
||||||
|
const int result = sqlite3_open_v2(path.c_str(), &opened,
|
||||||
|
SQLITE_OPEN_READWRITE | SQLITE_OPEN_FULLMUTEX | SQLITE_OPEN_PRIVATECACHE
|
||||||
|
| SQLITE_OPEN_NOFOLLOW,
|
||||||
|
nullptr);
|
||||||
|
if (result != SQLITE_OK) {
|
||||||
|
const std::string detail = opened == nullptr ? sqlite3_errstr(result) : sqlite3_errmsg(opened);
|
||||||
|
if (opened != nullptr) {
|
||||||
|
sqlite3_close(opened);
|
||||||
|
}
|
||||||
|
if (created) {
|
||||||
|
std::error_code ignored;
|
||||||
|
std::filesystem::remove(path, ignored);
|
||||||
|
}
|
||||||
|
throw std::runtime_error("unable to open sync cache '" + path.string() + "': " + detail);
|
||||||
|
}
|
||||||
|
database = opened;
|
||||||
|
|
||||||
|
try {
|
||||||
|
if (sqlite3_extended_result_codes(database, 1) != SQLITE_OK
|
||||||
|
|| sqlite3_busy_timeout(database, 5'000) != SQLITE_OK) {
|
||||||
|
throw_sqlite(database, "configuring connection error handling");
|
||||||
|
}
|
||||||
|
execute(database, "PRAGMA trusted_schema = OFF", "disabling trusted schema");
|
||||||
|
require_pragma_integer(
|
||||||
|
database, "PRAGMA trusted_schema", 0, "disabling trusted schema");
|
||||||
|
#ifdef SQLITE_DBCONFIG_DEFENSIVE
|
||||||
|
if (sqlite3_db_config(database, SQLITE_DBCONFIG_DEFENSIVE, 1, nullptr) != SQLITE_OK) {
|
||||||
|
throw_sqlite(database, "enabling defensive mode");
|
||||||
|
}
|
||||||
|
#endif
|
||||||
|
const int version = user_version(database);
|
||||||
|
if (version > SyncCache::current_schema_version) {
|
||||||
|
throw std::runtime_error("sync cache schema version " + std::to_string(version)
|
||||||
|
+ " is newer than supported version "
|
||||||
|
+ std::to_string(SyncCache::current_schema_version));
|
||||||
|
}
|
||||||
|
execute(database, "PRAGMA foreign_keys = ON", "enabling foreign keys");
|
||||||
|
require_pragma_integer(database, "PRAGMA foreign_keys", 1, "enabling foreign keys");
|
||||||
|
if (version == 0) {
|
||||||
|
Transaction migration(database);
|
||||||
|
const int locked_version = user_version(database);
|
||||||
|
if (locked_version == 0) {
|
||||||
|
execute(database, schema_v1, "creating schema version 1");
|
||||||
|
execute(database, "PRAGMA user_version = 1", "recording schema version 1");
|
||||||
|
} else if (locked_version > SyncCache::current_schema_version) {
|
||||||
|
throw std::runtime_error("sync cache schema changed to an unsupported version");
|
||||||
|
}
|
||||||
|
migration.commit();
|
||||||
|
}
|
||||||
|
execute(database, "PRAGMA journal_mode = WAL", "enabling WAL mode");
|
||||||
|
require_pragma_text(database, "PRAGMA journal_mode", "wal", "enabling WAL mode");
|
||||||
|
execute(database, "PRAGMA synchronous = FULL", "setting full synchronization");
|
||||||
|
require_pragma_integer(
|
||||||
|
database, "PRAGMA synchronous", 2, "setting full synchronization");
|
||||||
|
} catch (...) {
|
||||||
|
sqlite3_close(database);
|
||||||
|
database = nullptr;
|
||||||
|
if (created) {
|
||||||
|
std::error_code ignored;
|
||||||
|
std::filesystem::remove(path, ignored);
|
||||||
|
std::filesystem::remove(path.string() + "-wal", ignored);
|
||||||
|
std::filesystem::remove(path.string() + "-shm", ignored);
|
||||||
|
}
|
||||||
|
throw;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
~Impl() {
|
||||||
|
if (database != nullptr) {
|
||||||
|
sqlite3_close(database);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
std::filesystem::path path;
|
||||||
|
sqlite3* database{nullptr};
|
||||||
|
mutable std::mutex mutex;
|
||||||
|
};
|
||||||
|
|
||||||
|
SyncCache::SyncCache(std::filesystem::path path) : impl_(std::make_unique<Impl>(std::move(path))) {}
|
||||||
|
|
||||||
|
SyncCache::~SyncCache() = default;
|
||||||
|
SyncCache::SyncCache(SyncCache&&) noexcept = default;
|
||||||
|
SyncCache& SyncCache::operator=(SyncCache&&) noexcept = default;
|
||||||
|
|
||||||
|
const std::filesystem::path& SyncCache::path() const noexcept { return impl_->path; }
|
||||||
|
|
||||||
|
void SyncCache::upsert_account(const CachedAccount& account) {
|
||||||
|
validate_nonempty(account.id, "account id");
|
||||||
|
validate_nonempty(account.provider, "account provider");
|
||||||
|
validate_nonempty(account.remote_subject, "account remote subject");
|
||||||
|
|
||||||
|
std::scoped_lock lock(impl_->mutex);
|
||||||
|
Transaction transaction(impl_->database);
|
||||||
|
ensure_account_identity(impl_->database, account);
|
||||||
|
Statement statement(impl_->database, R"sql(
|
||||||
|
INSERT INTO accounts(id, provider, remote_subject, display_name) VALUES(?, ?, ?, ?)
|
||||||
|
ON CONFLICT(id) DO UPDATE SET
|
||||||
|
display_name = excluded.display_name
|
||||||
|
)sql", "upserting account");
|
||||||
|
statement.bind_text(1, account.id);
|
||||||
|
statement.bind_text(2, account.provider);
|
||||||
|
statement.bind_text(3, account.remote_subject);
|
||||||
|
statement.bind_text(4, account.display_name);
|
||||||
|
statement.run();
|
||||||
|
transaction.commit();
|
||||||
|
}
|
||||||
|
|
||||||
|
void SyncCache::replace_calendars_after_complete_listing(
|
||||||
|
std::string account_id, std::span<const CachedCalendar> calendars) {
|
||||||
|
validate_nonempty(account_id, "account id");
|
||||||
|
std::unordered_set<std::string> ids;
|
||||||
|
std::unordered_set<std::string> remote_ids;
|
||||||
|
for (const CachedCalendar& calendar : calendars) {
|
||||||
|
validate_nonempty(calendar.id, "calendar id");
|
||||||
|
validate_nonempty(calendar.remote_id, "calendar remote id");
|
||||||
|
validate_nonempty(calendar.raw_payload, "calendar raw payload");
|
||||||
|
if (calendar.account_id != account_id) {
|
||||||
|
throw std::invalid_argument("calendar belongs to a different account");
|
||||||
|
}
|
||||||
|
if (!calendar.active) {
|
||||||
|
throw std::invalid_argument("a completed calendar listing must contain active calendars");
|
||||||
|
}
|
||||||
|
if (!ids.insert(calendar.id).second || !remote_ids.insert(calendar.remote_id).second) {
|
||||||
|
throw std::invalid_argument("completed calendar listing contains duplicate identities");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
std::scoped_lock lock(impl_->mutex);
|
||||||
|
Transaction transaction(impl_->database);
|
||||||
|
require_account(impl_->database, account_id);
|
||||||
|
for (const CachedCalendar& calendar : calendars) {
|
||||||
|
ensure_calendar_identity(impl_->database, calendar);
|
||||||
|
}
|
||||||
|
{
|
||||||
|
Statement deactivate(impl_->database,
|
||||||
|
"UPDATE calendars SET active = 0 WHERE account_id = ?", "deactivating calendars");
|
||||||
|
deactivate.bind_text(1, account_id);
|
||||||
|
deactivate.run();
|
||||||
|
}
|
||||||
|
for (const CachedCalendar& calendar : calendars) {
|
||||||
|
Statement upsert(impl_->database, R"sql(
|
||||||
|
INSERT INTO calendars(
|
||||||
|
id, account_id, remote_id, name, color, read_only, active, raw_payload)
|
||||||
|
VALUES(?, ?, ?, ?, ?, ?, 1, ?)
|
||||||
|
ON CONFLICT(id) DO UPDATE SET
|
||||||
|
name = excluded.name,
|
||||||
|
color = excluded.color,
|
||||||
|
read_only = excluded.read_only,
|
||||||
|
active = 1,
|
||||||
|
raw_payload = excluded.raw_payload
|
||||||
|
)sql", "upserting calendar");
|
||||||
|
upsert.bind_text(1, calendar.id);
|
||||||
|
upsert.bind_text(2, calendar.account_id);
|
||||||
|
upsert.bind_text(3, calendar.remote_id);
|
||||||
|
upsert.bind_text(4, calendar.name);
|
||||||
|
upsert.bind_text(5, calendar.color);
|
||||||
|
upsert.bind_bool(6, calendar.read_only);
|
||||||
|
upsert.bind_text(7, calendar.raw_payload);
|
||||||
|
upsert.run();
|
||||||
|
}
|
||||||
|
transaction.commit();
|
||||||
|
}
|
||||||
|
|
||||||
|
void SyncCache::apply_pull_page(const PullPage& page) {
|
||||||
|
const SyncCheckpoint& checkpoint = page.checkpoint;
|
||||||
|
validate_nonempty(checkpoint.calendar_id, "checkpoint calendar id");
|
||||||
|
validate_nonempty(checkpoint.cursor, "checkpoint cursor");
|
||||||
|
if (checkpoint.window_start.empty() != checkpoint.window_end.empty()) {
|
||||||
|
throw std::invalid_argument("checkpoint window endpoints must both be present or both be empty");
|
||||||
|
}
|
||||||
|
|
||||||
|
std::unordered_set<std::string> event_ids;
|
||||||
|
std::unordered_set<std::string> event_remote_ids;
|
||||||
|
std::unordered_set<std::string> live_event_ids;
|
||||||
|
for (const CachedEvent& event : page.events) {
|
||||||
|
validate_nonempty(event.id, "event id");
|
||||||
|
validate_nonempty(event.remote_id, "event remote id");
|
||||||
|
if (event.calendar_id != checkpoint.calendar_id) {
|
||||||
|
throw std::invalid_argument("changed event belongs to a different calendar");
|
||||||
|
}
|
||||||
|
if (!event.deleted && event.raw_payload.empty()) {
|
||||||
|
throw std::invalid_argument("a live changed event must retain its raw provider payload");
|
||||||
|
}
|
||||||
|
if (!event_ids.insert(event.id).second
|
||||||
|
|| !event_remote_ids.insert(event.remote_id).second) {
|
||||||
|
throw std::invalid_argument("pull page contains duplicate event identities");
|
||||||
|
}
|
||||||
|
if (!event.deleted) {
|
||||||
|
live_event_ids.insert(event.id);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
std::unordered_set<std::string> instance_ids;
|
||||||
|
for (const CachedEventInstance& instance : page.instances) {
|
||||||
|
validate_nonempty(instance.id, "event instance id");
|
||||||
|
validate_nonempty(instance.event_id, "event instance event id");
|
||||||
|
if (instance.start_epoch_microseconds > instance.end_epoch_microseconds
|
||||||
|
|| (instance.all_day
|
||||||
|
&& instance.start_epoch_microseconds == instance.end_epoch_microseconds)) {
|
||||||
|
throw std::invalid_argument("event instance must have a valid half-open interval");
|
||||||
|
}
|
||||||
|
if (!instance_ids.insert(instance.id).second) {
|
||||||
|
throw std::invalid_argument("pull page contains duplicate event instance ids");
|
||||||
|
}
|
||||||
|
if (!live_event_ids.contains(instance.event_id)) {
|
||||||
|
throw std::invalid_argument(
|
||||||
|
"event instance must refer to a changed, nondeleted event in the same page");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
std::scoped_lock lock(impl_->mutex);
|
||||||
|
Transaction transaction(impl_->database);
|
||||||
|
require_active_calendar(impl_->database, checkpoint.calendar_id);
|
||||||
|
for (const CachedEvent& event : page.events) {
|
||||||
|
ensure_event_identity(impl_->database, event);
|
||||||
|
}
|
||||||
|
for (const CachedEvent& event : page.events) {
|
||||||
|
Statement upsert(impl_->database, R"sql(
|
||||||
|
INSERT INTO events(
|
||||||
|
id, calendar_id, remote_id, uid, etag, change_key, raw_payload, deleted)
|
||||||
|
VALUES(?, ?, ?, ?, ?, ?, ?, ?)
|
||||||
|
ON CONFLICT(id) DO UPDATE SET
|
||||||
|
uid = excluded.uid,
|
||||||
|
etag = excluded.etag,
|
||||||
|
change_key = excluded.change_key,
|
||||||
|
raw_payload = CASE
|
||||||
|
WHEN excluded.deleted = 1 AND length(excluded.raw_payload) = 0 THEN events.raw_payload
|
||||||
|
ELSE excluded.raw_payload
|
||||||
|
END,
|
||||||
|
deleted = excluded.deleted
|
||||||
|
)sql", "upserting changed event");
|
||||||
|
upsert.bind_text(1, event.id);
|
||||||
|
upsert.bind_text(2, event.calendar_id);
|
||||||
|
upsert.bind_text(3, event.remote_id);
|
||||||
|
upsert.bind_text(4, event.uid);
|
||||||
|
upsert.bind_text(5, event.etag);
|
||||||
|
upsert.bind_text(6, event.change_key);
|
||||||
|
upsert.bind_text(7, event.raw_payload);
|
||||||
|
upsert.bind_bool(8, event.deleted);
|
||||||
|
upsert.run();
|
||||||
|
}
|
||||||
|
for (const CachedEvent& event : page.events) {
|
||||||
|
Statement remove(impl_->database,
|
||||||
|
"DELETE FROM event_instances WHERE event_id = ?", "replacing event instances");
|
||||||
|
remove.bind_text(1, event.id);
|
||||||
|
remove.run();
|
||||||
|
}
|
||||||
|
for (const CachedEventInstance& instance : page.instances) {
|
||||||
|
Statement insert(impl_->database, R"sql(
|
||||||
|
INSERT INTO event_instances(
|
||||||
|
id, event_id, start_epoch_microseconds, end_epoch_microseconds, all_day,
|
||||||
|
title, location, description, time_zone)
|
||||||
|
VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||||
|
)sql", "inserting event instance");
|
||||||
|
insert.bind_text(1, instance.id);
|
||||||
|
insert.bind_text(2, instance.event_id);
|
||||||
|
insert.bind_int64(3, instance.start_epoch_microseconds);
|
||||||
|
insert.bind_int64(4, instance.end_epoch_microseconds);
|
||||||
|
insert.bind_bool(5, instance.all_day);
|
||||||
|
insert.bind_text(6, instance.title);
|
||||||
|
insert.bind_text(7, instance.location);
|
||||||
|
insert.bind_text(8, instance.description);
|
||||||
|
insert.bind_text(9, instance.time_zone);
|
||||||
|
insert.run();
|
||||||
|
}
|
||||||
|
{
|
||||||
|
Statement upsert(impl_->database, R"sql(
|
||||||
|
INSERT INTO sync_state(calendar_id, window_start, window_end, cursor, complete)
|
||||||
|
VALUES(?, ?, ?, ?, ?)
|
||||||
|
ON CONFLICT(calendar_id, window_start, window_end) DO UPDATE SET
|
||||||
|
cursor = excluded.cursor,
|
||||||
|
complete = excluded.complete
|
||||||
|
)sql", "advancing sync checkpoint");
|
||||||
|
upsert.bind_text(1, checkpoint.calendar_id);
|
||||||
|
upsert.bind_text(2, checkpoint.window_start);
|
||||||
|
upsert.bind_text(3, checkpoint.window_end);
|
||||||
|
upsert.bind_text(4, checkpoint.cursor);
|
||||||
|
upsert.bind_bool(5, checkpoint.complete);
|
||||||
|
upsert.run();
|
||||||
|
}
|
||||||
|
transaction.commit();
|
||||||
|
}
|
||||||
|
|
||||||
|
CacheSnapshot SyncCache::snapshot() const {
|
||||||
|
std::scoped_lock lock(impl_->mutex);
|
||||||
|
CacheSnapshot result;
|
||||||
|
{
|
||||||
|
Statement query(impl_->database,
|
||||||
|
"SELECT id, provider, remote_subject, display_name FROM accounts ORDER BY id",
|
||||||
|
"reading accounts");
|
||||||
|
while (query.next()) {
|
||||||
|
result.accounts.push_back({query.text_column(0), query.text_column(1),
|
||||||
|
query.text_column(2), query.text_column(3)});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
{
|
||||||
|
Statement query(impl_->database, R"sql(
|
||||||
|
SELECT id, account_id, remote_id, name, color, read_only, active, raw_payload
|
||||||
|
FROM calendars ORDER BY id
|
||||||
|
)sql", "reading calendars");
|
||||||
|
while (query.next()) {
|
||||||
|
result.calendars.push_back({query.text_column(0), query.text_column(1),
|
||||||
|
query.text_column(2), query.text_column(3), query.text_column(4),
|
||||||
|
query.bool_column(5), query.bool_column(6), query.text_column(7)});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
{
|
||||||
|
Statement query(impl_->database, R"sql(
|
||||||
|
SELECT id, calendar_id, remote_id, uid, etag, change_key, raw_payload, deleted
|
||||||
|
FROM events ORDER BY id
|
||||||
|
)sql", "reading events");
|
||||||
|
while (query.next()) {
|
||||||
|
result.events.push_back({query.text_column(0), query.text_column(1),
|
||||||
|
query.text_column(2), query.text_column(3), query.text_column(4),
|
||||||
|
query.text_column(5), query.text_column(6), query.bool_column(7)});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
{
|
||||||
|
Statement query(impl_->database, R"sql(
|
||||||
|
SELECT id, event_id, start_epoch_microseconds, end_epoch_microseconds,
|
||||||
|
all_day, title, location, description, time_zone
|
||||||
|
FROM event_instances ORDER BY id
|
||||||
|
)sql", "reading event instances");
|
||||||
|
while (query.next()) {
|
||||||
|
result.instances.push_back({query.text_column(0), query.text_column(1),
|
||||||
|
query.int64_column(2), query.int64_column(3), query.bool_column(4),
|
||||||
|
query.text_column(5), query.text_column(6), query.text_column(7),
|
||||||
|
query.text_column(8)});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
{
|
||||||
|
Statement query(impl_->database, R"sql(
|
||||||
|
SELECT calendar_id, window_start, window_end, cursor, complete
|
||||||
|
FROM sync_state ORDER BY calendar_id, window_start, window_end
|
||||||
|
)sql", "reading sync checkpoints");
|
||||||
|
while (query.next()) {
|
||||||
|
result.checkpoints.push_back({query.text_column(0), query.text_column(1),
|
||||||
|
query.text_column(2), query.text_column(3), query.bool_column(4)});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace nocal::storage
|
||||||
820
tests/sync_cache_tests.cpp
Normal file
820
tests/sync_cache_tests.cpp
Normal file
@@ -0,0 +1,820 @@
|
|||||||
|
#include "nocal/storage/sync_cache.hpp"
|
||||||
|
|
||||||
|
#include <sqlite3.h>
|
||||||
|
|
||||||
|
#include <algorithm>
|
||||||
|
#include <atomic>
|
||||||
|
#include <cerrno>
|
||||||
|
#include <exception>
|
||||||
|
#include <filesystem>
|
||||||
|
#include <fstream>
|
||||||
|
#include <functional>
|
||||||
|
#include <iostream>
|
||||||
|
#include <iterator>
|
||||||
|
#include <stdexcept>
|
||||||
|
#include <string>
|
||||||
|
#include <string_view>
|
||||||
|
#include <thread>
|
||||||
|
#include <utility>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
#if !defined(_WIN32)
|
||||||
|
#include <sys/stat.h>
|
||||||
|
#include <unistd.h>
|
||||||
|
#endif
|
||||||
|
|
||||||
|
namespace {
|
||||||
|
|
||||||
|
using nocal::storage::CacheSnapshot;
|
||||||
|
using nocal::storage::CachedAccount;
|
||||||
|
using nocal::storage::CachedCalendar;
|
||||||
|
using nocal::storage::CachedEvent;
|
||||||
|
using nocal::storage::CachedEventInstance;
|
||||||
|
using nocal::storage::PullPage;
|
||||||
|
using nocal::storage::SyncCache;
|
||||||
|
using nocal::storage::SyncCheckpoint;
|
||||||
|
|
||||||
|
void require(bool condition, std::string_view message) {
|
||||||
|
if (!condition) {
|
||||||
|
throw std::runtime_error(std::string{message});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename Function>
|
||||||
|
void require_throws(Function&& function, std::string_view message) {
|
||||||
|
try {
|
||||||
|
function();
|
||||||
|
} catch (const std::exception&) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
throw std::runtime_error(std::string{message});
|
||||||
|
}
|
||||||
|
|
||||||
|
class TempDirectory {
|
||||||
|
public:
|
||||||
|
TempDirectory() {
|
||||||
|
#if !defined(_WIN32)
|
||||||
|
std::string pattern =
|
||||||
|
(std::filesystem::temp_directory_path() / "nocal-sync-cache-tests.XXXXXX").string();
|
||||||
|
std::vector<char> writable(pattern.begin(), pattern.end());
|
||||||
|
writable.push_back('\0');
|
||||||
|
const char* created = ::mkdtemp(writable.data());
|
||||||
|
if (created == nullptr) {
|
||||||
|
throw std::runtime_error("mkdtemp failed");
|
||||||
|
}
|
||||||
|
path_ = created;
|
||||||
|
#else
|
||||||
|
path_ = std::filesystem::temp_directory_path() / "nocal-sync-cache-tests";
|
||||||
|
std::filesystem::remove_all(path_);
|
||||||
|
std::filesystem::create_directories(path_);
|
||||||
|
#endif
|
||||||
|
}
|
||||||
|
|
||||||
|
TempDirectory(const TempDirectory&) = delete;
|
||||||
|
TempDirectory& operator=(const TempDirectory&) = delete;
|
||||||
|
|
||||||
|
~TempDirectory() {
|
||||||
|
std::error_code ignored;
|
||||||
|
std::filesystem::remove_all(path_, ignored);
|
||||||
|
}
|
||||||
|
|
||||||
|
[[nodiscard]] const std::filesystem::path& path() const noexcept { return path_; }
|
||||||
|
|
||||||
|
private:
|
||||||
|
std::filesystem::path path_;
|
||||||
|
};
|
||||||
|
|
||||||
|
class SqliteConnection {
|
||||||
|
public:
|
||||||
|
explicit SqliteConnection(const std::filesystem::path& path) {
|
||||||
|
const int result = sqlite3_open_v2(
|
||||||
|
path.c_str(), &database_, SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE, nullptr);
|
||||||
|
if (result != SQLITE_OK) {
|
||||||
|
const std::string message = database_ == nullptr ? "sqlite open failed"
|
||||||
|
: sqlite3_errmsg(database_);
|
||||||
|
if (database_ != nullptr) {
|
||||||
|
sqlite3_close(database_);
|
||||||
|
database_ = nullptr;
|
||||||
|
}
|
||||||
|
throw std::runtime_error(message);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
SqliteConnection(const SqliteConnection&) = delete;
|
||||||
|
SqliteConnection& operator=(const SqliteConnection&) = delete;
|
||||||
|
|
||||||
|
~SqliteConnection() {
|
||||||
|
if (database_ != nullptr) {
|
||||||
|
sqlite3_close(database_);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void execute(std::string_view sql) {
|
||||||
|
char* error = nullptr;
|
||||||
|
const std::string statement{sql};
|
||||||
|
const int result = sqlite3_exec(database_, statement.c_str(), nullptr, nullptr, &error);
|
||||||
|
if (result != SQLITE_OK) {
|
||||||
|
const std::string message = error == nullptr ? sqlite3_errmsg(database_) : error;
|
||||||
|
sqlite3_free(error);
|
||||||
|
throw std::runtime_error(message);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
[[nodiscard]] std::string scalar_text(std::string_view sql) const {
|
||||||
|
sqlite3_stmt* statement = nullptr;
|
||||||
|
const std::string query{sql};
|
||||||
|
if (sqlite3_prepare_v2(database_, query.c_str(), -1, &statement, nullptr) != SQLITE_OK) {
|
||||||
|
throw std::runtime_error(sqlite3_errmsg(database_));
|
||||||
|
}
|
||||||
|
struct Finalize {
|
||||||
|
sqlite3_stmt* statement;
|
||||||
|
~Finalize() { sqlite3_finalize(statement); }
|
||||||
|
} finalize{statement};
|
||||||
|
if (sqlite3_step(statement) != SQLITE_ROW) {
|
||||||
|
throw std::runtime_error(sqlite3_errmsg(database_));
|
||||||
|
}
|
||||||
|
const unsigned char* value = sqlite3_column_text(statement, 0);
|
||||||
|
return value == nullptr ? std::string{} : reinterpret_cast<const char*>(value);
|
||||||
|
}
|
||||||
|
|
||||||
|
[[nodiscard]] int scalar_int(std::string_view sql) const {
|
||||||
|
sqlite3_stmt* statement = nullptr;
|
||||||
|
const std::string query{sql};
|
||||||
|
if (sqlite3_prepare_v2(database_, query.c_str(), -1, &statement, nullptr) != SQLITE_OK) {
|
||||||
|
throw std::runtime_error(sqlite3_errmsg(database_));
|
||||||
|
}
|
||||||
|
struct Finalize {
|
||||||
|
sqlite3_stmt* statement;
|
||||||
|
~Finalize() { sqlite3_finalize(statement); }
|
||||||
|
} finalize{statement};
|
||||||
|
if (sqlite3_step(statement) != SQLITE_ROW) {
|
||||||
|
throw std::runtime_error(sqlite3_errmsg(database_));
|
||||||
|
}
|
||||||
|
return sqlite3_column_int(statement, 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
[[nodiscard]] std::vector<std::string> first_column(std::string_view sql) const {
|
||||||
|
sqlite3_stmt* statement = nullptr;
|
||||||
|
const std::string query{sql};
|
||||||
|
if (sqlite3_prepare_v2(database_, query.c_str(), -1, &statement, nullptr) != SQLITE_OK) {
|
||||||
|
throw std::runtime_error(sqlite3_errmsg(database_));
|
||||||
|
}
|
||||||
|
struct Finalize {
|
||||||
|
sqlite3_stmt* statement;
|
||||||
|
~Finalize() { sqlite3_finalize(statement); }
|
||||||
|
} finalize{statement};
|
||||||
|
std::vector<std::string> values;
|
||||||
|
int result = SQLITE_ROW;
|
||||||
|
while ((result = sqlite3_step(statement)) == SQLITE_ROW) {
|
||||||
|
const unsigned char* value = sqlite3_column_text(statement, 0);
|
||||||
|
values.emplace_back(value == nullptr ? "" : reinterpret_cast<const char*>(value));
|
||||||
|
}
|
||||||
|
if (result != SQLITE_DONE) {
|
||||||
|
throw std::runtime_error(sqlite3_errmsg(database_));
|
||||||
|
}
|
||||||
|
return values;
|
||||||
|
}
|
||||||
|
|
||||||
|
private:
|
||||||
|
sqlite3* database_{nullptr};
|
||||||
|
};
|
||||||
|
|
||||||
|
CachedAccount account(std::string id = "account-a") {
|
||||||
|
return CachedAccount{std::move(id), "microsoft-graph", "subject-a", "Work account"};
|
||||||
|
}
|
||||||
|
|
||||||
|
CachedCalendar calendar(
|
||||||
|
std::string id = "calendar-a", std::string account_id = "account-a") {
|
||||||
|
return CachedCalendar{std::move(id), std::move(account_id), "remote-calendar-a", "Calendar A",
|
||||||
|
"blue", false, true, R"({"calendar":"a"})"};
|
||||||
|
}
|
||||||
|
|
||||||
|
CachedEvent event(std::string id = "event-a", std::string calendar_id = "calendar-a") {
|
||||||
|
return CachedEvent{std::move(id), std::move(calendar_id), "remote-event-a", "uid-a", "etag-a",
|
||||||
|
"change-a", R"({"event":"a"})", false};
|
||||||
|
}
|
||||||
|
|
||||||
|
CachedEventInstance instance(
|
||||||
|
std::string id = "instance-a", std::string event_id = "event-a") {
|
||||||
|
return CachedEventInstance{std::move(id), std::move(event_id), 1'000'000, 2'000'000, false,
|
||||||
|
"Event A", "Room", "Description", "Europe/London"};
|
||||||
|
}
|
||||||
|
|
||||||
|
SyncCheckpoint checkpoint(std::string cursor = "cursor-a", std::string calendar_id = "calendar-a",
|
||||||
|
std::string window_start = "opaque-start", std::string window_end = "opaque-end") {
|
||||||
|
return SyncCheckpoint{std::move(calendar_id), std::move(window_start), std::move(window_end),
|
||||||
|
std::move(cursor), false};
|
||||||
|
}
|
||||||
|
|
||||||
|
PullPage page(CachedEvent changed_event = event(), CachedEventInstance changed_instance = instance(),
|
||||||
|
SyncCheckpoint next = checkpoint()) {
|
||||||
|
return PullPage{{std::move(changed_event)}, {std::move(changed_instance)}, std::move(next)};
|
||||||
|
}
|
||||||
|
|
||||||
|
void seed_calendar(SyncCache& cache, const CachedAccount& cached_account = account(),
|
||||||
|
const CachedCalendar& cached_calendar = calendar()) {
|
||||||
|
cache.upsert_account(cached_account);
|
||||||
|
const std::vector calendars{cached_calendar};
|
||||||
|
cache.replace_calendars_after_complete_listing(cached_account.id, calendars);
|
||||||
|
}
|
||||||
|
|
||||||
|
[[nodiscard]] bool contains(const std::vector<std::string>& values, std::string_view value) {
|
||||||
|
return std::ranges::find(values, value) != values.end();
|
||||||
|
}
|
||||||
|
|
||||||
|
#if !defined(_WIN32)
|
||||||
|
void require_private_file(const std::filesystem::path& path, std::string_view description) {
|
||||||
|
struct stat metadata {};
|
||||||
|
require(::stat(path.c_str(), &metadata) == 0,
|
||||||
|
std::string{"could not stat "}.append(description));
|
||||||
|
require(S_ISREG(metadata.st_mode), std::string{description}.append(" is not a regular file"));
|
||||||
|
require((metadata.st_mode & (S_IRWXG | S_IRWXO)) == 0,
|
||||||
|
std::string{description}.append(" grants group or other permissions"));
|
||||||
|
require((metadata.st_mode & S_IRUSR) != 0 && (metadata.st_mode & S_IWUSR) != 0,
|
||||||
|
std::string{description}.append(" is not owner-readable and writable"));
|
||||||
|
}
|
||||||
|
#endif
|
||||||
|
|
||||||
|
void test_fresh_schema_permissions_and_secrets(const std::filesystem::path& root) {
|
||||||
|
const auto path = root / "fresh.db";
|
||||||
|
SyncCache cache{path};
|
||||||
|
require(cache.path() == path, "cache did not retain its path");
|
||||||
|
require(std::filesystem::is_regular_file(path), "fresh cache database was not created");
|
||||||
|
|
||||||
|
SqliteConnection inspection{path};
|
||||||
|
require(inspection.scalar_text("PRAGMA journal_mode") == "wal", "cache is not in WAL mode");
|
||||||
|
require(inspection.scalar_int("PRAGMA user_version") == SyncCache::current_schema_version,
|
||||||
|
"fresh cache schema version is incorrect");
|
||||||
|
|
||||||
|
const auto tables = inspection.first_column(
|
||||||
|
"SELECT name FROM sqlite_master WHERE type='table' ORDER BY name");
|
||||||
|
for (const std::string_view required : {"accounts", "calendars", "events",
|
||||||
|
"event_instances", "sync_state", "outbox", "conflicts"}) {
|
||||||
|
require(contains(tables, required), std::string{"missing cache table: "}.append(required));
|
||||||
|
}
|
||||||
|
|
||||||
|
for (const std::string_view child : {"calendars", "events", "event_instances", "sync_state"}) {
|
||||||
|
require(inspection.scalar_int(
|
||||||
|
std::string{"SELECT COUNT(*) FROM pragma_foreign_key_list('"}
|
||||||
|
.append(child)
|
||||||
|
.append("')"))
|
||||||
|
> 0,
|
||||||
|
std::string{"missing foreign key on "}.append(child));
|
||||||
|
}
|
||||||
|
|
||||||
|
const auto columns = inspection.first_column(
|
||||||
|
"SELECT lower(m.name || '.' || p.name) FROM sqlite_master AS m "
|
||||||
|
"JOIN pragma_table_info(m.name) AS p WHERE m.type='table'");
|
||||||
|
for (const auto& column : columns) {
|
||||||
|
require(column.find("token") == std::string::npos
|
||||||
|
&& column.find("secret") == std::string::npos
|
||||||
|
&& column.find("credential") == std::string::npos,
|
||||||
|
"cache schema contains a token, secret, or credential column");
|
||||||
|
}
|
||||||
|
|
||||||
|
#if !defined(_WIN32)
|
||||||
|
require_private_file(path, "cache database");
|
||||||
|
for (const auto suffix : {"-wal", "-shm"}) {
|
||||||
|
const std::filesystem::path sidecar = path.string() + suffix;
|
||||||
|
require(std::filesystem::exists(sidecar), "WAL sidecar was not created");
|
||||||
|
require_private_file(sidecar, "cache sidecar");
|
||||||
|
}
|
||||||
|
#endif
|
||||||
|
}
|
||||||
|
|
||||||
|
void test_existing_permissions_are_tightened(const std::filesystem::path& root) {
|
||||||
|
#if !defined(_WIN32)
|
||||||
|
const auto path = root / "permissive.db";
|
||||||
|
{
|
||||||
|
SqliteConnection database{path};
|
||||||
|
database.execute("PRAGMA user_version=0");
|
||||||
|
}
|
||||||
|
require(::chmod(path.c_str(), 0666) == 0, "could not make fixture permissive");
|
||||||
|
SyncCache cache{path};
|
||||||
|
require_private_file(path, "existing cache database");
|
||||||
|
#else
|
||||||
|
static_cast<void>(root);
|
||||||
|
#endif
|
||||||
|
}
|
||||||
|
|
||||||
|
void test_account_calendars_and_deterministic_snapshot(const std::filesystem::path& root) {
|
||||||
|
SyncCache cache{root / "listing.db"};
|
||||||
|
auto account_b = account("account-b");
|
||||||
|
account_b.remote_subject = "subject-b";
|
||||||
|
auto account_a = account();
|
||||||
|
cache.upsert_account(account_b);
|
||||||
|
cache.upsert_account(account_a);
|
||||||
|
|
||||||
|
auto calendar_b = calendar("calendar-b");
|
||||||
|
calendar_b.remote_id = "remote-calendar-b";
|
||||||
|
calendar_b.name = "Calendar B";
|
||||||
|
auto calendar_a = calendar();
|
||||||
|
cache.replace_calendars_after_complete_listing(
|
||||||
|
account_a.id, std::vector<CachedCalendar>{calendar_b, calendar_a});
|
||||||
|
|
||||||
|
auto first = cache.snapshot();
|
||||||
|
require(first.accounts.size() == 2 && first.accounts[0].id == "account-a"
|
||||||
|
&& first.accounts[1].id == "account-b",
|
||||||
|
"accounts are not returned deterministically by ID");
|
||||||
|
require(first.calendars.size() == 2 && first.calendars[0].id == "calendar-a"
|
||||||
|
&& first.calendars[1].id == "calendar-b",
|
||||||
|
"calendars are not returned deterministically by ID");
|
||||||
|
|
||||||
|
calendar_a.name = "Renamed";
|
||||||
|
cache.replace_calendars_after_complete_listing(
|
||||||
|
account_a.id, std::vector<CachedCalendar>{calendar_a});
|
||||||
|
const auto second = cache.snapshot();
|
||||||
|
require(second.calendars.size() == 2, "omitted calendar was deleted");
|
||||||
|
require(second.calendars[0].active && second.calendars[0].name == "Renamed",
|
||||||
|
"listed calendar was not updated and kept active");
|
||||||
|
require(!second.calendars[1].active, "omitted calendar was not deactivated");
|
||||||
|
require(second == cache.snapshot(), "unchanged snapshots are not deterministic");
|
||||||
|
}
|
||||||
|
|
||||||
|
void test_pull_pages_replacement_tombstone_and_windows(const std::filesystem::path& root) {
|
||||||
|
SyncCache cache{root / "pull.db"};
|
||||||
|
seed_calendar(cache);
|
||||||
|
|
||||||
|
auto event_b = event("event-b");
|
||||||
|
event_b.remote_id = "remote-event-b";
|
||||||
|
event_b.uid = "uid-b";
|
||||||
|
event_b.raw_payload = R"({"event":"b"})";
|
||||||
|
auto instance_b = instance("instance-b", "event-b");
|
||||||
|
instance_b.title = "Event B";
|
||||||
|
PullPage first{{event_b, event()}, {instance_b, instance()}, checkpoint("cursor-1")};
|
||||||
|
cache.apply_pull_page(first);
|
||||||
|
const auto initial = cache.snapshot();
|
||||||
|
require(initial.events.size() == 2 && initial.instances.size() == 2,
|
||||||
|
"pull page did not atomically add events and instances");
|
||||||
|
require(initial.events[0].id == "event-a" && initial.events[1].id == "event-b"
|
||||||
|
&& initial.instances[0].id == "instance-a"
|
||||||
|
&& initial.instances[1].id == "instance-b",
|
||||||
|
"events and instances are not returned deterministically by ID");
|
||||||
|
require(initial.checkpoints.size() == 1 && initial.checkpoints[0].cursor == "cursor-1",
|
||||||
|
"pull page did not advance its cursor");
|
||||||
|
|
||||||
|
cache.apply_pull_page(first);
|
||||||
|
require(cache.snapshot() == initial, "repeated pull page was not idempotent");
|
||||||
|
|
||||||
|
auto changed = event();
|
||||||
|
changed.etag = "etag-a-2";
|
||||||
|
changed.raw_payload = R"({"event":"a2"})";
|
||||||
|
auto replacement = instance("instance-a-2");
|
||||||
|
replacement.title = "Changed A";
|
||||||
|
cache.apply_pull_page(page(changed, replacement, checkpoint("cursor-2")));
|
||||||
|
auto replaced = cache.snapshot();
|
||||||
|
require(replaced.events.size() == 2 && replaced.instances.size() == 2,
|
||||||
|
"changed event replacement affected an untouched event");
|
||||||
|
require(contains(std::vector<std::string>{replaced.instances[0].id, replaced.instances[1].id},
|
||||||
|
"instance-a-2")
|
||||||
|
&& contains(std::vector<std::string>{
|
||||||
|
replaced.instances[0].id, replaced.instances[1].id},
|
||||||
|
"instance-b"),
|
||||||
|
"changed event instances were not replaced independently");
|
||||||
|
|
||||||
|
auto tombstone = event();
|
||||||
|
tombstone.deleted = true;
|
||||||
|
tombstone.raw_payload.clear();
|
||||||
|
cache.apply_pull_page(PullPage{{tombstone}, {}, checkpoint("cursor-3")});
|
||||||
|
const auto deleted = cache.snapshot();
|
||||||
|
const auto deleted_event = std::ranges::find_if(
|
||||||
|
deleted.events, [](const CachedEvent& value) { return value.id == "event-a"; });
|
||||||
|
require(deleted_event != deleted.events.end() && deleted_event->deleted,
|
||||||
|
"tombstone did not retain a deleted event record");
|
||||||
|
require(deleted_event->raw_payload == changed.raw_payload,
|
||||||
|
"empty tombstone did not retain the prior raw payload");
|
||||||
|
require(std::ranges::none_of(deleted.instances,
|
||||||
|
[](const CachedEventInstance& value) { return value.event_id == "event-a"; }),
|
||||||
|
"tombstone retained event instances");
|
||||||
|
|
||||||
|
cache.apply_pull_page(PullPage{{}, {}, checkpoint("other-cursor", "calendar-a",
|
||||||
|
"second-window-start", "second-window-end")});
|
||||||
|
const auto windows = cache.snapshot();
|
||||||
|
require(windows.checkpoints.size() == 2,
|
||||||
|
"independent checkpoint windows for one calendar were overwritten");
|
||||||
|
require(windows.checkpoints[0].window_start == "opaque-start"
|
||||||
|
&& windows.checkpoints[1].window_start == "second-window-start",
|
||||||
|
"checkpoints are not returned in deterministic scope order");
|
||||||
|
}
|
||||||
|
|
||||||
|
void test_multiple_connections_serialize(const std::filesystem::path& root) {
|
||||||
|
const auto path = root / "connections.db";
|
||||||
|
SyncCache first{path};
|
||||||
|
SyncCache second{path};
|
||||||
|
|
||||||
|
std::exception_ptr first_error;
|
||||||
|
std::exception_ptr second_error;
|
||||||
|
std::atomic<bool> begin{false};
|
||||||
|
auto write = [&begin](SyncCache& cache, CachedAccount value, std::exception_ptr& error) {
|
||||||
|
while (!begin.load(std::memory_order_acquire)) {
|
||||||
|
std::this_thread::yield();
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
cache.upsert_account(value);
|
||||||
|
} catch (...) {
|
||||||
|
error = std::current_exception();
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
auto account_b = account("account-b");
|
||||||
|
account_b.remote_subject = "subject-b";
|
||||||
|
std::thread one{write, std::ref(first), account(), std::ref(first_error)};
|
||||||
|
std::thread two{write, std::ref(second), account_b, std::ref(second_error)};
|
||||||
|
begin.store(true, std::memory_order_release);
|
||||||
|
one.join();
|
||||||
|
two.join();
|
||||||
|
|
||||||
|
require(first_error == nullptr && second_error == nullptr,
|
||||||
|
"ordinary concurrent writes leaked SQLITE_BUSY or another error");
|
||||||
|
const auto snapshot = first.snapshot();
|
||||||
|
require(snapshot.accounts.size() == 2, "serialized connections lost a write");
|
||||||
|
}
|
||||||
|
|
||||||
|
void test_input_validation_and_rollback(const std::filesystem::path& root) {
|
||||||
|
SyncCache cache{root / "validation.db"};
|
||||||
|
seed_calendar(cache);
|
||||||
|
auto account_b = account("account-b");
|
||||||
|
account_b.remote_subject = "subject-b";
|
||||||
|
cache.upsert_account(account_b);
|
||||||
|
auto calendar_b = calendar("calendar-b", "account-b");
|
||||||
|
calendar_b.remote_id = "remote-calendar-b";
|
||||||
|
calendar_b.raw_payload = R"({"calendar":"b"})";
|
||||||
|
cache.replace_calendars_after_complete_listing("account-b", std::vector{calendar_b});
|
||||||
|
cache.apply_pull_page(page());
|
||||||
|
const CacheSnapshot baseline = cache.snapshot();
|
||||||
|
|
||||||
|
auto expect_unchanged = [&](auto&& operation, std::string_view message) {
|
||||||
|
require_throws(std::forward<decltype(operation)>(operation), message);
|
||||||
|
require(cache.snapshot() == baseline, "failed operation changed the cache snapshot");
|
||||||
|
};
|
||||||
|
|
||||||
|
auto empty_account = account();
|
||||||
|
empty_account.id.clear();
|
||||||
|
expect_unchanged([&] { cache.upsert_account(empty_account); }, "empty account ID accepted");
|
||||||
|
|
||||||
|
auto reassigned = account();
|
||||||
|
reassigned.remote_subject = "different-subject";
|
||||||
|
expect_unchanged([&] { cache.upsert_account(reassigned); },
|
||||||
|
"account local ID identity reassignment accepted");
|
||||||
|
|
||||||
|
auto provider_reassigned = account();
|
||||||
|
provider_reassigned.provider = "different-provider";
|
||||||
|
expect_unchanged([&] { cache.upsert_account(provider_reassigned); },
|
||||||
|
"account provider identity reassignment accepted");
|
||||||
|
|
||||||
|
auto duplicate_calendar = calendar();
|
||||||
|
expect_unchanged(
|
||||||
|
[&] {
|
||||||
|
cache.replace_calendars_after_complete_listing(
|
||||||
|
"account-a", std::vector<CachedCalendar>{duplicate_calendar, duplicate_calendar});
|
||||||
|
},
|
||||||
|
"duplicate calendar ID accepted");
|
||||||
|
|
||||||
|
auto duplicate_remote_calendar = calendar("calendar-new");
|
||||||
|
expect_unchanged(
|
||||||
|
[&] {
|
||||||
|
cache.replace_calendars_after_complete_listing("account-a",
|
||||||
|
std::vector<CachedCalendar>{duplicate_calendar, duplicate_remote_calendar});
|
||||||
|
},
|
||||||
|
"duplicate calendar remote ID accepted");
|
||||||
|
|
||||||
|
auto empty_calendar = calendar();
|
||||||
|
empty_calendar.id.clear();
|
||||||
|
expect_unchanged(
|
||||||
|
[&] {
|
||||||
|
cache.replace_calendars_after_complete_listing(
|
||||||
|
"account-a", std::vector<CachedCalendar>{empty_calendar});
|
||||||
|
},
|
||||||
|
"empty calendar ID accepted");
|
||||||
|
|
||||||
|
auto wrong_account = calendar();
|
||||||
|
wrong_account.account_id = "account-b";
|
||||||
|
expect_unchanged(
|
||||||
|
[&] {
|
||||||
|
cache.replace_calendars_after_complete_listing(
|
||||||
|
"account-a", std::vector<CachedCalendar>{wrong_account});
|
||||||
|
},
|
||||||
|
"cross-account calendar accepted");
|
||||||
|
|
||||||
|
auto reused_calendar_id = calendar("calendar-a", "account-b");
|
||||||
|
reused_calendar_id.remote_id = "different-remote-calendar";
|
||||||
|
expect_unchanged(
|
||||||
|
[&] {
|
||||||
|
cache.replace_calendars_after_complete_listing(
|
||||||
|
"account-b", std::vector<CachedCalendar>{reused_calendar_id});
|
||||||
|
},
|
||||||
|
"calendar local ID was reassigned across accounts");
|
||||||
|
|
||||||
|
auto inactive_input = calendar();
|
||||||
|
inactive_input.active = false;
|
||||||
|
expect_unchanged(
|
||||||
|
[&] {
|
||||||
|
cache.replace_calendars_after_complete_listing(
|
||||||
|
"account-a", std::vector<CachedCalendar>{inactive_input});
|
||||||
|
},
|
||||||
|
"inactive completed-listing input accepted");
|
||||||
|
|
||||||
|
auto missing_raw_calendar = calendar();
|
||||||
|
missing_raw_calendar.raw_payload.clear();
|
||||||
|
expect_unchanged(
|
||||||
|
[&] {
|
||||||
|
cache.replace_calendars_after_complete_listing(
|
||||||
|
"account-a", std::vector<CachedCalendar>{missing_raw_calendar});
|
||||||
|
},
|
||||||
|
"calendar without raw payload accepted");
|
||||||
|
|
||||||
|
auto duplicate_event = event();
|
||||||
|
expect_unchanged(
|
||||||
|
[&] {
|
||||||
|
cache.apply_pull_page(PullPage{{duplicate_event, duplicate_event}, {},
|
||||||
|
checkpoint("duplicate-cursor")});
|
||||||
|
},
|
||||||
|
"duplicate event ID accepted");
|
||||||
|
|
||||||
|
auto duplicate_remote_event = event("event-new");
|
||||||
|
expect_unchanged(
|
||||||
|
[&] {
|
||||||
|
cache.apply_pull_page(PullPage{{duplicate_event, duplicate_remote_event}, {},
|
||||||
|
checkpoint("duplicate-remote-event")});
|
||||||
|
},
|
||||||
|
"duplicate event remote ID accepted");
|
||||||
|
|
||||||
|
auto empty_event = event();
|
||||||
|
empty_event.id.clear();
|
||||||
|
expect_unchanged(
|
||||||
|
[&] { cache.apply_pull_page(page(empty_event, instance(), checkpoint("empty-event"))); },
|
||||||
|
"empty event ID accepted");
|
||||||
|
|
||||||
|
auto cross_calendar_event = event();
|
||||||
|
cross_calendar_event.calendar_id = "calendar-b";
|
||||||
|
expect_unchanged(
|
||||||
|
[&] {
|
||||||
|
cache.apply_pull_page(
|
||||||
|
PullPage{{cross_calendar_event}, {}, checkpoint("cross-calendar")});
|
||||||
|
},
|
||||||
|
"cross-calendar event accepted");
|
||||||
|
|
||||||
|
auto reassigned_event = event("event-a", "calendar-b");
|
||||||
|
reassigned_event.remote_id = "different-remote-event";
|
||||||
|
expect_unchanged(
|
||||||
|
[&] {
|
||||||
|
cache.apply_pull_page(PullPage{{reassigned_event}, {},
|
||||||
|
checkpoint("reassigned-event", "calendar-b")});
|
||||||
|
},
|
||||||
|
"event local ID was reassigned across calendars");
|
||||||
|
|
||||||
|
auto missing_remote = event();
|
||||||
|
missing_remote.remote_id.clear();
|
||||||
|
expect_unchanged(
|
||||||
|
[&] { cache.apply_pull_page(page(missing_remote, instance(), checkpoint("remote"))); },
|
||||||
|
"event without remote ID accepted");
|
||||||
|
|
||||||
|
auto missing_raw_event = event();
|
||||||
|
missing_raw_event.raw_payload.clear();
|
||||||
|
expect_unchanged(
|
||||||
|
[&] { cache.apply_pull_page(page(missing_raw_event, instance(), checkpoint("raw"))); },
|
||||||
|
"nondeleted event without raw payload accepted");
|
||||||
|
|
||||||
|
auto duplicate_instance = instance();
|
||||||
|
expect_unchanged(
|
||||||
|
[&] {
|
||||||
|
cache.apply_pull_page(PullPage{{event()}, {duplicate_instance, duplicate_instance},
|
||||||
|
checkpoint("duplicate-instance")});
|
||||||
|
},
|
||||||
|
"duplicate instance ID accepted");
|
||||||
|
|
||||||
|
auto empty_instance = instance();
|
||||||
|
empty_instance.id.clear();
|
||||||
|
expect_unchanged(
|
||||||
|
[&] { cache.apply_pull_page(page(event(), empty_instance, checkpoint("empty-instance"))); },
|
||||||
|
"empty instance ID accepted");
|
||||||
|
|
||||||
|
auto orphan = instance();
|
||||||
|
orphan.event_id = "event-not-in-page";
|
||||||
|
expect_unchanged(
|
||||||
|
[&] { cache.apply_pull_page(page(event(), orphan, checkpoint("orphan"))); },
|
||||||
|
"orphan instance accepted");
|
||||||
|
|
||||||
|
auto backwards = instance();
|
||||||
|
backwards.end_epoch_microseconds = backwards.start_epoch_microseconds - 1;
|
||||||
|
expect_unchanged(
|
||||||
|
[&] { cache.apply_pull_page(page(event(), backwards, checkpoint("backwards"))); },
|
||||||
|
"backwards instance interval accepted");
|
||||||
|
|
||||||
|
auto empty_all_day = instance();
|
||||||
|
empty_all_day.all_day = true;
|
||||||
|
empty_all_day.end_epoch_microseconds = empty_all_day.start_epoch_microseconds;
|
||||||
|
expect_unchanged(
|
||||||
|
[&] { cache.apply_pull_page(page(event(), empty_all_day, checkpoint("all-day"))); },
|
||||||
|
"empty all-day interval accepted");
|
||||||
|
|
||||||
|
auto deleted_with_instance = event();
|
||||||
|
deleted_with_instance.deleted = true;
|
||||||
|
deleted_with_instance.raw_payload.clear();
|
||||||
|
expect_unchanged(
|
||||||
|
[&] {
|
||||||
|
cache.apply_pull_page(
|
||||||
|
page(deleted_with_instance, instance(), checkpoint("deleted-instance")));
|
||||||
|
},
|
||||||
|
"instance attached to tombstoned event accepted");
|
||||||
|
|
||||||
|
auto negative = event();
|
||||||
|
negative.id = "event-negative";
|
||||||
|
negative.remote_id = "remote-negative";
|
||||||
|
negative.uid = "uid-negative";
|
||||||
|
auto before_epoch = instance("instance-negative", "event-negative");
|
||||||
|
before_epoch.start_epoch_microseconds = -2'000'000;
|
||||||
|
before_epoch.end_epoch_microseconds = -1'000'000;
|
||||||
|
cache.apply_pull_page(page(negative, before_epoch, checkpoint("negative")));
|
||||||
|
require(std::ranges::any_of(cache.snapshot().instances,
|
||||||
|
[](const CachedEventInstance& value) {
|
||||||
|
return value.id == "instance-negative"
|
||||||
|
&& value.start_epoch_microseconds < 0;
|
||||||
|
}),
|
||||||
|
"valid pre-1970 instance was rejected");
|
||||||
|
}
|
||||||
|
|
||||||
|
void test_sql_failure_rolls_back_snapshot_and_cursor(const std::filesystem::path& root) {
|
||||||
|
SyncCache cache{root / "sql-rollback.db"};
|
||||||
|
seed_calendar(cache);
|
||||||
|
auto event_b = event("event-b");
|
||||||
|
event_b.remote_id = "remote-event-b";
|
||||||
|
event_b.uid = "uid-b";
|
||||||
|
event_b.raw_payload = R"({"event":"b"})";
|
||||||
|
auto instance_b = instance("instance-b", "event-b");
|
||||||
|
cache.apply_pull_page(
|
||||||
|
PullPage{{event(), event_b}, {instance(), instance_b}, checkpoint("baseline-cursor")});
|
||||||
|
const auto baseline = cache.snapshot();
|
||||||
|
|
||||||
|
auto changed = event();
|
||||||
|
changed.etag = "must-roll-back";
|
||||||
|
changed.raw_payload = R"({"event":"must-roll-back"})";
|
||||||
|
auto colliding_instance = instance("instance-b", "event-a");
|
||||||
|
require_throws(
|
||||||
|
[&] {
|
||||||
|
cache.apply_pull_page(
|
||||||
|
page(changed, colliding_instance, checkpoint("must-not-advance")));
|
||||||
|
},
|
||||||
|
"cross-event instance ID collision was accepted");
|
||||||
|
require(cache.snapshot() == baseline,
|
||||||
|
"SQL failure did not roll back event, instance, and cursor state together");
|
||||||
|
}
|
||||||
|
|
||||||
|
void test_checkpoint_calendar_and_shape_validation(const std::filesystem::path& root) {
|
||||||
|
const auto path = root / "checkpoint-validation.db";
|
||||||
|
SyncCache cache{path};
|
||||||
|
seed_calendar(cache);
|
||||||
|
|
||||||
|
auto expect_empty = [&](auto&& operation, std::string_view message) {
|
||||||
|
require_throws(std::forward<decltype(operation)>(operation), message);
|
||||||
|
require(cache.snapshot().events.empty() && cache.snapshot().checkpoints.empty(),
|
||||||
|
"invalid pull partially changed event or checkpoint state");
|
||||||
|
};
|
||||||
|
|
||||||
|
expect_empty(
|
||||||
|
[&] { cache.apply_pull_page(page(event(), instance(), checkpoint("", "calendar-a"))); },
|
||||||
|
"empty checkpoint cursor accepted");
|
||||||
|
expect_empty(
|
||||||
|
[&] {
|
||||||
|
cache.apply_pull_page(page(event(), instance(),
|
||||||
|
checkpoint("cursor", "calendar-a", "only-start", "")));
|
||||||
|
},
|
||||||
|
"half-specified checkpoint window accepted");
|
||||||
|
expect_empty(
|
||||||
|
[&] {
|
||||||
|
cache.apply_pull_page(page(event(), instance(),
|
||||||
|
checkpoint("cursor", "calendar-a", "", "only-end")));
|
||||||
|
},
|
||||||
|
"half-specified checkpoint window accepted");
|
||||||
|
expect_empty(
|
||||||
|
[&] {
|
||||||
|
cache.apply_pull_page(
|
||||||
|
page(event(), instance(), checkpoint("cursor", "missing-calendar")));
|
||||||
|
},
|
||||||
|
"missing checkpoint calendar accepted");
|
||||||
|
|
||||||
|
cache.apply_pull_page(PullPage{{}, {}, checkpoint("empty-window", "calendar-a", "", "")});
|
||||||
|
require(cache.snapshot().checkpoints.size() == 1
|
||||||
|
&& cache.snapshot().checkpoints[0].window_start.empty()
|
||||||
|
&& cache.snapshot().checkpoints[0].window_end.empty(),
|
||||||
|
"valid unwindowed opaque checkpoint was rejected");
|
||||||
|
|
||||||
|
cache.replace_calendars_after_complete_listing("account-a", {});
|
||||||
|
require(!cache.snapshot().calendars[0].active, "fixture calendar was not deactivated");
|
||||||
|
require_throws(
|
||||||
|
[&] { cache.apply_pull_page(page()); }, "inactive checkpoint calendar accepted");
|
||||||
|
require(cache.snapshot().events.empty(), "inactive-calendar pull changed event state");
|
||||||
|
}
|
||||||
|
|
||||||
|
void test_foreign_key_failure_rolls_back(const std::filesystem::path& root) {
|
||||||
|
const auto path = root / "foreign-key.db";
|
||||||
|
SyncCache cache{path};
|
||||||
|
seed_calendar(cache);
|
||||||
|
cache.apply_pull_page(page());
|
||||||
|
const auto baseline = cache.snapshot();
|
||||||
|
|
||||||
|
SqliteConnection external{path};
|
||||||
|
external.execute("PRAGMA foreign_keys=ON");
|
||||||
|
require_throws(
|
||||||
|
[&] { external.execute("DELETE FROM accounts WHERE id='account-a'"); },
|
||||||
|
"foreign-key violating direct delete succeeded");
|
||||||
|
require(cache.snapshot() == baseline, "foreign-key failure changed cached data");
|
||||||
|
}
|
||||||
|
|
||||||
|
void test_migration_failures_are_non_destructive(const std::filesystem::path& root) {
|
||||||
|
const auto collision_path = root / "collision.db";
|
||||||
|
{
|
||||||
|
SqliteConnection database{collision_path};
|
||||||
|
database.execute("CREATE TABLE accounts (sentinel TEXT NOT NULL)");
|
||||||
|
database.execute("INSERT INTO accounts VALUES ('keep-me')");
|
||||||
|
database.execute("PRAGMA user_version=0");
|
||||||
|
}
|
||||||
|
require_throws([&] { SyncCache cache{collision_path}; },
|
||||||
|
"incompatible migration collision was accepted");
|
||||||
|
{
|
||||||
|
SqliteConnection database{collision_path};
|
||||||
|
require(database.scalar_int("PRAGMA user_version") == 0,
|
||||||
|
"failed migration changed schema version");
|
||||||
|
require(database.scalar_text("SELECT sentinel FROM accounts") == "keep-me",
|
||||||
|
"failed migration changed preexisting data");
|
||||||
|
require(database.scalar_int(
|
||||||
|
"SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name!='accounts'")
|
||||||
|
== 0,
|
||||||
|
"failed migration left partially created v1 tables");
|
||||||
|
}
|
||||||
|
|
||||||
|
const auto future_path = root / "future.db";
|
||||||
|
{
|
||||||
|
SqliteConnection database{future_path};
|
||||||
|
database.execute("CREATE TABLE future_sentinel (value TEXT NOT NULL)");
|
||||||
|
database.execute("INSERT INTO future_sentinel VALUES ('future-data')");
|
||||||
|
database.execute("PRAGMA user_version=2");
|
||||||
|
}
|
||||||
|
require_throws([&] { SyncCache cache{future_path}; }, "future schema version was accepted");
|
||||||
|
{
|
||||||
|
SqliteConnection database{future_path};
|
||||||
|
require(database.scalar_int("PRAGMA user_version") == 2,
|
||||||
|
"future schema rejection changed schema version");
|
||||||
|
require(database.scalar_text("SELECT value FROM future_sentinel") == "future-data",
|
||||||
|
"future schema rejection changed data");
|
||||||
|
require(database.scalar_int("SELECT COUNT(*) FROM sqlite_master WHERE type='table'") == 1,
|
||||||
|
"future schema rejection created v1 tables");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void test_invalid_files_are_refused(const std::filesystem::path& root) {
|
||||||
|
const auto corrupt_path = root / "corrupt.db";
|
||||||
|
{
|
||||||
|
std::ofstream output(corrupt_path, std::ios::binary);
|
||||||
|
output << "this is not a SQLite database";
|
||||||
|
require(static_cast<bool>(output), "failed to write corrupt database fixture");
|
||||||
|
}
|
||||||
|
const std::string before = [&] {
|
||||||
|
std::ifstream input(corrupt_path, std::ios::binary);
|
||||||
|
return std::string{std::istreambuf_iterator<char>{input}, {}};
|
||||||
|
}();
|
||||||
|
require_throws([&] { SyncCache cache{corrupt_path}; }, "non-SQLite cache file was accepted");
|
||||||
|
std::ifstream after_input(corrupt_path, std::ios::binary);
|
||||||
|
const std::string after{std::istreambuf_iterator<char>{after_input}, {}};
|
||||||
|
require(after == before, "corrupt cache rejection changed the source file");
|
||||||
|
|
||||||
|
#if !defined(_WIN32)
|
||||||
|
const auto target_path = root / "symlink-target.db";
|
||||||
|
{
|
||||||
|
SqliteConnection target{target_path};
|
||||||
|
}
|
||||||
|
const auto symlink_path = root / "symlink.db";
|
||||||
|
require(::symlink(target_path.c_str(), symlink_path.c_str()) == 0,
|
||||||
|
"failed to create symlink fixture");
|
||||||
|
require_throws([&] { SyncCache cache{symlink_path}; }, "symlink cache path was accepted");
|
||||||
|
|
||||||
|
const auto directory_path = root / "directory.db";
|
||||||
|
std::filesystem::create_directory(directory_path);
|
||||||
|
require_throws([&] { SyncCache cache{directory_path}; },
|
||||||
|
"nonregular cache path was accepted");
|
||||||
|
#endif
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace
|
||||||
|
|
||||||
|
int main() {
|
||||||
|
try {
|
||||||
|
TempDirectory temporary;
|
||||||
|
test_fresh_schema_permissions_and_secrets(temporary.path());
|
||||||
|
test_existing_permissions_are_tightened(temporary.path());
|
||||||
|
test_account_calendars_and_deterministic_snapshot(temporary.path());
|
||||||
|
test_pull_pages_replacement_tombstone_and_windows(temporary.path());
|
||||||
|
test_multiple_connections_serialize(temporary.path());
|
||||||
|
test_input_validation_and_rollback(temporary.path());
|
||||||
|
test_sql_failure_rolls_back_snapshot_and_cursor(temporary.path());
|
||||||
|
test_checkpoint_calendar_and_shape_validation(temporary.path());
|
||||||
|
test_foreign_key_failure_rolls_back(temporary.path());
|
||||||
|
test_migration_failures_are_non_destructive(temporary.path());
|
||||||
|
test_invalid_files_are_refused(temporary.path());
|
||||||
|
} catch (const std::exception& error) {
|
||||||
|
std::cerr << "sync cache test failure: " << error.what() << '\n';
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
std::cout << "sync cache tests passed\n";
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user