Top
Best
New

Posted by Peter5 10/28/2024

SQLite Rsync: Database Remote-Copy Tool for SQLite(sqlite.org)
75 points | 11 comments
vanous 10/28/2024|
Thank you very much.

It would be useful to have dates for created and for edited on the articles. With the huge scope of the SQLite project, I have no idea whether this is historical, current, or upcoming.

unsnap_biceps 10/28/2024|
History on the SQLite repo seems to say this was added Sep 10, 2024

https://github.com/sqlite/sqlite/commit/a9c8f7cf34545f410e94...

rrevi 10/28/2024||
And made it to the news section of the SQLite site on Oct. 21st 2024:

https://sqlite.org/news.html

rich_sasha 10/28/2024||
Fantastic, looks like something I need! Some questions in case someone knows:

- is the source DB blocked in any way when this is running? E.g. is this like an ordinary read, or something different? I know WAL mode gives more concurrency but still writes are queued up in the wal file during a long read

- can this operate on a ~continuous basis? Or can I run it every few mins?

Thanks!

zimpenfish 10/28/2024|
> is the source DB blocked in any way when this is running?

Doesn't sound like it: "If other processes change the content of ORIGIN while this command is running, those changes will be applied to ORIGIN, but they are not transferred to REPLICA"

Sounds like it uses the internal snapshot mechanism[0] and replicates that.

[0] https://www.sqlite.org/c3ref/snapshot.html

config_yml 10/28/2024||
This is great, could be useful for keeping a standby up to date with a systemd timer/cron.
ollybee 10/28/2024||
It seems I have some scripts to update today
avinassh 10/28/2024||
checkout Simon Willison's notes if you want to know how to get this running and try: https://til.simonwillison.net/sqlite/compile-sqlite3-rsync
rrevi 10/28/2024||
Which now begs the question, sqlite3_rsync vs Litestream[0]?

[0] https://litestream.io/

zimpenfish 10/28/2024||
Why not both? They solve different problems, after all.

I'm currently using `litestream` for backups but I might add `sqlite3_rsync` as a point-in-time replica for things that benefit from "remote" sqlite access - easier than restoring a version from `litestream`, safer than copying it myself, and a lot easier than transporting changes over NATS / HTTP / whatnot.

kgeist 10/28/2024||
Looks like Litestream streams changes but sqlite3_rsync only updates a snapshot at the time it was called.