Posted by ajayvk 11 hours ago
but....the SQLite file is locked against reads while writing in order to achieve serializable isolation. which sort of indicates you're better off doing your database work in an offline file, then just swapping the new file for the old one that's in production. which sort of indicates....just use a tar file, or a separate directory that you swap in to update the new content.
it is so much easier to serve static files statically rather than from some program that's trying to manage live SQLite connections and achieve some weird "update concurrency" magic when this problem is not at all that hard to solve. It's fine to manage your CMS in a sqlite database but when you serve the content live and it's static, use static files.
Not with WAL: https://www.sqlite.org/wal.html
Otherwise the file system is far superior for every "benefit" they list. And far easier to work with other software and more robust over time (nothing to change/break).
From what I remember, this was an atomic operation, and if there was a download in progress it would continue using the old file, because the data was still on disk and the filename was simply a pointer to the node.
This may behave differently on other file systems. This was done on an old Linux server with ext3
Seems like a simpler solution than using a db
I think the bigger benefit though is the increased read performance on many small files (saving system call overhead). To which amount that applies to static files that a smart server might keep in cache, I don't know.
So let's go over the claims about SQLite:
>Transactional Updates : This is the main benefit. Updating multiple files can be done in one transaction. Isolation ensures that there are no broken webapps during the update.
Yep, big one
>Deployment Rollbacks: Another of the transactional benefits is the ability to roll back deployment in case of errors. If multiple apps are being updated, all of them can be rolled back in one go. Rolling back a database transaction is much easier than cleaning up files on the file system.
I guess if you don't keep old artifact of what you are serving up. We zip up our entire static site as part of pipeline and dump into Azure Storage before deployment.
>File De-duplication Across Versions: Clace automatically versions all updates. This can lead to lots of duplicate files. The file data is stored in a table with the schema
INTERNAL SCREAMING Deduplication gives me nightmares from cache issues and so forth. Show me SQLite level of tests here to save a few kilobytes.
>De-duplication Across Apps : Each production app in Clace has an staging app. Apps can have multiple previews apps. This can lead to lots of duplication of files. Using the database helps avoid all the duplication. Even across apps, there are files which have the same contents. Files are de-duplicated across apps also.
INTERNAL SCREAMING Oh god, I hope this section of the code is extremely battle tested because de-duplication at this level gives me SRE nightmares.
>Easy Backups: Using SQLite means that backups are easy. The state of the whole system, metadata and files can be backed up easily using SQLite backup tools like Litestream.
tar directory is difficult?
>Content Hashing: For content caching on the browser, web servers return a ETag header. Using the database for files makes it easy to save the content SHA once during file upload without having to recompute it later.
Most web servers handle this for you.
>Compression: The file contents are saved Brotli compressed in the SQLite table. The database approach has the advantage that contents can be saved in multiple formats easily. GZip compressed data and uncompressed data can be added by just adding a column in the files table.
Cool, now you have to uncompress it every time you want to serve it? Is there massive shortage on disk space and I missed the memo?
I mean, I don't know what Clace does, seriously OP, reading your front page and I have no idea but disk space/speed is rarely a factor. Hell, most stuff probably gets loaded into RAM and is served from there at the end. Meanwhile, your content is being served out this proprietary file system and you are kind of locked in.
Clace is built for use cases where teams want to be able to easily deploy Streamlit/Gradio/FastAPI type internal apps (python is supported out of the box, other languages work if they have a Dockerfile present). Clace adds the image building, GitOps, OAuth access control, blue-green staged deployment, secrets management etc on top. So use cases where Kubernetes and an IDP would be overkill for.
If this person is anything like me, it's this "feature" that is giving them nightmares. This code has to be rock solid and impossible for anyone to mess up. It only takes one little bug and EVERY SINGLE APP GOES DOWN. Congrats on inventing a new single-point-of-failure!
Not necessarily! If the client supports Brotli encoded data (Accept-Encoding: br), you can pass it straight through without decompressing. Ditto for gzip, or zstd for newer browsers.