Posted by fortuitous-frog 5 hours ago
I'm advocating for JJ to build a proper daemon that runs "checks" per change in the background. So you don't run pre-commit checks when committing. They just happen in the background, and when by the time you get to sharing your changes, you get all the things verified for you for each change/commit, effortlessly without you wasting time or needing to do anything special.
I have something a bit like that implemented in SelfCI (a minimalistic local-first Unix-philosophy-abiding CI) https://app.radicle.xyz/nodes/radicle.dpc.pw/rad%3Az2tDzYbAX... and it replaced my use of pre-commit hooks entirely. And users already told me that it does feel like commit hooks done right.
I haven't yet submitted it to upstream for design discussion, but I pushed up my branch[1]. You can also declare a revset that the target revision must match, for extra belts and suspenders (eg., '~conflicts()')
[1] https://github.com/paulsmith/jj/tree/protected-bookmarks
I personally can't stand my git commit command to be slow or to fail.
[0]: such as https://github.com/watchexec/watchexec
That’s reversing the flow of control, but might be workable!
Basically what I would want is write a commit (because I want to commit early and often) then run the lint (and tests) in a sandboxed environment. if they pass, great. if they fail and HERAD has moved ahead of the failing commit, create a "FIXME" branch off the failure. back on main or whatever branch head was pointed at, if tests start passing, you probably never need to revisit the failure.
I want to know about local test failures before I push to remote with full CI.
automatic branching and workflow stuff is optional. the core idea is great.
I'm not sure if I fully understood. But SelfCI's Merge-Queue (mq) daemon has a built-in hook system, so it's possible to do custom stuff at certain points. So probably you should be able to implement it already, or it might require couple of minor tweaks (should be easy to do on SelfCI side after some discussion).
Also, how do you like Radicle?
SelfCI is _very_ minimal by design. There isn't really all that much to document other than what is described in the README.
> Also, how do you like Radicle?
I enjoy that it's p2p, and it works for me in this respect. Personally I disagree with it attempt to duplicate other features of GitHub-like forge, instead of the original collaborate model of Linux kernel that git was built for. I think it should try to replicate something more like SourceHut, mailinglist thread, communication that includes patches, etc. But I did not really _collaborated_ much using Radicle yet, I just push and pull stuff from it and it works for that just fine.
I'm sure this is more reliably than pre-commit, but you still have hooks building Python wheels and whatnot, which fails annoyingly often.
The VFS stuff is not quite finished yet though (it's really complicated). If anyone wants to help me with that it would be welcome!
I think wasi is a cool way to handle this problem. I don't think security is a reason though.
Changes to code would obviously need to be reviewed before they are committed. That's still much better than with pre-commit, where e.g. to do simple things like banning tabs you pretty much give some guy you don't know full access to your machine. Even worse - almost everyone that uses pre-commit also uses tags instead of commit hashes so the hook can be modified retroactively.
One interesting attack would be for a hook to modify e.g. `.vscode/settings.json`... I should probably make the default config exclude those files. Is that what you meant? Even without that it's a lot more secure than pre-commit.
pre-commit considered harmful if you ask me. prek seems to largely be an improvement but I think it's improving on an already awful platform so you should not use it.
I know I am working on a competing tool, but I don't share the same criticism for lefthook or husky. I think those are fine and in some ways (like simplicity) better than hk.
There should be a .gitextensions in the repo that the repo owners maintain just like .gitignores and . gitattributes etc etc. Everything can still be opt in by every user but at least all git clients would be able to know about, pull down, and install per user discretion.
It seems pretty basic in this day and age but it's still a gaping hole. You still need to manually call LFS install for goodness sake.
Is prek much better?
The main advantage for me is that prek has support for monorepo/workspaces, while staying compatible with existing pre-commit hooks.
So you can have additional .pre-commit-config.yaml files in each workspace under the root, and prek will find and run them all when you commit. The results are collated nicely. Just works.
Having the default hooks reimplemented in Rust is minor bonus (3rd party hooks won't be any faster) and also using uv as the package manager speeds up hook updates for python hooks.
Dedicated a whole chapter to it in my latest book, Effective Testing.
The trend of fast core (with rust) and convenient wrapper is great while we are still writing code.
Why not just call a shell script directly? How would you use these with a CI/CD platform?
From an org standpoint you can have them (mandate?) as part of the developer experience.
(Our team doesn't use them, but I can see the potential value)
The checks in those pre-commit hooks would need to be very fast - otherwise they'd be too slow to run on every commit.
Then why would it save time and money if they only get run at the pipeline stage? That would only save substantial time if the pipepline is architected in a suboptimal way: Those checks should get run immediately on push, and first in the pipeline so the make the pipeline fail fast if they don't pass. Instant Slack notification on fail.
But the fastest feedback is obviously in the editor, where such checks like linting / auto-formatting belong, IMHO. There I can see what gets changed, and react to it.
Pre-commit hooks sit in such a weird place between where I author my code (editor) and the last line of defense (CI).
That's still multiple minutes compared to an error thrown on push - i.e. long enough for the dev in question to create a PR, start another task, and then leave the PR open with CI failures for days afterwards.
> But the fastest feedback is obviously in the editor, where such checks like linting / auto-formatting belong, IMHO.
There are substantial chunk of fast checks that can't be configured in <arbitrary editor> or that require a disproportionate time investment. (e.g. you could write and maintain a Visual Studio extension vs just adding a line to grep for pre-commit)
The prek documentation has a list of many large projects (such as CPython and FastAPI, to name a few) who use it; each link is a PR of how they integrated it into CI if you want to see more: https://prek.j178.dev/#who-is-using-prek
You run the same hooks in CI as locally so it's DRY and pushes people to use the hooks locally to get the early feedback instead of failing in CI.
Hooks without CI are less useful since they will be constantly broken.
i regularly edit history of PRs for a variety of reasons and avoid pre-commit when possible.
put it all in CI thank you please — gimme a big red X on my pipeline publicly telling me i’ve forgotten to do something considered important.
Slow hooks are also not a problem in projects I manage as I don't use them.
> Slow hooks are also not a problem in projects I manage as I don't use them.
You bypass the slow hooks you mentioned? Why even have hooks then?
I have a shell utility similar to make that CI/CD calls for each step (like for step build, run make build) that abstracts stuff. I'd have Prek call this tool, I guess, but then I don't get what benefit there is here.
* CI (I understand pre-commit shifts errors left)
* in editor/IDE live error callouts for stuff like type checking, and auto-formatting for things like "linters".
Do you run tests? How do you know _which_ tests to run, and not just run every test CI would run, which could be slow?
If it’s on a pull/merge request, you’re wasting reviewer time.
If the hook is blocking secrets, you can’t un-push it with 100% certainty so you have to revoke credentials.
For texts, I tend to have the equivalent of “pytest tests/unit/“ since those are fast and a good sanity check, especially for things like refactoring.
I also run our pre-commit checks in CI for consistency so we’re never relying on someone’s local environment (web editors exist) and to keep everyone honest about their environment.
I can't, because the point of our pre-commit use isn't to run logic in hooks that can't be run otherwise.
e.g. We use pre-commit to enforce that our language's whitespace formatting has been applied. This has the same configuration in the IDE, but sometimes devs ignore IDE warnings or just open files in a text editor for a quick edit and don't see IDE warnings or w/e.
"Replaced by CI" isn't really meaningful in our context - pre-commit is just a tool that runs as part of CI - some things get done as pre-commit hooks because they're fast and it's a convenient place to put them. Devs are encouraged to also run pre-commit locally, but there's no enforcement of this.
> Do you run tests? How do you know _which_ tests to run, and not just run every test CI would run, which could be slow?
We have performance metrics for pre-commit hooks and pre-push hooks. I forget the exact numbers, but we want stuff to "feel" fast, so e.g. if you're rebasing something locally with a few dozen commits it should only take seconds. Pre-push hooks have a bit more latitude.