Posted by ibobev 1 day ago
My question is, why is this taken as a given? Is it so hard to have docs and code live together in version control after a certain scale? If so, what is the specific problem and what is the cause?
I ask because I've never been that satisfied with the various ways I've tried to organize projects in git. Recently I've been trying to keep the source, tests and docs together in the same tree so that changes are more localized. It seems to be helping me keep track of things, especially with coding agents so eager to make changes all over the place. I find their proclivity to repeat the same idea in multiple locations (agent instructions, docs, docstrings, help strings, comments) especially problematic.
Most people advocating a monorepo have never worked on a project large enough to see the issues with a monorepo and so are arguing for a monorepo without understanding the problems with them. For most people a monorepo is the correct answer because their project is small.
By the time you’re “at scale” (who knows), and all these monorepo at scale problems start to overwhelm, you can switch strategy, because the economy of polyrepos is so obvious by then.
So far, I’ve started a new job a handful of times by collapsing a premature polyrepo strategy: people were not experienced enough to merge two git repos without a common root.
I’ve only once went the other way, and it incurred so much overhead, it decreased developer productivity by some small but not insignificant percentage.
To be clear: I’m not a maximalist. All of my open-source work is exceedingly compartmentalised. My DNS library is separate from my external-dns webhook is separate from my fork of external-dns. They could all live in one repo. But FOSS encourages reusability, commercial software encourages clumping and vendoring.
Conversely to your experience, I have worked at a handful of places who have a monorepo that has been creaking under its own weight for years, but its structure as a monorepo now underpins the business, and so migration to a polyrepo simply never happens, and developers are now checking out a 50GB repo in its entirety periodically.
Instead I deal with a 30+ repo clusterfuck (technically we have 60+ services, but I only have to run half...) that is held together by hopes and prayers, takes literal hours of actual effort to bring everything up to date on master, and has become a fractured hellscape where people are afraid to leave their tightly constrained silos of service combinations.
Long story short... I will take a bad monorepo over bad multirepo any day of the week.
Really? I feel like most of the stuff I have read advocating monorepos are from people at Google, which is a HUGE monorepo.
Google also is very much in the yell loudly and ignore anyone who points out the problems of a monorepo.
Managers and others won't touch the repo. (Sometimes it's better the don't...)
I have not taken full advantage yet, but every source sub-directory can have its own CLAUDE.md or AGENTS.md file, with instructions for a given directory, whenever something like Claude opens a file in a folder, if there's an appropriate agent doc in the housing folder, it should read / apply its contents when working. Not sure how this happens with Sub-Agents on that note.
I think if you want both, you might as well add a ./docs/ directory, and put your md documents in there however you want, this has the upside of letting you have docs with your code always, as well as letting you link to direct source code files.
I always initialize my projects with a src, and docs, directory, for exactly this reason.
My reasoning is that I shouldn't have to go hunting for the docs for the code, or vice versa.
edit: I have had one pain point: devs/managers ask me why I do that, and that I stop. I refuse.
Not a big deal if you're basically the only developer and handroll the process - but poly repositories make release and dependency management a lot more straightforward to wrangle
It is unlikely GB/TB is ever a measure, though if your repo is that big and some people only need a subset of the repo it would be.
The linked URL to the home page there can even serve as an example: the "trunk" is a check-in name (https://fossil-scm.org/home/doc/trunk/www/checkin_names.wiki) that points to the newest check-in on the "trunk" branch. You can replace it with any other reference to get the old version; eg, version-2.20 would work to get the version 2.20 of the docs, 2015-03-14 would work to get the version from 14 March 2015, etc.
Having said that, there are central hosting projects, but where that has some value with git, git proper is distributed and does not need it but all the auxiliary stuff does. with fossil it provides almost no value. All the auxiliary stuff is built in so all it provides is a place to host. Which is fair but hosting is not hard with fossil.
With everything included in fossil I abuse it as a personal social platform(think discord) super easy to host and it gets me chat, forums, wiki, and file storage. None of them great, but it is so easy I don't really care. voice and video do require an additional service, so there is that. Now I just need to find some friends...
There are only a couple things that I miss:
- Grouping repos together into a combined project. I'd love to be able to have a single Fossil server/instance with a single set of users, wiki pages, tickets, etc. but multiple independent codebases. Closest I've gotten to that is to simply have multiple independent branches instead of a single trunk (example: https://fsl.yellowapple.us/avorion/home), and it's worked surprisingly well, but it's clear Fossil wasn't designed with this workflow in mind, so there are some rough edges with it (albeit minor and easy to work around).
- Compatibility with things that assume you're using Git. Being able to export to Git helps a lot here, but it's still extra steps. Ideal solution here would be for the Fossil server to be able to double as a Git forge and present repos accordingly, such that I can point things like CI/CD pipelines or Terragrunt module calls or whatever directly to the Fossil repo itself over the Git interface those things expect instead of having to setup a Git forge manually and somehow synchronize everything w.r.t. access controls.
- An equivalent to Git's submodules. This would IMO help address the “grouping repos together into a combined project” case as well.
Never wanted to do what you want but it sounds possible
set up the fossil server on a directory of repos probably with --repolist to get a list
fossil server --repolist /var/fossil/
change the menus in each repo to point to the core repo. Based on some quick tests I suspect a full http:// url is required.
- Forum /forum {@2 3 4 5 6}
+ Forum https://mydomain.org/core/forum {@2 3 4 5 6}In that case, when the docs are not associated with a code change, you want to make getting those improvements into the docs as frictionless as possible, otherwise the changes aren't going to get made.
Personally, I've found that making docs updates incredibly fast + easy to be far more valuable than anything you get from forcing doc changes through the full SDLC process. If someone has feedback on your docs changes they would have shared in a review, they can just update the docs instead.
In my experience, this is also true of a lot of code as well. Your dev scripts should probably have much more relaxed standards than your service source or CI/CD. Ideally I could define merge requirements by directory without doing some weird shenanigans with the CODEOWNERS file and a bot.
You can even do live edits on the web if you don't want to use a command line.
I can think of one other possibility. It's easier to write in a wiki via the browser. I can open that on my phone and edit docs. I can open it in my browser and edit docs.
On desktop it's a tiny bit more to pull the repo and open it in your editor (and you might already be there) but that tiny bit can be enough to stop you from writing documentation. For me, writing documentation must be totally painless so that I'll actually do it.
Why am I not a fan of the add-ons like PR's, wiki's, discussions, projects, and issues? Because they each introduce vendor lock-in to varying degrees.
[0]: https://docs.github.com/en/communities/documenting-your-proj...
The counter-argument i envision is: "update your docs and code at the same time in the same PR!" That works great, until you want to document something that isn't precisely tied to a single piece of code. In fact I think the most useful docs describe high level systems rather than being associated with specific pieces of code. Use comments for that; in contrast, docs should be easily editable by anyone at all times, otherwise they never get updated (an evergreen problem in any scenario).
Docs do not.
So what results is either allowing developers to push (but not force push) directly to main (you can always push revert commits if needed), or a PR process that exists to enforce linters and build-ability, but if those pass, allow the developer to merge independently, without human review.
But I think the biggest benefit, which we shouldn’t overstate, is that agents will just update docs as they find them. Including for big picture systems. Keeping docs updated is a PITA.
Writing style of AI often sucks. But I’ve found it pretty easy to rectify. And having some correct context is better than nothing or outdated docs in a lot of cases.
Wikis typically connote distributed, anonymous edits. This feature is partially covered by git already.
That's because the original sin of GitHub "wikis" is that they weren't (and most of them still aren't) even wikis. There's this perverse thing that happened during the wiki age, where people unable or unwilling to get on board decided to just start calling things "wikis" even though they exemplify the very thing that the wiki was invented as a response to. The reckless debasing of the word then infected adjacent spaces. Sourcehut's "read-only wikis" (wat) aren't even designed to be edited in the browser; on Sourcehut, "Publishing your changes is as easy as committing them and pushing them upstream." Newsflash: That's not a wiki.
Apparently it can even do CI stuff. Still wouldn't recommend it though, for the other reasons outlined in TFA and this thread.
> You can edit wikis directly on GitHub, or you can edit wiki files locally.
https://docs.github.com/en/communities/documenting-your-proj...
> Every wiki provides an easy way to clone its contents down to your computer. Once you've created an initial page on GitHub, you can clone the repository to your computer with the provided URL.
Every GitHub wiki you visit also has a section at the bottom that points to that URL with the description
> Clone this wiki locally
as well.
The initial version of this post opened with “You can use the wiki or a docs folder for your GitHub project, both are valid choices” but as I wrote more, I realised that there is a single reason to use a wiki, and many more reasons not to use the wiki. So many in fact, that I consider using the wiki on GitHub is an anti-pattern.
A very straightforward example of McCulloch's quote that "Writing is thinking."