I’ve evaluated both RustFS and Garage, and here’s the breakdown:
Release Cadence: Garage feels a bit slower, while RustFS is shipping updates almost weekly.
Licensing: Garage is on AGPLv3, but RustFS uses the Apache license (which is huge for enterprise adoption).
Stability: Garage currently has the edge in distributed environments.
With MinIO effectively bowing out of the OSS race, my money is on RustFS to take the lead.
I guess I'm curious if I'm understanding what you mean here, because it seems like there's a huge number of counterexamples. GNU coreutils. The linux kernel. FreeBSD. NFS and iSCSI drivers for either of those kernels. Cgroups in the Linux kernel.
If anything, it seems strange to expect to be able to monetize free-as-in-freedom software. GNU freedom number 0 is "The freedom to run the program as you wish, for any purpose". I don't see anything in there about "except for business purposes", or anything in there about "except for businesses I think can afford to pay me". It seems like a lot of these "open core" cloud companies just have a fundamental misunderstanding about what free software is.
Which isn't to say I have anything against people choosing to monetize their software. I couldn't afford to give all my work away for free, which is why I don't do that. However, I don't feel a lot of sympathy to people who surely use tons of actual libre software without paying for it, when someone uses their libre software without paying.
The trick here is that people may not want to be coding MinIO. It's like... just not that fun of a thing to work on, compared to something more visible, more elevator-pitchy, more sexy. You spend all your spare time donating your labour to a project that... serves files? I the lowly devops bow before you and thank you for your beautiful contribution, but I the person meeting you at a party wonder why you do this in particular with your spare time instead of, well, so many other things.
I've never understood it, but then, that's why I'm not a famous open-source dev, right?
I haver not used it but will be likely a good minio alternative for people who want to run a server and don't use minio just as s3 client.
No test suite, plenty of regressions, and data loss bugs on core code paths that should have been battled tested after so many years. There are many moving parts, which is both its strength and its weakness as anything can break - and does break. Even Erasure Coding/Decoding has had problems, but a guy from Proton has contributed a lot of fixes in this area lately.
One of the big positive in my opinion, is the maintainer. He is an extremely friendly and responsive gentleman. Seaweedfs is also the most lightweight storage system you can find, and it is extremely easy to set up, and can run on servers with very little hardware resources.
Many people are happy with it, but you'd better be ready to understand their file format to fix corruption issues by hand. As far as i am concerned, i realized that after watching all these bugs, the idea of using seaweedfs was causing me more anxiety than peace of mind. Since we didn't need to store billions of files yet, not even millions, we went with creating a file storage API in ASP.NET Core in 1 or 2 hours, hosted on a VPS, that we can replicate using rsync without problem. Since i made this decision, i have peace of mind and no longer think about my storage system. Simplicity is often better, and OSes have long been optimized to cache and serve files natively.
If you are not interested in contributing fixes and digging into the file format when a problem occurs, and if your data is important to you, unless you operate at the billions of files scalability tier Seaweedfs shines at, i'd suggest rolling your own boring storage system.
For our use case (3 nodes, 61TB of NVMe) it seems like the best option out of what I looked at (GarageFS, JuiceFS, Ceph). If we had 5+ nodes I'd probably have gone with Ceph though.
Any alternatives besides racking own servers?
*EDIT* Did a little ChatGPT and it recommended tiny t4g.micro then use EBS of type cold HDD (sc1). Not gonna be fast, but for offsite backup will probably do the trick.
The closest alternative seems to be RustFS. Has anyone tried it? I was waiting until they support site replication before switching.
I hadn't heard of RustFS and it looks interesting, although I nearly clicked away based on the sheer volume of marketing wank on their main page. The GitHub repo is here: https://github.com/rustfs/rustfs
I don’t begrudge it that. I get the impression that Garage isn’t necessarily focussed on this kind of use case.
After years of using Garage for S3 for the homelab I’d never pick anything else. Absolutely rock solid, no problem whatsoever. There isn’t ONE other piece of software I can say that about, not ONE.
Major kudos to the guys at deuxfleurs. Merci beaucoup!
This was going to be my next project, as I am currently storing my Kopia/Ente on MinIO in a non-distributed way. MinIO project going to shi*s is a good reason to take on this project faster than later.
They have a lot of obviously fake quotes from non-existent people at positions that don’t even mention what company it is. The pictures are misgendered and even contain pictures of kids.
Feels like the whole page is AI generated.
So, arguably worse than MinIO.
This is not that. This is not even a license. They want a full transfer of intellectual property ownership. Sure that enables them to use it in a commercial product, but it also enables them to sue if contributors contribute similarly to other projects. Obviously that would create a shit storm, and there is an exception with the public license, but riddle me this: can you legally make similar contributions to multiple projects that have this type of CLA?
Let us take a step back and instead look where such terms are more common: employment contracts.
You don't need assignment to the project if you are not planning to change project's license.
You do need assignment to the project if you need to ever rugpull the community and close the code
I'd take a look at garage (didn't try seaweed yet).
Is it open to the public? I'd like to check it out
It's promising, but definitely check the roadmap before deploying at scale.
The core is stable at this point, but the user/policy management and the web interface is still in the works.
That said, if there was contributed AGPL code, they couldn't change the license on that part of the code w/o a CLA. AGPL also doesn't necessarily mean you have to make the code publicly available, just available to those that you give the program to (I'm assuming AGPL is like the GPL in this regard).
So, that I'd be curious about it is -- (1) is there any contributed AGPL code in the current version? (2) what license is granted to customers of the enterprise version?
Minio can completely use whatever license they want for their code. But, if there was contributed code w/o a CLA, then I'm not sure how a commercial/enterprise license would play with contriubuted AGPL code. It would be an interesting question to find out.
This is the crucial difference between the AGPL and the GPL: the AGPL requires you to make the code available to users for whom you run the code, as well as users you give the program to.
But, what I don't know is -- is there any other AGPL code that minio doesn't own, but that was otherwise contributed to minio? Because, presumably, they aren't actually giving their customers the minio program with an AGPL license, rather they have whatever their enterprise license agreement is. If this is the case, and there is AGPL code that's not owned by Minio, I can foresee problems in the future.
Additionally, in order to CHANGE the license, if others contributed code under that license, you would need their permission, on top of the fact, you cannot retroactively revoke the license for previous versions.
I can't see how this is a defensible position for Minio, but I'm not sure they really care that much at this point.
(I personally choose not to contribute to projects with CLAs, I don't want my contributions to become closed-source in the future.)
https://blog.min.io/weka-violates-minios-open-source-license...
They think they can revoke someone's AGPL license. That's not at all how that license works!
I have zero knowledge about the squabble between MinIO and Weka. I don't know, and don't care, if either of them is in the right. But if Weka isn't complying with the terms of the AGPL, then MinIO has the legal right to tell them they can no longer distribute MinIO's licensed work at all, because nothing else grants them that privilege.
If that weren't true, there'd be no teeth to the A?GPL whatsoever.
> All rights granted under this License are granted for the term of copyright on the Program, and are irrevocable provided the stated conditions are met.
it also says
> You may not propagate or modify a covered work except as expressly provided under this License. Any attempt otherwise to propagate or modify it is void, and will automatically terminate your rights under this License (including any patent licenses granted under the third paragraph of section 11).
> However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation.
> Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice.
This is in common with the GPLv3. It is much longer than the corresponding terms of the GPLv2 to remedy a sort of fragility in the GPLv2 which says your license terminates permanently if you ever violate the GPL, even temporarily and by accident.
I have no knowledge of whether Weka did or didn't violate the license, but if they did violate it and refused to fix it, MinIO's revocation of their license is completely in accordance with the terms of the license as written. I don't think a GPL termination case has yet been litigated.
It has, though it has mainly been under the "breach of contract" approach and not under "copyright infringement" approach. See https://en.wikipedia.org/wiki/Open_source_license_litigation
I do have a separate AGPL project (see github) where I have nearly all of the copyright and have looked into how one would be able to enforce this in the US at some point and it did look pretty bleak -- it is a civil suit where you have to show damages etc. but IANAL.
I did not like the FUD they were spreading about AGPL at the time since it is a good license for end-user applications.
Looks like a great alternative.
What is the performance like for reads, writes, and deletes?
And just to play devil's advocate: What would you say to someone who argues that you've essentially reimplemented a filesystem?
W.r.t. filesystem: Yes, aware of this. Initially used minio and also implemented the use case directly on XFS as well and only had problems at larger scales (that still fit on a machine) with it. Ceph went into a similar direction with BlueStore (reimplementing the filesystem, but with RocksDB).
https://github.com/minio/minio/issues/13308#issuecomment-929...
https://github.com/minio/minio/discussions/13571#discussionc...
> you may be violating AGPLv3 if you are using MinIO to build commercial products or using it for commercial purposes
Should I contact a MinIO salesman to purchase an enterprise license ASAP or is it fine if I license my kids and advent of code solutions under the AGPLv3 license ?
Or just that the users would need to make minio sources, including modifications, freely available?
I guess that's kind of the big question inherent to the AGPL?
If you use Minio or another AGPL licensed service internally to support your own product without a customer ever touching it's API, it should be fine.
So if the minio maintainers (or anybody that forks the project and wants to work it) can fix any security issues that may occur I don't see any problems with using it.
AFIK they removed it only to move it to their paid version, didn't they?
The UI was useful when first configuring the buckets and permissions; if you've got it working (and don't need to change anything) you're good to go. Also, everything can be configured without the UI (not so easily of course).
The concerning language for me is this part that was added:
> Critical security fixes may be evaluated on a case-by-case basis
It seems to imply that any fixes _may_ be merged in, but there's no guarantees.
Minio is/was pretty solid product for places where rack of servers for Ceph wasn't an option (it does have quite a bit higher memory requirements), or you just need a bit of S3 (like we have small local instances that just run as build cache for CI/CD)
But that's not where money is
"foo" and "foo/bar" are valid S3 object names that cannot coexist on a POSIX filesystem
This is a wild sentence--how can you criticize them for abandoning POSIX support __and__ building a massively over-complicated product? Making a reliable POSIX system is inherently very complex.
POSIX can be complicated, but it puts you in a nice ecosystem, so for some use-cases complex POSIX support is not over complicated. It is just… appropriately complicated.
IANAL, of course
Meanwhile, MinIO's own contributions and the distribution itself (outbound license) were AGPL licensed.
It's effectively a CLA, just a bit weaker, since they're still bound by the terms of Apache 2 vs. a full license assignment like most CLAs.
What is an AI Stor (e missing on purpose because that is how it is branded: https://www.min.io/product/aistor)
For production you'd need a proper cluster deployed via Helm, but for trying it out locally that setup is easy to get running.
Replication
A trusted identity provider is a
key component to single sign on.
It’s probably just Minio but it costs more money.
Choosing AGPL with contributors giving up rights is a huge red flag for "hey, we are going to rug pull".
Just AGPL by companies without even allowing contributor rights is saying, "hey, we are going to attempt to squeeze profit out and don't want competition on our SaaS offering."
I wish companies would stop trying to get free code out of the open source community. There have been so many rug pulls it should be expected now.