Top
Best
New

Posted by jorangreef 2 days ago

Spiral(spiraldb.com)
258 points | 88 commentspage 3
holoduke 2 days ago|
So basically this is a file system that runs on your gpu?
whalesalad 2 days ago||
The hot new aesthetic these days is either "receipt printer" or "liquid glass". I dig it, tbh.
sys13 2 days ago||
I wonder how much we need this vs implementing it as part of Delta Lake or Iceberg
mritchie712 2 days ago|
yeah, I think you could get much of the benefits if Iceberg (or ducklake) added support for `.vortex` (e.g. ducklake only supports `.parquet` right now)
skywhopper 2 days ago||
Man, they are really proud of that initial seed round funding aren’t they? Forgive me, but $22 million does not sound like enough to truly revolutionize data processing technology.

The gist seems to be that they can overcome network latency issues when dealing with huge numbers of smallish objects in S3-like storage systems that need to be fed into GPUs? Yeah, those formats and systems were not designed to feed that type of processor. You’re doing it wrong if this is your problem.

After a lot of nonsense, it sounds like they just reformat the data into something more efficient instead. But they forget about the network latency and blame CPUs for slowing things down? And what was that sidetrack about S3 permissions?

I wouldn’t jump right onto this… well, it’s not clear what this even is exactly. But you can probably wait it out.

bflesch 2 days ago||
Even though Linux Foundation is only a 501(c)(6) nonprofit and not a 501(c)(3) nonprofit I wonder if all this legalese about "donation" might have very positive tax consequences for the donor.
datadrivenangel 2 days ago||
$22 million gets you a potentially commercially viable narrow featured database and some time to get enough sales to validate the VCs throwing another $50m at you to flush it out and build out the sales arm seriously.
mlhpdx 2 days ago||
I stopped reading at “new era”. At this point in time with the deluge of content, start with a problem and solution in a concise statement if you want my attention. I’m not reading your opinion piece.
bflesch 2 days ago||
Big ick from my side. Manifest-style marketing blog post talking about revolutionary things but it seems their main metric is in the image above the post: "hey, we've raised $22M in funding".

Landing pages of both spiral and vortex are GPU-hugging animations and void of any technical information. Empty nothing-statements like "machine scale". They claim 100x improvements but don't link any metrics.

Maybe this is a "don't hate the player, hate the game" situation but somehow the collective of likeminded AI engineers decided to upvote this post to #1 on HN.

msteffen 2 days ago||
There's this: https://bench.vortex.dev/, which links to https://github.com/vortex-data/vortex/tree/develop/bench-vor.... I haven't tried pulling the repo or anything but it seems like they might be runnable?

Of course I don't know what benchmarks or performance metrics they might have for the db layer, but it is something.

bflesch 2 days ago||
Thanks, it was behind a "see benchmarks" link at the very bottom of https://spiraldb.com/. I noticed these claims on both vortex.dev and https://spiraldb.com/vortex website, but both had no hyperlink to any actual metrics.
indoordin0saur 2 days ago||
> Vortex is designed to support decoding data directly from S3 to GPU, skipping the CPU bottleneck entirely.

If this is true I'm inclined to believe their claims.

bflesch 2 days ago||
MY PERSONAL BOTTLENECK between S3 and GPU is my credit card and not some new cargo module by some already-rich AI engineer and a fancy marketing website that must've cost a couple hundred grand.

And if this module provides a benefit I'm sure it will find its way into our stack, just like PostgreSQL did. And PostgreSQL never had $22M to begin with - no shiny marketing, just technological skills.

The whole "donated by spiral" on the vortex.dev website also gives big tax write-off vibes.

IMO best case is that this will be a mongodb scenario, but with the current track record of tech grifters enshittifying everything they might find a creative new way.

xyzzy_plugh 2 days ago|||
> The whole "donated by spiral" on the vortex.dev website also gives big tax write-off vibes.

I've never heard of this sort of OSS work being used as a tax write-off. Could someone please either clarify, or enlighten me?

bflesch 2 days ago||
Look at the website linked in the vortex website footer: https://lfprojects.org/ It has all the bells and whistles of using an expensive law firm.

I have no idea who exactly is behind this, but to me it does definitely not seem like a no-name open source genius, I assume it is some lucky AI grifter. They have two nicely designed, expensive marketing websites. They have all the legal documents for the parent LLC in Delaware.

The delaware corp "donates" the multi-million-worth tech to linux foundation, and uses it as tax write-off to offset gains from some other lucky AI grifter play the person did.

Just the chuzpe to self-compare yourself to something like PostgreSQL is what gets me. Why can't they just be rich and leave people doing actual work for the benefit of our common good be. No, they must make big blog posts claiming they are the next big thing after PostgreSQL.

So many red flags..

aduffy 2 days ago||
Believe it or not, this is how the Linux Foundation organizes itself. It's more legwork than something simpler like Apache Foundation.

Basically in the US you need a legally recognized entity to hold intellectual property. "Donating" the project involves setting up a "Series LLC" that is nested underneath the top-level Linux Foundation corporation, and donating the IP into it.

Checkout https://docs.linuxfoundation.org/lfx/project-control-center/... and ctrl-f "LF Projects, LLC"

bflesch 2 days ago||
Oh, thanks for pointing that out. I got it all mixed up.

But I think my argument still stands. Linux foundation is a 501(c)(6) nonprofit, see https://www.linuxfoundation.org/legal/bylaws

So you might still be able to do an "intellectual property transfer" to them and use it as a tax write-off. The "LF Projects LLC" is then the new owner, only the operating company who has the ongoing hosting contracts for the websites.

Edit: Not sure if a donation to 501(c)(6) can be used as write-off without using some other legal loopholes. Quick AI search told me that only 501(c)(3) can do the donation tax write-off thing.

I'm sure there are some good tax lawyers behind this, who am I to understand it as a mere mortal I am just jealous.

aduffy 2 days ago||
We don't get a tax write-off.

The motivation is to move the IP and trademark into a separate organization so it's no longer owned by Spiral. This means we can't re-license it later, we'd have to fork it, because the Vortex trademark and all that is controlled by LF.

bflesch 2 days ago||
Thanks for explaining it. All the best.
aduffy 2 days ago|||
> The whole "donated by spiral" on the vortex.dev website also gives big tax write-off vibes.

Donated is the Linux Foundation terminology.

Sadly the last time I filed a tax return there was no way to itemize a Github repo. Alas.

curtisszmania 2 days ago||
[dead]
SomeHacker44 2 days ago||
"100KiB images"... This is odd. Most of my images are 2.5-4 MB. My raw images are 3-10x larger.
turnsout 2 days ago|
I bet this refers to some common training use case that leverages 512px or 1024px images. Or it’s just Palantir scanning security camera frames.
raziel2p 2 days ago|
> Vortex is designed to support decoding data directly from S3 to GPU, skipping the CPU bottleneck entirely.

how is this significant? surely either the network or the GPU calculations is the bottleneck here?