Top
Best
New

Posted by bastitx 15 hours ago

Kolibri: A Sovereign Open-Weight Model(aleph-alpha.com)
tech report: https://aleph-alpha.com/downloads/tech-report.pdf

additional paper: https://tej.as/blog/aleph-alpha-kolibri

481 points | 294 commentspage 5
mistyvales 11 hours ago|
The Sega 32X game??
12949468 9 hours ago||
It still steals my IP without attribution. Now we have state sanctioned sovereign theft instead of foreign theft.
Larrikin 8 hours ago||
The name really evokes strong Kotlin library naming vibes.
9dev 12 hours ago||
Aleph Alpha is just a sad joke by now. The talent isn't there anymore, they never managed to catch up to the other labs, failed to deliver on several projects, and by now are just a cash grab for the investors.
gchamonlive 11 hours ago||
Only if you think in terms of short term gains, but thinking in the long run, doesn't really matter if these models are crap today, all models will eventually be obsolete, unless we plateau hard on every aspect of the tech.

What matters is to have good sovereign models in 5, 10, 20 years time.

roncesvalles 11 hours ago|||
I'm curious why you think sovereign models matter that much if open source ones exist (unless that stops at some point). Sovereign model hosting services, yes.
HotHotLava 11 hours ago|||
Just open weights are not enough, I think as a nation state you'd at least want a whole sovereign training pipeline. Otherwise you are stuck when Kimi K4 stops releasing their weights.

But I don't see why this needs to be done on a state-by-state level, a sovereign EU model seems to be much more realistic in terms of funding.

rixed 10 hours ago||
Wouldn't you also want to be able to manufacture some sort of GPU?
DoctorOetker 8 hours ago||
I actually think we should be revisiting PMT photomultipler tube construction, and electron-"optical" matrix multiplication.

The annoying part of optical matrix multiplication is the conversion from electronic to optic and back.

Why not use electron-optics for matrix multiplication and use dynodes for amplification.

pstuart 7 hours ago||
While an intriguing concept, that doesn't sound like it could scale to the amount of "cores" needed to do the work. Or am I missing something?
tensor 42 minutes ago||||
Because if you can't build the model from scratch you don't have a model. Open weights are nice but fixed in time. This isn't a technology that stays static, it improves, and sovereignty means being able to continue to improve independently.
gchamonlive 11 hours ago||||
We have lots of capable open weights models, but very few have actual datasets and pipelines open for reproducibility. This is a major issue that even if big companies addressed, policies could well just change, as you also acknowledge.

Society can't depend on the whims of the private sector.

brainwad 8 hours ago|||
> Society can't depend on the whims of the private sector.

Yeah we can, we do it all the time. Nothing in society works without relying on private actors to keep doing what they have always done. Unless you go full soviet command economy, which in fact works worse.

gchamonlive 8 hours ago||
Society and the private sector cooperates, but any technical dependency is detrimental because of monopolization of interests. You don't need to go full soviet, but knowledge in general must be socialized, that includes access to technology.
brainwad 8 hours ago||
We literally grant 20 year monopolies on technologies to their inventors and have for centuries and it's fine. The entire economy runs on chips made by like, 3 foundries using lithography machines made by 1 company. I think you are overstating the necessity.
gchamonlive 8 hours ago||
This example proves the exact opposite, that is a national security disaster waiting to happen
brainwad 6 hours ago||
Autarky would be a national security disaster manifested. You can't be a modern prosperous society and not rely on either trade nor private enterprise.
gchamonlive 6 hours ago||
I totally agree, hence my previous comment

https://news.ycombinator.com/item?id=49945572

ordersofmag 7 hours ago|||
I think you confused "can't" with "shouldn't".
gchamonlive 6 hours ago||
Yes, society in general shouldn't, what I meant to say is that democratic society can't, otherwise it'll devolve invariably into an oligarchy.
hypfer 11 hours ago||||
The first thought that comes to mind is biases that are baked into the weights.

The next thought is culture-specific workflows, requirements etc that are best trained into weights by people that actually understand those needs.

HarHarVeryFunny 11 hours ago||||
Like it or not, AI has become part of modern warfare and well as part of the surveillance state (good for crime fighting, not for privacy), so it is an important part of national security, and you'd prefer to be as much self-sufficient for that as you can be.

Not all needs are going to be met by using or finetuning general purpose models, so ability to build your own SOTA ML models (LLMs or not) is also important.

stefs 11 hours ago||||
Open weight and -source models aren't without biases. Giving a model away for free would be a good way to distribute an agenda. With a sovereign model you can at least control the agenda and align it with your values.
mistrial9 11 hours ago|||
amazing to see blunt unfiltered commentary about making war and society level propaganda as important and necessary for a political state.
9dev 11 hours ago||||
For the same reason it didn't matter whether coach makers were able to build carriages that went 5% faster when the automobile was introduced, and for the same reason it won't matter if VW and Mercedes are going to improve their EV lineup in the next 5, 10, 20 years time. Their window of opportunity will be long gone and the momentum elsewhere; most probably, the market will even have evolved further by then.
HotHotLava 11 hours ago|||
Ford started mass producing cars in 1913, and yet there are plenty of car companies that started production 20+ years later and are still relevant.
HarHarVeryFunny 11 hours ago||
I came here to say the same thing.

AI is here to stay - will be around for hundreds/millions of years. Whether company A is a few years ahead of company B is irrelevant. In 10 years time company A, who started the industry, may be gone completely - also irrelevant.

Quothling 10 hours ago||||
> VW

VW EV sales are up in South America, Canada and Europe. It's true that their total global numbers are down, but that's mainly because they lost like 30% in China. Which obviously sucks considering how big the Chinese market is, but then, the Chinese market is it's own sort of thing.

Don't get me wrong, I think your point is valid for a lot of traditional European car brands. Mercedes is certainly one of them which has lost it's "our engines are nicer" brand, but I suspect VW is one of the brands that will do just fine. Especially with the id polo coming out next year.

pbmonster 10 hours ago||
> Mercedes is certainly one of them which has lost it's "our engines are nicer" brand

Mercedes is doing something interesting with its level 3 DRIVE PILOT. As far as I know, it's the only autonomous driving system/auto pilot on the market that actually takes full legal responsibility for the vehicle and its actions. If you put a new Mercedes in self driving and it crashes, Mercedes takes full liability.

Yes, it currently only goes up to 100 kph and it will buzz you to take over if it can't guarantee prefect safety anymore. But this approach is really the only one I think is interesting. All this self driving shit is worthless to me if I'm still liable in the end. So I respect Mercedes for putting their money where their mouth is.

mmasu 11 hours ago||||
maybe to have a global market share, but there are plenty of players that picked up pace when they “decided” to build cars (Japan? South Korea? and many more) or electronics. Having a good sovereign model in 10/20 years might not need having the absolute best there is, and perhaps future optimizations will make building one not as prohibitively expensive as it is today. Keeping some sort of homebrew capacity now is definitely better than having none
KronisLV 11 hours ago||||
> it won't matter if VW and Mercedes are going to improve their EV lineup in the next 5, 10, 20 years time

I mean, given the fuel prices it would be very nice, alongside more charging infrastructure in the EU. :(

Not to detract from the overall point, though to be honest, it's probably worthwhile to do improvements and refinements with the current technology, even if the future holds something vastly different. Both cause of gaining expertise and also maybe an improvement or two along the way, that might carry forward.

Like I haven't seen many steam locomotives around and for me the difference between saturated steam engines and superheated ones isn't very material in regards to transportation, but the latter is used in modern turbines.

leonidasrup 9 hours ago||
There are many "steam" locomotives around in China and India, but you just don't see them.

Coal is burned and converted to electricity in coal power plants and the electricity is used to move the electric locomotives around.

Even with distribution losses, the thermal efficiency of coal power plant -> electric network -> electric train is much higher then the old steam locomotives (Only about 5% of the potential energy produced by a steam locomotive’s boiler is translated to the wheels in the form of actual driving power.)

gchamonlive 11 hours ago||||
When you mention a window of opportunity, what do you mean by that? A window of opportunity for what?
hypfer 11 hours ago|||
And who are you, exactly?
switchbak 10 hours ago||
Well, they're "Head of Engineering @ matchory.com"

And who are you to be asking such questions?

hypfer 10 hours ago||
Someone mad at this random guy who can't even fix his de domains SSL cert (cloudflare of course) destructively shitting on any efforts at doing something good.

Someone that builds things instead of merely "managing".

Someone pissed by this exact style of default github pfp business-person ruining this country. If they haven't left for SV already, in which case I hope that they will stay there.

switchbak 2 hours ago|||
So you're mad that they mentioned German cars? And that they're a manager? And something about an SSL cert, really?

Serious projection issues, and anger to boot. Maybe take a walk outside, that was a very silly comment to get so worked up about.

9dev 8 hours ago||||
So many assumptions there. For one, I’m absolutely proud to be founding engineer of a German company that’s staying one, and I’m the first to make a stand for European technology. You just won’t get a high five from me for a company that isn’t managed well, does not deliver on their promises, and ultimately doesn’t really serve the goal of European AI independence well.
gchamonlive 10 hours ago|||
Feel your pain but ad hominen isn't the way my dude
hypfer 10 hours ago||
Dude I did not start with

> Aleph Alpha is just a sad joke by now.

And this isn't ad-hominem for lack of factual critic. The personal stuff _is_ the reason. How else do you call out someone driven by ego without mentioning them and their ego?

The guy is only ego. There is no substance to attack.

gchamonlive 10 hours ago||
> And who are you, exactly?

this is, because it can mean so much stuff that you can't really tell you are calling out their "ego" or just trolling.

Until you can at least open yourself to the possibility that there are better more inviting ways to talk to people online, you'd think this is just the natural way people talk online, but I'd say it's not. It's just an aesthetic choice.

hypfer 9 hours ago||
We're not going to discuss this.

If you truly think that I am the problem deep down in this nested subthread that violates the spirit and letter of the rules of the platform, I have zero faith in the calibration of your perception.

gchamonlive 9 hours ago||
Good, leave faith for God, question everything, but there is room there for improving your communication whether you are ready for this conversation or not
solenoid0937 10 hours ago|||
In 5 years we'll have superintelligence, and thinking on 20 year timelines is simply absurd.
gchamonlive 10 hours ago||
[dead]
smokel 7 hours ago|||
Would you mind substantiating these claims a bit?
rjzzleep 11 hours ago||
It's not that talent isn't there anymore. It's that retards are at the helm everywhere and therefore talent is running away the first chance they have.
sajithdilshan 12 hours ago||
> It knows less from memory, Multi-turn tool calling is weaker, It’s not the best coding agent

Then what does it good at? Sending faxes?

mhitza 12 hours ago||
I don't have the hardware to test this, but being trained to say "I don't know" instead of misleadingly talking confidently about what it does not know is interesting in itself.
pettijohn 11 hours ago|||
Seems like it's good at reading German documents and reasoning over them. Custom German-language tokenizer, and one of the least hallucinating models.
hasley 7 hours ago|||
I suspect there are a lot of German companies that have thousands of documents that describe their software/product. Adding or changing a feature means you have to ask 20 people whether there might something that can break some edge case behavior for which no (automatic) test exists. In this case, it would be nice to have a virtual employee who knows the content of all documents.

I even experienced the opposite case: Some feature was first rejected since everyone agreed that it would break some behavior. After talking to several people again, I discovered that the use case which required this behavior did not exist anymore and had been officially phased out years before. Maybe, some RAG LLM could have told me, based on company documents, that there is some edge case that can be removed making the way free for the new feature.

skrebbel 12 hours ago|||
Well that’s a pretty important skill for German users!
sajithdilshan 11 hours ago||
Unfortunately yes.
moooo99 11 hours ago||
I know this is a long standing joke, but I am 27 now, born and raised in Germany, and genuinely never had the opportunity to use a Fax machine. It honestly kind of bums me out because there is something that seems cool about Fax (being fully aware that it should be fully obsolete by now)
jfengel 9 hours ago|||
Fax was always awkward and weird. Resolution sucked and the paper handling was unreliable.

They were never common. They were mostly business tools, and consumers never wanted them. Sometimes you'd be required to send someone a fax, and you'd have to go to an office store. (And pay rather a lot; dollars per page.)

I hope you get to give one a try some day, for kicks, but the thrill will wear off fast.

ketzu 7 hours ago||
> They were never common.

Are you sure about that? I remember we had one at home when I grew up, so I assumed they were very common at the time. (And it's not like we had this crazy business driven family use of it.)

jfengel 5 hours ago||
About the best data point I can cite is that I never saw one in a TV show.I don't think the Huxtables had a home fax machine. (Though maybe I just watched the wrong programs, and we're talking about a very long time ago so memory could fail.)
tchalla 11 hours ago||||
You have plenty of opportunities you’ve just not been forced to use them. Look into your contracts right now, I bet rhrrr are opportunities
IshKebab 11 hours ago|||
Fax was never cool.
hobo123 10 hours ago|||
As a kid I interned at a bookstore, and before closing we'd write the book orders on a sheet of paper, slide it into the Fax machine to send to the book wholesale. I think the Fax printed out some kind of response a minute later.

Next day, the books would arrive. This was maybe 1996, but I thought it was pretty cool.

literalAardvark 9 hours ago|||
Fax was mind blowingly cool.

Maybe you can relate to seeing webcams or Facetime when it came out or something.

Being able to see a signed document or a picture someone took 5000km away quickly was mind boggling in its age.

stogot 12 hours ago|||
GPT was better at chat and rag before coding
m00dy 11 hours ago|||
I was thinking about the use cases for Germany residents actually, but yes, you're right. A German AI model needs to be able to handle fax machines (there's a bit variety of them) and also needs to handle letters coming home. german image-to-text.
gunthercuckolso 11 hours ago||
[flagged]
hypfer 11 hours ago||
The ignorant, hostile, negative, and, frankly, kinda racist comments here really are just a sad showing for the currently online crowd.

But anyway. I think the main oversight when dismissing this is that not every use-case is coding a SV-style startup app. That market is quite saturated, so it would make sense to create something locally for the use-cases currently underserved by LLMs.

We will probably learn more about what this can really do once quants become available that can be run by people without an SV salary (and the biases that come with that).

gilfoyle_7 9 hours ago||
does it have GDPR compliance?
shevy-java 10 hours ago||
Is Germany sovereign? It outsourced its defence onto the USA. Recently Trump wanted more diesel; Germany insta-submitted, also because oddly enough Macron submitted before Germany (Macron is suspicious). Before that, Leyen committed to insta-submission with a deal that made europeans poorer (and perhaps Leyen benefits from that). Canada shows the way. Many of the smaller countries in the EU too, such as Netherlands, Denmark, Finland, to some extent Sweden as well. Every time I read "sovereign" here I have to object. Nothing is sovereign here. The whole hardware is definitely not sovereign. Perhaps some of the software is, but that's about it. Plus, who gets all the data? The big US mega-corporations sniff non-stop. Remember how Facebook sniffed Libgen and Anna's Archive dry etc..., then suddenly libgen went down. The US corporations act as huge global leeches on every step of the stair. And lobbyists benefit from this too.
More comments...