Top
Best
New

Posted by todsacerdoti 1 day ago

Zig's New Async I/O(andrewkelley.me)
https://www.youtube.com/watch?v=mdOxIc0HM04
147 points | 39 comments
synergy20 1 day ago|
have not played with zig for a while, remain in the C world.

with the cleanup attribute(a cheap "defer" for C), and the sanitizers, static analysis tools, memory tagging extension(MTE) for memory safety at hardware level, etc, and a zig 1.0 still probably years away, what's the strong selling point that I need spend time with zig these days? Asking because I'm unsure if I should re-try it.

pron 8 minutes ago||
I don't think any language that's not well-established, let alone one that isn't stabilised yet, would have a strong selling point if what you're looking for right now is to write production code that you'll maintain for a decade or more to come (I mean, companies do use languages that aren't as established as C in production apps, including Zig, but that's certainly not for everyone).

But if you're open to learning languages for tinkering/education purposes, I would say that Zig has several significant "intrinsic" advantages compared to C.

* It's much more expressive (it's at least as expressive as C++), while still being a very simple language (you can learn it fully in a few days).

* Its cross-compilation tooling is something of a marvel.

* It offers not only spatial memory safety, but protection from other kinds of undefined behaviour such as tagged unions.

flohofwoe 1 day ago||
My 2ct: A much more useful stdlib (that's also currently the most unstable part though, and I don't agree with all the design decisions in the stdlib - but mostly still better than a nearly useless stdlib like C provides - and don't even get me started about the C++ stdlib heh), integrated build system and package manager (even great for pure C/C++ projects), and comptime with all the things it enables (like straightforward reflection and generics).

It also fixes a shitton of tiny design warts that we've become accustomed to in C (also very slowly happening in the C standard, but that will take decades while Zig has those fixes now).

Also, probably the best integration with C code you can get outside of C++. E.g. hybrid C/Zig projects is a regular use case and has close to no friction.

C won't go away for me, but tinkering with Zig is just more fun :)

lukaslalinsky 12 hours ago|||
I really like Zig as a language, but I think the standard library is a huge weakness. I find the design extremely inconsistent. It feels like a collection of small packages, not a coherent unit. And I personally think this std.Io interface is on a similar path. The idea of abstracting our all I/O calls is great, but the actual interface is sketchy, in my opinion.
cassepipe 16 hours ago|||
Could you expand on what are the design decisions you disapprove of ? You got me curious
flohofwoe 11 hours ago||
Many parts of the stdlib don't separate between public interface and internal implementation details. It's possible to accidentially mess with data items which are clearly meant to be implementation-private.

I think this is because many parts of the stdlib are too object-oriented, for instance the containers are more or less C++ style objects, but this style of programming really needs RAII and restricted visibility rules like public/private (now Zig shouldn't get those, but IMHO the stdlib shouldn't pretend that those language features exist).

As a sister comment says, Zig is a great programming language, but the stdlib needs some sort of basic and consistent design philosophy whitch matches the language capabilities.

Tbf though, C gets around this problem by simply not providing a useful stdlib and delegating all the tricky design questions to library authors ;)

smj-edison 8 hours ago||
Honestly getting to mess with internal implementation details is my favorite part of using Zig's standard library. I'm working on a multithreaded interpreter right now, where each object has a unique index, so I need effectively a 4GB array to store them all. It's generally considered rude to allocate that all at once, so I'm using 4GB of virtual memory. I can literally just swap out std.MultiArrayList's backing array with vmem, set its capacity, and use all of its features, except now with virtual memory[1].

[1] https://github.com/smj-edison/zicl/blob/bacb08153305d5ba97fc...

flohofwoe 8 hours ago||
I would at least like to see some sort of naming convention for implementation-private properties, maybe like:

    const Bla = struct {
        // public access intended
        bla: i32,
        blub: i32,
        // here be dragons
        _private: struct {
            x: i32,
            y: i32,
        },
    };
...that way you can still access and mess up those 'private' items, but at least it's clear now which of the struct items are part of the 'public API contract' and which are considered internal implementation details which may change on a whim.
KallDrexx 12 minutes ago|||
This is my main frustration with the push back against visibility modifiers. It's treated as an all or nothing approach, that any support for visibility modifiers locks anyone out from touching those fields.

It could just be a compiler error/warning that has to be explicitly opted into to touch those fields. This allows you to say "I know this is normally a footgun to modify these fields, and I might be validaing an invariant condition, but I am know what I'm doing".

smj-edison 1 hour ago|||
Aah, that's a good point. I suppose you could argue that the doc notes indicate whether something it meant to be accessed directly, like when ArrayList mentions "This field is intended to be accessed directly.", but that's really hard to determine at a glance.
sanity 1 day ago||
Andrew Kelley is one of my all-time favorite technical speakers, and zig is packed full of great ideas. He also seems to be a great model of an open source project leader.
throwawaymaths 20 minutes ago|
I would say zig is not just packed full of great ideas, there's a whole graveyard of ideas that were thrown out (and a universe of ideas that were rejected out of hand). Zig is full of great ideas that compose well together.
ancarda 1 day ago||
I really hope this is going to be entirely optional, but I know realistically it just won't be. If Rust is any example, a language that has optional async support, async will permeate into the whole ecosystem. That's to be expected with colored functions. The stdlib isn't too bad but last time I checked a lot of crates.io is filled with async functions for stuff that doesn't actually block.

Async clearly works for many people, I do fully understand people who can't get their heads around threads and prefer async. It's wonderful that there's a pattern people can use to be productive!

For whatever reason, async just doesn't work for me. I don't feel comfortable using it and at this point I've been trying on and off for probably 10+ years now. Maybe it's never going to happen. I'm much more comfortable with threads, mutex locks, channels, Erlang style concurrency, nurseries -- literally ANYTHING but async. All of those are very understandable to me and I've built production systems with all of those.

I hope when Zig reaches 1.0 I'll be able to use it. I started learning it earlier this month and it's been really enjoyable to use.

lukaslalinsky 12 hours ago||
It's optional to the point that you can write single-threaded version without any io.async/io.concurrent, but you will need to pass the io parameter around, if you want to do I/O. You are mistaking what is called "async" here for what other languages call async/await. It's a very different concept. Async in this context means just "spawn this function in a background, but if you can't, just run it right now".
eddd-ddde 1 day ago|||
> I do fully understand people who can't get their heads around threads and prefer async.

Those are independent of each other. You can have async with and without threads. You can have threads with and without async.

rowanG077 19 hours ago||
Yeah, it always mystifies me when people talking about async vs threads when they are completely orthogonal concepts. It doesn't give me the feeling they understand what they are talking about.
flohofwoe 1 day ago|||
> I'm much more comfortable with threads

The example code shown in the first few minutes of the video is actually using regular OS threads for running the async code ;)

The whole thing is quite similar to the Zig allocator philosophy. Just like an application already picks a root allocator to pass down into libraries, it now also picks an IO implementation and passes it down. A library in turn doesn't care about how async is implemented by the IO system, it just calls into the IO implementation it got handed from the application.

throwawaymaths 18 minutes ago||
or you can pick one manually. Or, you can pick more than one and use as needed in different parts of your application. (probably less of a thing for IO than allocator).
audunw 21 hours ago|||
You don’t have to hope. Avoiding function colours and being able to write libraries that are agnostic to whether the IO is async or not is one of the top priorities of this new IO implementation.

If you don’t want to use async/await just don’t call functions through io.async.

pkulak 18 hours ago|||
> can't get their heads around threads and prefer async

Wow. Do you expect anyone to continue reading after a comment like that?

hugs 23 hours ago|||
same. i don't like async. i don't like having to prepend "await" to every line of code. instead, lately (in js), i've been playing more with worker threads, message passing, and the "atomics" api. i get the benefits of concurrency without the extra async/await-everywhere baggage.
iknowstuff 23 hours ago||
lol it’s just a very different tradeoff. Especially in js those approaches are far more of a “baggage” than async/await
hugs 22 hours ago||
probably, but i'm petty like that. i just really don't like async/await. i'm looking forward to eventually being punished for that opinion!
duped 22 hours ago|||
It sounds like your trouble with async is mistaking concurrency for parallelism.
Maxatar 21 hours ago||
Weird claim since threads were originally introduced as a concurrency primitive, basically a way to make user facing programs more responsive while sharing the same address space and CPU.

The idea of generalizing threads for use in parallel computing/SMP didn't come until at least a decade after the introduction of threads for use as a concurrency tool.

metaltyphoon 18 hours ago||
> Weird claim since threads were originally introduced as a concurrency primitive

Wasn't this only true when CPUs were single core only? Only when multi core CPUs came, true parallelism could happen (outside of using multiple CPUs)

speed_spread 1 day ago|||
I agree, async is way more popular than it should be. At least (AFAIU) Zig doesn't color functions so there won't be a big ecosystem rift between blocking and async libraries.
01HNNWZ0MV43FF 23 hours ago|||
I understand threads but I like using async for certain things.

If I had a web service using threads, would I map each request to one thread in a thread pool? It seems like a waste of OS resources when the IO multiplexing can be done without OS threads.

> last time I checked a lot of crates.io is filled with async functions for stuff that doesn't actually block.

Like what? Even file I/O blocks for large files on slow devices, so something like async tarball handling has a use case.

It's best to write in the sans-IO style and then your threading or async can be a thin layer on top that drives a dumb state machine. But in practice I find that passable sans-IO code is harder to write than passable async. It makes a lot of sense for a deep indirect dependency like an HTTP library, but less sense for an app

tomck 23 hours ago||
> I do fully understand people who can't get their heads around threads and prefer async

This is a bizarre remark

Async/await isn't "for when you can't get your head around threads", it's a completely orthogonal concept

Case in point: javascript has async/await, but everything is singlethreaded, there is no parallelism

Async/await is basically just coroutines/generators underneath.

Phrasing async as 'for people who can't get their heads around threads' makes it sound like you're just insecure that you never learned how async works yet, and instead of just sitting down + learning it you would rather compensate

Async is probably a more complex model than threads/fibers for expressing concurrency. It's fine to say that, it's fine to not have learned it if that works for you, but it's silly to put one above the other as if understanding threads makes async/await irrelevant

> The stdlib isn't too bad but last time I checked a lot of crates.io is filled with async functions for stuff that doesn't actually block

Can you provide an example? I haven't found that to be the case last time I used rust, but I don't use rust a great deal anymore

dang 35 minutes ago|||
> This is a bizarre remark

> makes it sound like you're just insecure

> instead of just sitting down + learning it you would rather compensate

Can you please edit out swipes like these from your HN posts? This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html.

Your comment would be fine without those bits.

ksec 21 hours ago||||
>Case in point: javascript has async/await, but everything is singlethreaded, there is no parallelism, Async/await is basically just coroutines/generators underneath.

May be I just wish Zig dont call it async and use a different name.

vjerancrnjak 23 hours ago|||
It's more about how the code ends up evolving.

Async-await in JS is sometimes used to swallow exceptions. It's very often used to do 1 thing at a time when N things could be done instead. It serializes the execution a lot when it could be concurrent.

    if (await is_something_true()) {
      // here is_something_true() can be false
    }
And above, the most common mistake.

Similar side-effects happen in other languages that have async-await sugar.

It smells as bad as the Zig file interface with intermediate buffers reading/writing to OS buffers until everything is a buffer 10 steps below.

It's fun for small programs but you really have to be very strict to not have it go wrong (performance, correctness).

tomck 21 hours ago||
I think you replied to the wrong person.

That being said, I don't understand your `is_something_true` example.

> It's very often used to do 1 thing at a time when N things could be done instead

That's true, but I don't think e.g. fibres fare any better here. I would say that expressing that type of parallel execution is much more convenient with async/await and Promise.all() or whatever alternative, compared to e.g. raw promises or fibres.

gethly 1 day ago||
I almost fell out of my chair at 22:06 from laughter :D
yohbho 31 minutes ago|
due to broken audio?
AndyKelley 23 hours ago||
text version: https://andrewkelley.me/post/zig-new-async-io-text-version.h...
dang 38 minutes ago|
Thanks - we'll switch the URL to that above and re-up the thread.
pragmatic 18 hours ago|
https://andrewkelley.me/post/zig-new-async-io-text-version.h...

Text version.

The desynced video makes the video a bit painful to watch.

dang 37 minutes ago|
Yup, we'll use the text version and keep the video link in the toptext.