Top
Best
New

Posted by olexsmir 19 hours ago

A new Go API for JSON(go.dev)
344 points | 175 commentspage 2
hoppp 14 hours ago|
I like it. I never really liked marshalling anyways, It was weird.

I'm happy it's getting fixed!

troupo 17 hours ago||
> in any, out any

Completely unrelated: can anyone tell me why language authors introduce in/out/inout function parameters (see also C#).

It feels like this is some rather bad workaround for something that breaks assumptions about parameters being passed into the function.

hxtk 16 hours ago||
In Go, it’s a question of who owns the allocation. If you return a pointer type allocated within the function, the answer is “no one,” so the pointer escapes and has to be managed by the garbage collector.

If the allocation is owned by the calling function which uses it internally and then doesn’t return it, it can be a stack variable. Or the same allocation can be reused many times in a loop. Or an array of allocations can be made all at once with foreknowledge of the total number.

throwaway894345 9 hours ago||
If you pass a pointer into a function, the Go compiler can (and sometimes does) allocate it on the heap. Go has no allocation/ownership semantics, tragically, so the best we can do is depend on undefined behavior regarding the escape analyzer and hope the Go team doesn’t break it (which is a relatively safe assumption, considering that it’s unlikely that the Go team would start allocating things on the heap that were previously allocated on the stack—Go rarely regresses performance like this).
ahmedtd 15 hours ago|||
The parameters named "out" here are not "output parameters" in the way you seem to be thinking. They are "input parameters" named "out", because they are the buffer or pipe you want the data written to.
jayd16 15 hours ago|||
`out` params in C# are params that must be set within the function. They are way to implement multiple return values and tell the caller "this is written to, not read from."

But you might be conflating that with in/out for Contra/Co-variance when specifying a generic.

metaltyphoon 12 hours ago||
> They are way to implement multiple return values

They are one way to implement multiple return values. For multiple returns, tuples can be used. In fact, async methods can’t declare out params

duskwuff 16 hours ago|||
"in" and "out" are the names of the parameters, not qualifiers.
CharlieDigital 16 hours ago||
C# `out` is kinda nice because it can be `var`d and used directly.
troupo 15 hours ago||
Erm... what do you mean?
metaltyphoon 12 hours ago||

  if (int.TryParse(str, out var value) && value == 5)
  {
      // value is available 
  }

   // value also available
wiseowise 17 hours ago||
This is how you roll out features, C++. Congrats to Go team and users.
SkiFire13 16 hours ago||
> nil slices and maps are marshaled as an empty JSON array or object by default

I feel that is the kind of broken default that will come back and bite a bunch of people, and will eventually need to be fixed in a v3.

travisd 16 hours ago||
I think it'll bite some people, but probably is the better default for JSON specifically. Most of the nil-maps-become-null thing is just an extension of Go defaulting to nil for empty arrays and maps, which is usually not what you expect when you're writing a JSON schema like

    type ListUsersResponse struct {
        Users []User `json:"users"`
    }
in Go. It's pretty common have code like

    userRows := db.Query(...)
    var users []api.User
    for _, user := range userRows {
        users = append(users, api.User{...})
    }
where you don't _really_ intend for the default nil-ness to leak out. (Apologies for any broken syntax, I'm realizing in real time that it's been a while since I've written Go)
SkiFire13 7 hours ago||
> Most of the nil-maps-become-null thing is just an extension of Go defaulting to nil for empty arrays and maps, which is usually not what you expect when you're writing a JSON schema

This is definitely not what I expect in other language. I would argue this is a quick of Go though and not JSON's fault.

devmor 16 hours ago|||
Can you explain why? This seems like the appropriate expectation to me, given how JSON is constructed and consumed.
Someone 6 hours ago||
Not the OP, but serializing both nil and empty arrays to the same json string means round tripping will not work for all objects.

That seems like a bad default to me.

throwaway894345 9 hours ago|||
How is this a broken default? In Go, I almost always want a nil slice to be treated exactly as an empty slice, and JSONv1 was the ~only place this didn’t work as expected, leading to frequent bugs.
the_gipsy 5 hours ago|||
The root cause is this idea of "zero values". Any de/serialization in go will always be crutchwork of guessing and incorrect validation.

Just look at the inane rules for map and slice access, what gets nil, what gets the zero value, which built-in mist be used. Even if that were consistent, then you'd still left to wonder if json.field was truly zero or not there.

hujun236 10 hours ago||
[flagged]
gosen 15 hours ago||
wohoo, good news indeed better later than never :D as an active Golong user I'm very happy
lolpanda 19 hours ago||
the marshal api has been very annoying. why does marshal return an error? the only reason the marshal can return an error is that the input type is not marshalable. it's certainly a code bug. to me it's like int+str returning an error instead of compiler error
duskwuff 18 hours ago||
> why does marshal return an error? the only reason the marshal can return an error is that the input type is not marshalable

Not necessarily. Among other things, MarshalJSON methods called during marshaling can return an error for whatever reason the developer sees fit - one possibility that comes to mind is that part of the object tree being marshalled is lazily loaded, and failed to do so.

And, as others have mentioned, there are types like "any" whose marshalability can't be determined at compile time. Those types aren't necessarily at the top level, either; they could appear anywhere in the object tree.

sgsjchs 18 hours ago|||
Types may be mostly marshallabble, have a subset of values that are unmarshallable. E.g. non-finite floats.
hnlmorg 18 hours ago|||
I wouldn’t class unmarshallable data as a compile time error.

Plus the point of the marshal interface is that it’s reusable.

thayne 18 hours ago||
It could be if go had a more powerful type system.
hnlmorg 18 hours ago||
This isn’t a type system problem. It’s a data origin problem. Ie it could be any random JSON blob a user entered. The compiler doesn’t know this ahead of time.

If you wanted type safety with JSON in Go then don’t use the marshal interface. Period.

thayne 17 hours ago||
For unmarshalling, but the original complaint was about marshalling. And for that the type system could help if the marshal API only allowed you to marshal objects of types that supported marshalling.

And even for unmarshalling you could get a compile time error if you try to unmarshall to a type that doesn't support it.

hnlmorg 17 hours ago|||
I’d already addressed that when I said the marshal API is an interface that’s meant to be reusable across generalise use cases. it can be used for a variety of data formats and not just JSON.

Go has this concept baked into the language. For example read and write interfaces so you can write data to an endpoint without having to worry about whether it’s a file system object, TCP/UDP socket, gzip achieve, and a plethora of other options too.

It’s a nice convenience feature and makes up considerably for the restrictions that the Go type system places.

And if you need higher performance code, or compile time safety, or have any other constraints that limit your ability to use these generalised interfaces, then you can use different APIs from the Go standard library to achieve the same thing.

You don’t need to call json.Marshal() if you don’t want to. It’s an option, but it’s just one API into the JSON toolbox.

thayne 16 hours ago||
First of all, the json2 Marshall API is json specific.

Secondly, other languages like rust, haskell, scala, etc. Have json APIs that are typesafe and give you compile time errors if you try to use a type that doesn't support (de) serialization . In the case of rust at least, the API is also general and can be used for other data types as well.

Even in go, you could have some more type safety if Marshall took an interface (such as MarshallerTo) rather than `any`. Although then you have the problem of how you implement that interface. So I guess you probably want some kind of "derive" functionality, or macros that can approximate it for a convenient and type-safe API.

P.S. gophers like to talk about how explicit being better than implicit, but using reflection for marshalling is very implicit.

hnlmorg 6 hours ago||
Except it’s not JSON specific. The article literally talks about how their function design was chosen for similarity with the v1 API. And I’d wager that the go team are planning on reusing that function signature for other encoders too. Or at least expecting it to be reused by the wider ecosystem.

If you want type safety then pass a struct to the marshal. Which is actually the idiomatic way to do it anyway. The type safety would be in that.

I’m not going to disagree with you that Gos type system is a rather blunt tool nor that Gos expressiveness is extremely rigid compared to many other popular languages. All of that is completely true. But I don’t agree that the v2 JSON API is a good example of that problem.

awesome_dude 13 hours ago|||
I'm confused by this - you're saying that a "type system" would address this, even though there are two (incompatible) type systems at play here - Go's and JSON's

Compile time cannot check for runtime values (not even in Rust) - is your position that fields can only be populated at compile time?

thayne 9 hours ago||
In go, the Marshall and unmarshal functions take `any` for the source or destination. If you use a value of a type that doesn't support marshaling or unmarshalling (say because it contains an unserializable type like a channel, function pointer, etc) then you get a runtime error. Whereas in rust, Haskell, scala, etc. you can get a compile time error if you try to use a type that doesn't support serialization.
hnlmorg 6 hours ago||
You don’t get runtime errors for marshalling pointers to JSON. I actually do this all the time. What you get is the address of the pointer marshalled.

I think the same is true for channels too but I’m not near a computer to check.

I’ve abused the v1 API heavily* and haven’t found anything it couldn’t marshal. Though I’m yet to apply the same “fuzzing” (for lack of a better term) to v2.

* the v1 JSON marshaller underpins some core functionality in my alternative $SHELL. So I’ve thrown a lot of exotic type structures (by Gos standards at least) at it.

bicarbonato 17 hours ago|||
AFAIK it's mostly because of floats. JSON doesn't support `NaN` and `infinity` for example.

Because they are common types, makes sense to precautionary have the error return.

0x696C6961 18 hours ago|||
Then write yourself a MustMarshal wrapper that panics.
jeremyloy_wt 18 hours ago|||
A generic Must[T] you mean ;)
hnlmorg 17 hours ago|||
The GP is talking about compile time errors. You’re discussing exceptions, which is still a runtime check
0x696C6961 1 hour ago||
Their complaint is about the signature. The comment about the compile time checking was a misguided justification for their position.
leetrout 18 hours ago|||
JSON is a runtime input of unknown type but in your example those are known types so the compiler can catch them.
lolpanda 18 hours ago|||
i meant i hoped it's panic as opposed to return an error. if there was an error, how am i supposed to handle it? can i tell the user hey sorry my bad it's likely a code error? crash is better than possible eating the error siliently
iaaan 18 hours ago|||
It is trivial to ignore an error in Go if you know your use case cannot result in one. But at the end of the day, json.Marshal takes an `any` as a parameter, so it must return an error.
NoDodgeQuestion 18 hours ago||
do not know at compile time that input is not marshalable, every language marshal must return error for this reason, not only golang
slopnt 16 hours ago||
Now guess what will happen when they add monadic error handling. Yes, correct, there will be another JSON Go API
rednafi 2 hours ago|
json/v3 ...so what? The v* idea in the stdlib allows things to move when needed.
muddi900 5 hours ago||
"There only ONE way to do each thing! Only ONE! No exception"

"What if it sucks?"

"There are now TWO ways to do each thing! No more than TWO! I promise!"

It's like python again.

Mawr 4 hours ago|
Sure, if you ignore the massive difference in the rate such changes are happening at and that this new api is strictly better than the previous one, so there's no two ways that need to be decided between but rather the new go-to and the old-to-be-moved-away-from option, so there's no cognitive load here at all.

So yeah, if you ignore most of what matters, everything is the same.

videotextly 6 hours ago||
[flagged]
quietfeeds 11 hours ago|
[flagged]
More comments...