Top
Best
New

Posted by ryanhn 1 day ago

What if writing tests was a joyful experience? (2023)(blog.janestreet.com)
104 points | 44 commentspage 2
TacticalCoder 1 day ago|
Amazing to see Jane Street uses Emacs. And property-based testing too.

> you don’t just get a build failure telling you that you want 610 instead of a blank string

So I had to scratch my head a bit because I was thinking: "Wait, the whole point is that you don't know whether what you're testing is correct or not, so how can you rely on that as input to your tests!?".

But even though I didn't understand everything they do yet I do see at least a big case where it makes lots of sense. And it happens to be a case where a lot of people see the benefits of test: before refactoring.

> What does fibonacci(15) equal? If you already know, terrific—but what are you meant to do if you don’t?

Yeah a common one is reuse a function in the same language which you believe is correct (you probably haven't proven it to be correct). Another typical one is you reuse a similar function from another language (once again, it's probably not been proven it is correct). But if two implementation differ, you know you have an issue.

> let d = create_marketdata_processor () in > ( Do some preprocessing to define the symbol with id=1 as "APPL" )

Typo. It's AAPL, not APPL. It's correctly used as AAPL later on.

FWIW writing tests better become a joyful experience for we're going to need a lot* of these with all our AI generated code.

zem 1 day ago|
> And it happens to be a case where a lot of people see the benefits of test: before refactoring.

it's also very nice if you have a test-last working style, that is, develop the code first using some sort of ad hoc testing method, then when you're convinced it's working you add tests both as a final check that the output is what you expect across a lot of different corner cases, and to prevent regressions as you continue development.

breatheoften 1 day ago||
I really like this style of testing -- code that can be tested this way is also the most fun kind of code to work with and the most likely to behave predictably.

I love determinism and plain old data.

Joel_Mckay 1 day ago|
Could look at high-level constraint modelling languages:

https://www.minizinc.org/

It often bypasses the need to get bogged down in probabilistic markdown syntax =3

https://www.youtube.com/watch?v=X6WHBO_Qc-Q

o_nate 1 day ago||
This is a cool idea. I wish something like this existed for C#.
Smaug123 21 hours ago||
The thing that most surprises me is that IDEs don't have a standard protocol for this, so you basically need a custom test runner if you want one-click "this snapshot failed; update it" self-modifying tests.

I wrote WoofWare.Expect for F#, which has an "update my snapshots on disk" mode, but you can't go straight from test failure to snapshot update without a fresh test run, even though I'm literally outputting a patience diff that an IDE could apply if it knew how.

Worse, e.g. Rider is really bad at knowing when files have changed underneath it, so you have to manually tell it to reload the files after running the update or else you clobber them in the editor.

RhysU 18 hours ago||
> ...if you want one-click "this snapshot failed; update it" self-modifying tests.

I am envisioning the PR arguments now when the first instinct of the junior developer is to clobber the prior gold standard outputs. Especially lovely when testing floating point functionality using tests with tolerances.

Some things should be hatefully slow so one's brain has sufficient chance to subconsciously mull over "what if I am wrong?"

legulere 16 hours ago|||
But there is: https://www.meziantou.net/inline-snapshot-testing-in-dotnet....
PretzelPirate 1 day ago||
An Agentic coding tool like Github Copilot will do this for you.
i_don_t_know 15 hours ago||
mdx[1] is another variation on this, also in the Ocaml ecosystem. It’s Ocaml’s version of documentation tests as in Elixir and Rust.

But it’s not limited to that. You can write tests in markdown files independently from your documentation. Use “dune test” to run the tests and review failures with “git diff”. Accept the changes if they are correct (changed behavior) with “dune promote”. Very nice workflow.

[1] https://github.com/realworldocaml/mdx

9rx 13 hours ago||
Since when has writing tests not been a joyful experience?

I do see a lot of useless tests out in the wild. I can see writing those not bringing any joy. That is true of any useless activity. Is that what we're thinking of here?

3vidence 1 day ago|
In my experience the lack of joy or difficulty with tests is almost always that the test environment is usually different enough from the real environment that you end up needing to kind of stretch your code to fit into the test env instead of actually testing what you are interested in.

This doesn't apply to very simple functions but tests on simple functions are the least interesting/ valuable.