Test code and production code in a symmetrical pair has lots of benefits. It’s a bit like double entry accounting - you can view the code’s behavior through a lens of the code itself, or the code that proves it does what it seems to do.

You can change the code by changing either tests or production code, and letting the other follow.

Code reviews are a breeze because if you’re confused by the production code, the test code often holds an explanation - and vice versa. So just switch from one to the other as needed.

Lots of benefits. The downside is how much extra code you end up with of course - up to you if the gains in readability make up for it.

Literate programming sounds great in a blog post, but it falls apart the moment an agent starts hallucinating between the prose and the actual implementation. We’re already struggling with docstrings getting out of sync; adding a layer of philosophical "intent" just gives the agent more room to confidently output garbage. If you need a wall of text to make an agent understand your repo, your abstractions are probably just bad. It feels like we're trying to fix a lack of structural clarity with more tokens.

the interesting twist with agents is that the "audience" for the code is now split — humans still need to read it, but so does the model generating/modifying it. literate programming was designed for the human reader; I wonder if there's a format that optimizes for both simultaneously, or if those constraints are actually at odds

We were taught Literate Programming and xtUML at university. In both courses, the lecturers (independently) tried to convince us that these technologies were the future. I also did an AI/ML course. That lecturer lamented that the golden era was in the past.

Even on the latest models, LLMs are not deterministic between "don't do this thing" and "do this thing". They are both related to "this thing" and depending on other content in the context and seed, may randomly do the thing or not. So to get the best results, I want my context to be the smallest possible truthful input, not the most elaborated. More is not better. I think good names on executable source code and tightest possible documentation is best for LLMs, and probably for people too.

I'd much much rather the model write the code blocks than the prose myself. In my experience LLM can produce pretty decent code, but the writing is horrible. If anything I would prefer an agentic tool where you don't even see the slop. I definitely would rather it not be committed.

I wrote something similar where you specify the intent in Markdown at the file level. That can also be done by an AI agent. Each intent file compiles to a source file.

It works, but needs improvement. Any feedback is welcome!

https://intentcode.dev

https://github.com/jfilby/intentcode

Not sure if the author know about CUE, here's the HN post from early this year on literate programming with CUE [1].

CUE is based of value-latticed logic that's LLM's NLP cousin but deterministic rather than stochastic [2].

LLMs are notoriously prone to generating syntactically valid but semantically broken configurations thus it should be used with CUE for improving literate programming for configs and guardrailing [3].

[1] CUE Does It All, But Can It Literate? (22 comments)

https://news.ycombinator.com/item?id=46588607

[2] The Logic of CUE:

https://cuelang.org/docs/concept/the-logic-of-cue/

[3] Guardrailing Intuition: Towards Reliable AI:

https://cue.dev/blog/guardrailing-intuition-towards-reliable...

I dont know Org, but Rakudoc https://docs.raku.org/language/pod is useful for literate programming (put the docs in the code source) and for LLM (the code is "self documenting" so that in the LLM inversion of control, the LLM can determine how to call the code).

https://podlite.org is this done in a language neutral way perl, JS/TS and raku for now.

Heres an example:

  #!/usr/bin/env raku
  =begin pod
  =head1 NAME
  Stats::Simple - Simple statistical utilities written in Raku

  =head1 SYNOPSIS
      use Stats::Simple;

      my @numbers = 10, 20, 30, 40;

      say mean(@numbers);     # 25
      say median(@numbers);   # 25

  =head1 DESCRIPTION
  This module provides a few simple statistical helper functions
  such as mean and median. It is meant as a small example showing
  how Rakudoc documentation can be embedded directly inside Raku
  source code.

  =end pod

  unit module Stats::Simple;

  =begin pod
  =head2 mean

      mean(@values --> Numeric)

  Returns the arithmetic mean (average) of a list of numeric values.

  =head3 Parameters
  =over 4
  =item @values
  A list of numeric values.

  =back

  =head3 Example
      say mean(1, 2, 3, 4);  # 2.5
  =end pod
  sub mean(*@values --> Numeric) is export {
      die "No values supplied" if @values.elems == 0;
      @values.sum / @values.elems;
  }

  =begin pod
  =head2 median

      median(@values --> Numeric)

  Returns the median value of a list of numbers.

  If the list length is even, the function returns the mean of
  the two middle values.

  =head3 Example
      say median(1, 5, 3);     # 3
      say median(1, 2, 3, 4);  # 2.5
  =end pod
  sub median(*@values --> Numeric) is export {
      die "No values supplied" if @values.elems == 0;

      my @sorted = @values.sort;
      my $n = @sorted.elems;

      return @sorted[$n div 2] if $n % 2;

      (@sorted[$n/2 - 1] + @sorted[$n/2]) / 2;
  }

  =begin pod
  =head1 AUTHOR
  Example written to demonstrate Rakudoc usage.

  =head1 LICENSE
  Public domain / example code.
  =end pod

This does seem exciting at first glance. Just write the narrative part of literate programming and an LLM generates the code, then keep the narrative and voila! Literate programming without the work of generating both.

However I see two major issues:

Narrative is meant to be consumed linearly. But code is consumed as a graph. We navigate from a symbol to its definition, or from definition to its uses, jumping from place to place in the code to understand it better. The narrative part of linear programming really only works for notebooks where the story being told is dominant and the code serves the story.

Second is that when I use an LLM to write code, the changes I describe usually require modifying several files at once. Where does this “narrative” go relative to the code.

And yes, these two issues are closely related.