Top
Best
New

Posted by ravenical 12 hours ago

Ki Editor - an editor that operates on the AST(ki-editor.org)
337 points | 119 comments
scriptsmith 10 hours ago|
The "First-class syntactic selection" reminds me of my most used shortcut(s) in Jetbrains IDEs: the Expand / Shrink Selection.

  Ctrl + W
  Ctrl + Shift + W
https://www.jetbrains.com/help/idea/working-with-source-code...

It really changed my perspective on interacting with the 'text' of a file.

VS Code, Zed, etc. have similar operations, but in my experience they expand and shrink too coarsely.

jasonjmcghee 6 hours ago||
Mine are:

Cmd+Shift+V - Stacked clipboard, you can start typing to search or hit a number to choose what to paste (keeps everything you've copied/cut inside jetbrains for a while)

Cmd+Shift+E - Recent locations, you can start typing to search - shows little buffers of where you've been recently

Cmd+Shift+A - Action tab of the command palette - fuzzy search for any command (really the only shortcut you need, other than maybe Shift+Shift for main command palette shortcut)

--- Through the Action bar...

Local History / Local History of Selection - you can start typing to search quite far back the history of all changes of the current file or selection - you can also right click a folder or the project and do the same. Much finer grained than git.

The general concept of being able to search for something and edit directly in the buffer of the search results.

the__alchemist 6 hours ago||
Hero! I had not done my homework/have not been aware, but these all look fantastic! The stacked clipboard is something I periodically mentally complain about (Why is clipboard on every OS/tool I've used single item?)

I will add one that are possibly more well-known:

  - ctrl + shift + F: Find text in any file
  - ctrl + N: Find types (structs, classes etc)
  - ctrl + shift + N: Find any file by name or path
leoc 5 hours ago|||
Windows has had a pretty usable stacking clipboard for a while! You just have to activate it. Since you can pin thing into it it’s also quite useful as a rough and ready way to type special characters you use frequently.
the__alchemist 5 hours ago||
Wow. Looks to not even be a PowerToys feature. Win + V
jasonjmcghee 2 hours ago|||
I use shift+shift and type for all three of these.

Although i do use cmd+shift+r for global replace

enkursigilo 10 hours ago|||
Also available as `incremental selection` in Neovim via tree-sitter.
hudsonwillis 5 hours ago||
yep and most of these actions can be implemented with simple mappings based on https://github.com/nvim-treesitter/nvim-treesitter-textobjec...
hhhAndrew 6 hours ago|||
Mathematica is the earliest thing I am aware of with this feature where it was Alt+. to expand selection in their notebook interface starting in the early 90s. But the thing I miss most that I still can't shake the muscle memory of after almost a decade of not using much Mathematica, is that single/double/triple/n-click scaled this way as well. So double-click selected a whole word (as in all editors), triple-click selected all the comma-separated multiple args of a function, 4-click for f(a,r,g,s), and so on.
lejalv 3 hours ago||
Ctrl-Space in TeXmacs, where the document is an actual tree (http://texmacs.org)
cyberax 9 minutes ago|||
JetBrains also experimented with AST-based editing: https://www.jetbrains.com/help/mps/fast-track-to-mps.html#st...

An overview video: https://www.youtube.com/watch?v=XGm_khXZl44

I tried it, but it just was too clumsy. Sometimes refactoring/editing needs to go through phases where the AST is invalid, and MPS makes that just too clumsy.

But with AI this might be a different story.

gritzko 4 hours ago|||
I work on AST based revision control. I have a stack of ideas on how to achieve the same Ctrl+W effect with commits/diffs/cherry-picks. All still in flux. If you have some thoughts to share, please do.

[1]: https://github.com/gritzko/librdx/tree/master/be#readme

joquarky 1 hour ago|||
One problem is I got so used to Ctrl-W that I use it in other applications and usually wind up inadvertently closing the tab.
chamomeal 7 hours ago|||
I use it constantly in helix too. The vscode one is meh. I think I saw a discussion in github once about switching to tree-sitter, which would improve AST-related actions. I don't think it went anywhere though.

I love AST aware editing. I think it's one reason it's always been so nice to edit lisps. Stuff that is complicated to describe in javascript (and doesn't have LSP support) p much requires a whole AST parser, but in lisp it's just a simple list operation. When I go back typescript after a weekend of clojure, I reeaally miss slurp! and other paredit commands

mystifyingpoi 8 hours ago|||
I agree, top feature. Combined with things like "extract method" makes mundane refactorings super fast.
the__alchemist 5 hours ago|||
To me, it feels like Zed and VsCode perform most operations in a general way on the text; they don't seem to (in Python and Rust at least) have an understanding of the code structure in the way JB does. (And based on some digging on Ki the way it does as well?) So, I would bet they are using that text-based model, which would be hit/miss here.
layer8 7 hours ago|||
Yes, Java IDEs have had these since sometime in the 2000s.
exidex 6 hours ago|||
And I am sure that people have been complaining about the hand gymnastics you have to do to press those shortcuts since around that time as well
lkjdsklf 4 hours ago|||
It’s still some time in the 2000s and will be for the next 974 years
layer8 4 minutes ago||
Not according to common usage: https://en.wikipedia.org/wiki/2000s
upcoming-sesame 6 hours ago|||
Cool feature. what's the key for it in visual studio code?
lolpython 6 hours ago||
Expand Selection: Alt+Shift+→ (Windows/Linux) or Option+Shift+→ (Mac) Shrink Selection: Alt+Shift+← (Windows/Linux) or Option+Shift+← (Mac)
the__alchemist 6 hours ago|||
Ty! I have been missing out. Adding this to the repertoire.
parallax_error 9 hours ago|||
This has got to be my favourite feature of IntelliJ, along with the dumb context actions menu
hurflmurfl 6 hours ago||
Ah yes.

The shortcut I use the most in Jetbrains IDEs. Also the one I miss the most in VSCode (whatever is present there just doesn't seem to work right).

Also the shortcut that has caused me to close so many browser tabs inadvertently...

__MatrixMan__ 19 minutes ago||
I think the challenge with AST editing is discoverability. Like, I know what I want to select, I can see it there on the screen, but I don't know its name.

I've been dreaming of writing a plugin that surrounded the cursor in differently colored scopes. So instead of "next function" I'd be thinking "next blue" (blue being the color that functions are currently painted in).

arikrahman 6 minutes ago||
A great addition to the helix workflow, I knew something was missing from the movement -> action approach. This completes the trifecta.
radford-neal 5 hours ago||
Many years ago, I created an editor operating on syntax trees that I think is more "hard-core" than this - that is, only tree-oriented operations are done. There is no parsing of text, since entering plain text, rather than a tree, is impossible. Hence, there can be no syntactically invalid programs.

The challenge is getting this to be a useable way of entering programs. I think I made progress on this, but the feasibility varies with the programming language.

I can't run it any more, since the display hardware it assumed is no longer available, but you can read about it at https://ucalgary.scholaris.ca/items/da8b823b-c344-4ffb-aa37-...

Normal_gaussian 4 hours ago||
> The challenge is getting this to be a useable way of entering programs.

Well exactly.

When the path between Program A and Program B can only be valid programs, you are going to end up with either a much longer, less intuitive path, or deleting everything and starting again. It can also be quite possible to invent structures which are valid but have no valid path to creating them.

radford-neal 4 hours ago|||
Well, I think most common transformations work reasonably well. One usually doesn't want to do things completely contrary to the AST, such as convert "while a<b" to "whilea = b".

And it's certainly possible to create any valid AST in the editor I describe. The set of valid trees is extended to those with "holes" in places, which one fills in when entering a program, and it's always possible to do this.

The challenge is one of finding an intuitive user interface, not whether it's possible at all. One issue is that infix notation is unnatural for entering trees (prefix is more natural).

socalgal2 2 hours ago||
no syntax error editing seems like https://scratch.mit.edu/
12_throw_away 2 hours ago|||
> It can also be quite possible to invent structures which are valid but have no valid path to creating them.

I'm curious if you have an example of such a structure?

Pedantically: if, for every valid tree, there exists a bidirectional path to the empty root node, there's always at least one path between all given pairs of valid trees ... albeit one that no developer would ever take.

JNRowe 4 hours ago|||
If you want to relive it then simh¹ with mame² might be an option. There appears to be some support for VT11³, along with docs⁴ to use as a starting point.

No, I'm not claiming to have read all one hundred pages already. However, from what I have read I'd love to see a functional demo.

¹ https://simh.trailing-edge.com/

² https://www.mamedev.org/

³ https://github.com/simh/simh/blob/master/PDP11/pdp11_vt.c

⁴ https://wiki.mamedev.org/index.php/MAME_and_SIMH

conartist6 1 hour ago|||
Now you're out of the realm of Ki, but what you're talking about is still being worked on in the modern era, by me! I'm building BABLR which is a modern follow-up to your idea, built on top of a powerful, generic system of parsers which has gaps/holes but no error recovery, so that it works with trees which may be incomplete but which must not be invalid.

The hard part is that we need to be able to talk about these structures. Even just here on this forum we need to be able to communicate precisely about them. I often use · as a typesetting symbol so that I can easily write and read expressions like 2 + · which you would read as "two plus gap". The · symbol is only for typesetting as I say thought because you it's not safe to assume that any one character is reserved for our use in every programming language. Instead we wrap the parts that aren't syntactic in quotes and we use <//> as the symbol for a gap so that it looks more like this:

  <*> "2 + " <//> </>
The * there is a flag on the node, it means this node a leaf of the tree -- a token.

We can parse 2 + · into a proper tree now:

  <BinaryExpression>
    left: <*Number "2" />
    #: " "
    operator: <* "+" />
    #: " "
    right: <//>
  </>
And yes, BABLR can really parse this. If we've piqued your curiosity, our Discord server is currently the hub of our community and we'd love to see you. https://discord.gg/NfMNyYN6cX
exidex 4 hours ago|||
Pantograph[0] seems to be a more recent attempt to implement the same idea. It is still not a general editor but generalizing it to ranges of tree selections looks promising

[0]: https://pantographeditor.github.io/Pantograph/

armchairhacker 4 hours ago||
There are more examples in https://github.com/yairchu/awesome-structure-editors
jskdev 1 hour ago||
Wow! I stumbled onto your paper a while ago when I was looking into structural editing and thinking of a masters in CS, exploring editor interfaces / feedback loops.

(Maybe your paper is famous and it’s not wild that I read it, but it was wild to see after so many years)

I never took that path, spent time in tech industry confused why people didn’t seem interested in structural editing and better editing tools.

Out of curiosity, how do you think LLMs and genai affect the value of structural editors and similar tooling?

Part of me wants to stay disciplined— of course it’s valuable to work efficiently and work on the AST and with a repl. The other part of me gets paid to work on essentially a punch card system (building dev, ship to prod and see what happens)

sigbottle 8 hours ago||
I feel so illiterate when it comes to AST editing sometimes. I understand what an AST is from a computer science perspective. But I've never worked on a huge software refactor before that required direct AST textobject editing. Maybe an indication of my skill level...

The extent of my usage is having nice textobjects to easily interact with arglists and functions which aren't native to (neo)vim. Very cute and nice to just write "daf" somewhere in a function and just have it "just delete". Or hook it up with basic macros: search for regex, "daf".

I guess it's hard for me to edit things that I don't see right in front of me or aren't super simple changes (like name changes). Or at least, basic things I can reason about (such as finding by regex then deleting by textobject or something).

As for LSP's, I do use go to definition and rename all references, which is nice. But the huge structural refactoring part I have never really done. I don't really use many LSP features besides those two either...

Basically, I gotta up my editor game.

aragonite 6 hours ago||
> I guess it's hard for me to edit things that I don't see right in front of me or aren't super simple changes (like name changes). Or at least, basic things I can reason about (such as finding by regex then deleting by textobject or something).

This is actually what's nice about tools like ast-grep. The pattern language reads almost like the code itself so you can see the transformation right in front of you (at least for small-scale cases) and reason about it. TypeScript examples:

  # convert guard clauses to optional chaining
  ast-grep -pattern '$A && $A.$B' --rewrite '$A?.$B' -lang ts

  # convert self-assignment to nullish coalescing assignment
  ast-grep -pattern '$X = $X ?? $Y' --rewrite '$X ??= $Y' -l ts

  # convert arrow functions to function declarations (need separate patterns for async & for return-type-annotated though)
  ast-grep -pattern 'const $NAME = ($$$PARAMS) => { $$$BODY }' --rewrite 'function $NAME($$$PARAMS) { $$$BODY }' -l ts

  # convert indexOf checks to .includes()
  ast-grep -pattern '$A.indexOf($B) !== -1' --rewrite '$A.includes($B)' -l ts
The $X, $A etc. are metavariables that match any AST node and if the same metavariable appears twice (e.g. $X = $X ?? $Y), it requires both occurrences to bind to the same code so `x = x ?? y` will match but `x = y ?? z` won't. You can do way more sophisticated stuff via yaml rules but those are less visually intuitive.

Sadly coding agents are still pretty bad at writing ast-grep patterns probably due to sparse training data. Hopefully that improves. The tool itself is solid!

nine_k 5 hours ago|||
While at it, https://github.com/semgrep/semgrep was around for several years, too.
WickyNilliams 5 hours ago|||
Came here to recommend ast-grep too!

If your editor of choice supports an extension (vscode does for example) it's a very easy on-ramp for a better search/replace than regex offers. It's syntax-aware so you don't need care about whitespace, indentation etc. Very easy to dip your toes in where a regex would get complex fast, or require multiple passes.

I converted a codebase from commonjs to esm trivially with a few commands right after first installing it. Super useful.

I hope LLMs eventually start operating at this level rather than raw text. And likewise for them to leverage the language server to take advantage of built in refactorings etc

socalgal2 1 hour ago||
I don't know which editors support this but there's also, for lack of a better word, context aware grep. For example, search and replace foo with bar but only inside strings, or only inside comments, or only variables, or only methods, not class names (maybe ast-grep does this).
hou32hou 7 hours ago|||
TBH, it's actually not as hard as you think, most of the time, what I do is just select the whole syntax node and delete it, copy it, or replace it, and only 20% of the time would actually require deliberate understanding of how the AST is structured in the current language I'm coding in.
marssaxman 5 hours ago|||
Maybe, or maybe not. I think there are many different kinds of work all lumped together under the "software engineering" umbrella, and tools which are indispensable for some people can seem unimpressive or unnecessary to others. We're not all spending our time in the same way, working on the same kinds of tasks.

I don't find myself doing huge structural refactoring often enough that it would be worth my while to learn a specialized tool for doing it; I'm a quick typist, and it's easy enough to just blat through the changes by hand on the rare occasion such a thing is necessary. But I don't work on giant line-of-business apps or sprawling web services, and I can see someone who did a lot of that sort of work having a different take on the matter.

freedomben 7 hours ago||
I feel the same way. Learning Elixir and getting into macro writing was really helpful for me. Lisp is also the same way though for me the syntax of Elixir resonated a lot better for me. Many paths to the destination and all that, but figured I'd share in case it's helpful to someone
shashurup 10 hours ago||
In my classification of editors:

1. Orthodox. Mostly focused on looks and integrations.

2. Modal, Vim improvement. Focus on keeping basic Vim keybindings with minor improvements.

3. Modal, rethinking Vim approach.

Ki falls into the third category which I constantly monitor.

noosphr 10 hours ago||
4. All of the above.

Which is Emacs.

whobre 8 hours ago|||
Right, but we are discussing editors here.
noosphr 14 minutes ago|||
Evil mode is a perfectly reasonable text editor mode.
xpe 8 hours ago|||
Hold on. Operating systems and long-running esoteric history are always on topic here, right?

(This is spoken with something close to affection; I look fondly on my former Emacs days. I'm probably more likely than not to enjoy the company of a human who has at least tried Emacs and had some thoughtful reaction to it. I may not use it now, but the ideas live on in how I think and what I expect. Editors that push the limits, whatever those limits are, are part of the old school ethos I love about programming.)

shashurup 10 hours ago||||
I know, I use Emacs+Evil. I'm going to give Ki a try and, in case of success, to think about writing Ki binding for Emacs.
amusingimpala75 8 hours ago||
emacs already has structural editing packages like combobulate
ablob 10 hours ago||||
This is the first time I've heard about Emacs trying to look nice.
hou32hou 6 hours ago||||
"Emacs is an editor" is as true as "Toyota Hilux is a gattling gun carrier"
skydhash 10 hours ago|||
Vim is Emacs applied to Vi
gpvos 2 hours ago|||
Nah. NeoVim a tiny little bit, maybe.
umanwizard 5 hours ago|||
Not even close.
skydhash 4 hours ago||
How is it not?

Scripting language? Check!

Custom commands? Check!

Windows management? Check!

Build tools integration and error-based navigation? Check!

File manager? Check!

antiframe 53 minutes ago|||
I think the "Scripting language? Check!" is missing an important distinctions between Emacs and most other extensible editors. The proportion of the code editor features that are written in the configuration/extension language matters

Looking at the Emacs and NeoVim codebases in GitHub: Emacs is 74.6% ELisp or CommonLisp and NeoVim is 71.9% Lua or VimScript. Never mind, NeoVim is close!

But, can you at runtime modify the NeoVim core functions? Honest question because I've only used vim and VimScript was definately limited compared to elisp. If so, then I say NeoVim and Emacs are both highly-extensible editors.

johnisgood 3 hours ago|||
Operating system? Check or not? :D
skydhash 3 hours ago||
Vim has stopped before that particular threshold. But if ever Neovim get a GUI version, I believe the community will soon have its own Gnus, Eww, and EMMS.
gorjusborg 8 hours ago|||
Exactly the same for me. So far, there have been many challengers but the same champion (subjective, I know).
jshmrsn 9 hours ago|||
I am working on a modal code editor project that you might find interesting then. It also operates on an AST directly, which is represented as UI nodes which closely resemble normal text layout. Email in profile if you’d like to give it a try and possibly give early feedback (still in early development).
hwhshs 10 hours ago||
Vim improvement

So Ed Visual Mode Improved Improved!

ramon156 8 hours ago||
I didn't fully understand until I saw the examples

> First-class syntactic modification

> Notice the comma between the current and the next node is also deleted. > Notice how comma is added automatically.

This is awesome! And I bet it arguably requires less logic to do so as well. Cool stuff.

Now I'm wondering how much effort it would be to get a ki integration (or at least an AST-first rewrite) in Zed

evelant 2 hours ago|
I wrote the vscode extension for Ki. It would almost certainly be usable with Zed since it talks with the editor over a websocket. Check out the code in the Ki repo.
evelant 9 hours ago||
I made the vscode integration for this. I feel bad that I haven’t contributed much since, it’s a really cool project. IMO it’s important to try to innovate in the foundational tools of our craft (editors, languages, tooling, OS, etc) which Ki does.
Sammi 8 hours ago||
This one?

https://marketplace.visualstudio.com/items?itemName=ki-edito...

evelant 2 hours ago||
Yeah that’s it. Bundles the editor binary with the extension so you don’t have to install it separately.
herrherrmann 5 hours ago||
Thank you for that! I find it really daunting to try a new editor, but having a vs code extension as an entry point really helps me and increases the chances of trying it out soon!
muixoozie 8 hours ago||
I can't wait to try it. Love that it's keyboard layout agnostic. A lot of other good sounding ideas in the docs. Especially inspiration from Emacs as everything being an editable buffer. There's always some massive tradeoffs between editors though. Guess I'll have to see.
exidex 7 hours ago||
It's a shame that editors are build around the editing model, so much time has been wasted by recreating everything. Neovim is a perfect editor in many aspects but editing model is not one of them, in my opinion. Helix or even Ki would not be needed if Neovim allowed to completely replace the vim part of it with plugins
wizzwizz4 7 hours ago||
Can you elaborate on how this might look, both from an abstract perspective, and (if you've thought about it) from an API perspective?
exidex 7 hours ago||
I didn't really spent time thinking about it, because from reading their issue tracker, I very much doubt neovim would consider something like that. Partially because of how much of it is already vim specific and it would be very difficult to abstract that away, but mostly because of ideology.

You can already to a lot in Neovim with plugins, including recreating Helix, Ki, whatever. But the problem is that the vim state machine is always present underneath it, resulting in unwanted behaviors or impossibility to implement some things. I would envision it to pretty much allow to listen on raw keyboard and mouse events. However cursed it sounds, it should be possible to fully implement vscode or any other editing model in neovim

adrian_b 6 hours ago|||
> I would envision it to pretty much allow to listen on raw keyboard and mouse events.

This is pretty much how the BRIEF editor for programmers worked, which was used in MS-DOS around 1990. In my opinion this was one of the best programs that I have ever encountered and I have never encountered again some of the good UI features that existed in the BRIEF text editor and in the XTree file manager, both for MS-DOS.

BRIEF had a tiling-window user interface with optional menus and it came with a default behavior, but it could be customized extremely. For each kind of key press or mouse event you could bind a function to be executed, which replaced the default behavior. For instance, the default behavior for pressing most keys was to invoke a function "self_insert", which inserted in the current text buffer and the current cursor position the character corresponding to the pressed key, but it was easy to change the behavior to anything else, e.g. to do auto-completion or inserting some program template.

The functions bound to events were written in a custom language similar to LISP, so in this respect BRIEF resembled Emacs. A great number of text manipulation functions were provided, so it was easy to describe complex actions in the event handling functions.

This was what I liked most at BRIEF, that by rewriting all the event handling functions it was very easy to convert it in an editor that no longer resembled the default BRIEF editor at all, but which behaved in whatever way you believed that an editor for program sources should behave.

So the text editor provided an extensive library of functions for handling text, text buffers, windows and files and a LISP-like language for gluing all these into a text editor. It also had a default implementation of a text editor, to be able to use it immediately, but that could be replaced partially or totally, when you had more experience and you knew what kind of behavior you want.

I assume that Emacs can be customized in a similar manner with BRIEF, but when I looked at Emacs it required a much greater initial effort than BRIEF, to reach a stage where you are able to modify it.

The default editing mode of BRIEF was very intuitive, at least for someone familiar with the IBM PC keyboard, so you could use it easily out of the box, before learning to customize it. Then, like most MS-DOS programs of that time, it came with excellent documentation, so it was very easy and fast to learn how to rewrite its event handling functions. With Emacs, both learning how to just edit texts with it, and then how to modify it according to what you prefer, is much steeper and more time consuming than it was with BRIEF.

wizzwizz4 7 hours ago|||
I'm not sure how that's not "recreating everything". What's still shared if you replace the whole state machine (handling file management, etc) and also replace the whole input handling system? A terminal emulator and a copy of Lua?
exidex 6 hours ago||
There are certainly different possibilities of where the line would be drawn. I am not that familiar with vim internals, but I don't consider file management to be part of vim editing model. And we are only talking about editing model (e.g. b being beginning of a word), not the whole editor (plugin system, ui, lsp and treesitter integration, etc). Though it is true that some parts are more tightly related to "vim". There some cases were Neovim already wants to disconnect itself from vim legacy like allowing to scroll viewport without moving the cursor

By "recreating everything" I was also referring to the ecosystem around the editor

wizzwizz4 5 hours ago||
But doesn't the ecosystem around the editor depend on the editor being, at its core, Vim? https://neovim.io/doc/user/lua-guide/#lua-guide-api describes "three different layers": basic examples like https://neovim.io/doc/user/lua/#watch-file call into the Nvim layer.

I think the idea's good, but I don't think you could modify Neovim to support it while retaining the ecosystem – not without also modifying much of the ecosystem, at which point it may be easier to do a straight port to a different editor, or construct a compatibility layer (essentially reimplementing enough of Neovim in the new editor that the ecosystem still works).

exidex 4 hours ago||
Add that to a long list of side projects I would like to do "at some point"
SloopJon 6 hours ago||
An editor that tries to be layout agnostic is almost certainly going to be a nightmare for people like me who set the layout in the keyboard itself. I downloaded the editor (I'm on Windows at the moment), and tried it with my keyboard set to Dvorak, which was plainly broken. I'm sure there's a way to fix the mapping, but when software thinks it's smarter than you, you end up feeling pretty dumb.
dataangel 4 hours ago|
As always emacs has it already

https://github.com/mickeynp/combobulate

XJ6w9dTdM 4 hours ago|
Yeah and using combobulate feels so natural, I just wish it was easier to add support for other languages. In addition to the navigation, and simple edits (e.g. M-k kills node) you have Searching/editing/etc. using tree-sitter queries directly within emacs, possibly replacing other uses of external tree-sitter based tools.

It makes editing supported languages almost as pleasant as editing Lisp.

With how well the integration already work in existing editors, I'm curious how the UX can be improved further by an editor made specifically with AST editing in mind.

More comments...