Posted by futohq 2 days ago
So for the longest time, I've wanted a new keyboard layout specifically designed for swiping. In the same way that Dvorak was optimized for ergonomically typing English words, I want a keyboard layout designed to minimize word overlap/ambiguity when swiping.
It doesn't even necessarily have to have 26 keys, e.g. maybe there could be one key overloaded for v/w/x/z (and you long-press it if you ever want to type a single letter). On the other hand, maybe there need to be separate keys for 'e' and 'ee', or a special key for "double the previous letter".
Because I love swiping, but all my problems with it come from the fact that the QWERTY layout is far from ideal for it. I am 100% willing to learn a new layout if anyone will develop an optimal one for English so that swiping has a 99.9% accuracy rate instead of what currently feels more like 90% or 95%.
https://github.com/futo-org/futo-keyboard-layouts/issues/163
It's one of those things I've been wanting for a decade now, sounds like it's been around for about a year and a half.
Man, this is the kind of stuff I come to HN for.
EDIT: Ugh! I have an iPhone, and FUTO is Android-only, and ClearFlow is only available in the Android Gboard, not the iOS Gboard. :(
I'm happy with the switch. Like any keyboard switch (I've gone from Qwerty to Dvorak and now a Colemak-dh derivative with about ten years on each) it takes some time to learn the layout. Overall I'm happy with it though and there are less frustrating misinterpretations and corrections needed.
This post was swiped on it with only two corrections and the second one was my fault as i misremembered a key location.
[1]: https://clearflowkeyboard.github.io/#section_activate [2]: https://youtu.be/rSfbvE9cEKE?si=NbJC93sTiOHqw4lX
I'm writing down a few impressions: - the layout is unusual, but I get the motivation. Distances are minimised and letters are arranged so that ambiguity is removed. - although I'm very slow, I haven't made a single mistake so far. Clearflow allows me to swipe much more accurately than stock gboard. - the square keyboard layout unfortunately means that half the letters are constantly hidden behind my thumb. As I'm unfamiliar with the layout, this means that before swiping a word, I have to look at the layout, memorise letter locations and plan the movement - since I write in multiple languages and Clearflow is available in only one of them, I would have to memorise a completely new layout for a language I write in only half the time.
For learning ClearFlow, I used the Games app available from the "Clearflow Games" section on their website: https://clearflowkeyboard.github.io/
I also have the issue of the thumb getting in the way so I spent a couple of days playing the games to get my layout memory up and then it became usable without frustration and I'm not looking back now although I occasionally still forget the odd letter location.
It could be interesting for applying it to different languages (or modified word corpus).
Maybe I'm mistaken though. Are there any physical clearflow keyboards? Are they any good, or does clearflow really only work well with swipe?
1. Switching to Dvorak on my computer AND 2. Switching to Colemak on my phone
I already had a feeling that my Dvorak experience on phone wasn't gonna help typing in a physical keyboard and indeed that was the case. But also, the physical keyboard experience with Colemak didn't really help with thumb typing/swiping on the phone. On a physical keyboard, I can only type QWERTY at around 30WPM and Colemak at around 130WPM. However, on the phone QWERTY is my strongest layout (even when I can only type at around 50WPM with it).
Then there's the design philosophy. Colemak aggressively keeps the most used letters on the homerow. Paste a random Wikipedia article into this page and you'll see (https://www.patrick-wied.at/projects/heatmap-keyboard/ alternatively, here's the heatmap for the introduction to today's featured article about Augustus: https://postimg.cc/9RT8P4N6). On a phone this means that a lot of the times you'll be swiping back and forth horizontally. Resulting in identical swipe paths for a lot of common words.
And if Clearflow was ported to a physical keyboard, most of your fingers won't be doing any work and the active fingers will be doing weird tap dance. The position of spacebar (undoubtedly the most used key on the keyboard) to the sides where you'd be using your pinky would make it an RSI machine.
My point is that if you're comfortable with QWERTY on the phone, that's not because you're comfortable with QWERTY on the desktop. That's because you got used to QWERTY on the phone.
90-95% is a very good estimate! That's about what we measure on our test set. I have good news for you, and we will have a blog post about it soon. Because of how our models are built, we are able to optimize for detection accuracy directly by constructing synthetic swipes on each layout for ~50k words, and then testing them through the model. We tested around 800,000 layouts this way.
The biggest issue with QWERTY is that there are far too many words that swipe colinear or obtuse angle letter trigrams. These are both hard to detect and frustrating for swipe users, because you can't clearly indicate the letters you're gesturing. Neural swipe models (at least ours) look for indicators in the gesture pattern that suggests a user was targeting a specific letter, rather than trying to match a gesture shape like algorithmic detection does.
The shape of the keyboard can significantly improve the way the gestures are formed so that there is better indication of letters. The model can still respond to dwell times because unlike shape matching it uses the temporal information. But dwell interrupts flow, and in my opinion should be minimized in swipe layouts.
The ContextLM model is a very small language model that is trained for a single language. It's used to improve the quality of predictions by eliminating nonsensical words given the preceding words in the sentence. It only requires text data for training.
http://www.fitaly.com/fitaly/fitaly.htm
However, there’s no Fitaly keyboard available for modern phone OSes.
You mean like the two E’s in “feel” or the two L’s in “fell”? I just tried and it handles them well. Are you aware of the circling technique? When you want to double up on a letter, you briefly circle it slightly. I believe some keyboards let you hover momentarily without circling.
Try it, swipe F-E-L, it should complete to “fell”, then do the same thing again but form a small, tight circle over the E, it should then complete to “feel”. Works for me every time.
So for feel, you start at F, go to E, loop once, then L. For fell, start at F, go to E, then loop on L. Very easy to pick up as a physical habit.
I just tested those two on futo, and it easily picked them both out.
I looked in the "Swipe Typing" options and there doesn't seem to be a config for it.
Is it possible that your keyboard’s particular dictionary knows the words you’re more likely to use and adjusts for it?
Edit - Also got ‘grill’. Notice how the -t in felt and -I- in grill are not near path to L.
A workaround is to use the Notes app and use the return key to make a new line after each try, rather than deleting. That should give you more consistent results.
Eg enter Bürger Dienste and have it autocorrect to Bürgerdienste. Or even Führung Kraft and turn it into Führungskraft (inserting an s).
Edit: apparently there's a modern successor? https://play.google.com/store/apps/details?id=inc.flide.vi8
* https://github.com/dessalines/thumb-key
It has a similar sort of 'It doesn't have to have 26 keys on something the size and shape of a mobile 'phone.' thinking as 8vim has, whilst raising a good 'You know 'phones worked fine with a 3 by 4 grid for 60 years, ne?' point, but adding a modern twist of 'We can swipe, in the 21st century.' to the old notion of multiple letters on a button.
There are still these people thinking outside of the typewriter-keyboard-on-a-'phone box. (-:
But it just can't touch swiping for speed. Frankly, the keyboard I miss most is the T9 predictive text from my old school pre smart phone era.
Nothing has come close to the same expressiveness and speed while being usable completely blind, only by feel.
I do feel like mobile keyboards have stagnated in a bad spot, though.
In fact (with Gboard) the suggestions don't change with cursor position. Surely, when I place the cursor you the right of a letter I'm planning on removing that letter, or adding a letter, but the suggestions don't change according to cursor placement.
There's also no apparent frecency - I had to correct cursor every time.
I swiped "frecency" as "decency" - corrections offered were "d doubt difference". The swipes for those are wildly different. d->e is NNW (350°), d->i, d->o is ENE (~70°).
It's really so basic. Surely they can do better than this.
I hope FUTO does start caring about language support, because for example their AI powered text prediction is only available for English. I'd happily train a model for them in my native language if they provided instructions on how to do so. And I'd help with swipe typing too.
http://networkimprov.net/alphatap/light.html
It was set up for just this.
c.f., the opensource research project Dasher
There are a few issues, like it randomly capitalizes words in the middle of sentences. Also, it doesn't seem to take context into account when suggesting words, so words that clearly wouldn't follow the last word will often show up.
It's not as good as gboard yet, but close enough that I'm going to stick with it.
Note that if you have a more powerful device, you can get larger models for voice and larger dictionaries from their site. They make a noticeable difference.
The only fundamental issue I have with it, they seem to be ideologically opposed to adding a GIF search, which I miss occasionally. https://github.com/futo-org/android-keyboard/issues/293#issu...
Nice to see the hour of swiping I did adding to their dataset actually helped. I'm using it now and it feels as good as the Google keyboard.
Edit: It is sending me a little that it keeps swiping "whats" instead of "what's" though, hopefully they fix that later.
- https://gitlab.futo.org/keyboard/swipe-library/-/blob/master...
- https://github.com/futo-org/android-keyboard/blob/master/LIC...
https://huggingface.co/futo-org/futo-swipe/blob/main/LICENSE...
Is it this part?
you may not remove or obscure any functionality in the software related to payment to the Licensor in any copy you distribute to others.
As an aside, Eron Wolf, the billionaire behind FUTO, has some rather... out of touch views[0] on the meaning of open source, and seems very committed to diluting the term to mean something closer source-available by removing the most of the rights granted (as defined by FSF, OSI, DFSG and others).
[0]: https://gitlab.futo.org/eron/public/-/wikis/Thoughts-on-Open... - please keep in mind that the RMS quote at the top is taken out of context; he is arguing for more freedom, not less
this is not an out of touch view. on the contrary. what is out of touch is to believe that Open Source is perfect as it is, and nothing should be changed about it.
the problem that FUTO is trying to address is the commercial exploitation by large companies (like AWS) at the expense of the original creators. a problem that several companies (redis, mongodb, elasticsearch, zerotier, terraform, vagrant, and many more...) in recent years had to face, causing them to change their license.
it's a problem that bruce perens is also trying to address: https://news.ycombinator.com/item?id=38783500 "What comes after open source? Bruce Perens is working on it"
btw: i recently asked bruce about this and he said that AI now changes everything and he wants to wait and see how that will affect Open Source.
it is not obvious that FUTO's approach is the right one. it is an attempt at addressing the problem, and i expect that it will take more such experiments to shake out what the best approach to this problem really is.
i wrote about this before:
This is because 99.9% open source projects not targeted enterprise never ever see more than $100 of donations and being maintainer of such software is literally thankless job that will never pay you anything.
Blaming organizations for giving money or maintainers for taking money is worse you can do no matter who sponsor is: FSF, FUTO, Cloudflare, Microsoft, Facebook, Oracle, DARPA or MAGA INC.
Dont like FUTO or its owner? Make a better fund, give your money to SFC, FSF or whatever open source sponsor organization is acceptible to you.
There is so little money in end-user open source software and making pie even smaller or antoganizing thankless people maintaining it is awful. Everyone doing this is either dumb, malicious or both.
Clearly not a cuck license (https://lukesmith.xyz/articles/why-i-use-the-gpl-and-not-cuc...) so looks good to me.
It's just a commercial license with very mild terms.
The source code is fully available, none of the features are paywalled. They only prohibit you from taking their code and reselling it.
If you take a look at the Play Store, there are thousands of instances where open source projects are lazily renamed and sold for $5 or $10. It's the definition of scummy, pathetic, worthless behavior, and I'm glad the license prevents those kinds of leeches from succeeding.
I know this isn't the only case, but it's the majority of cases. So I have no problem with their license at all.
So no, the license doesn't matter.
It might be not a foolproof solution, but I think the license is better than nothing. Then you have a legal precedent that you can cite when you file a lawsuit against these rats.
Most of these people doing this probably aren't in the same country. But whatever. It's better than nothing.
They prohibit you from removing the constant nags about buying a licence.
1. Paying and clicking “I paid”
2. Not paying and still clicking “I paid”
So it’s an honour system right now.
That said, if they ever implement e.g. license keys or some other mean of actually checking that you’ve paid, seems that you would be able to remove it and recompile, you just can’t help others do that:
> Notwithstanding the above, you may not remove or obscure any functionality in the software related to payment to the Licensor in any copy you distribute to others.
(IANAL)
Or you know, also buy the license and click it honestly.
Is that really true? My memory of the original iPhone's touch screen is that it was pretty much pixel-accurate.
The article mentions that the keyboard wasn't accurate enough: "But by early 2006, the iPhone keyboard still didn’t have the accuracy Apple needed to ship the phone." I don't think that means the screen wasn't accurate; all it means is that the original iPhone had a small screen, so the buttons on the keyboard were tiny, and hitting them precisely was difficult. That's why the hit boxes of more likely keys were enlarged.
The base reason is the size of the keyboard compared with the size of thumbs and the imprecision of thumb typing. Adjusting the hit boxes results in a better error rate. It isn’t because of the resolution of the screen or touch detection.
See https://www.grammarly.com/blog/engineering/deep-learning-swi... for more details - it's very similar to the architecture described by the FUTO folks.
One key difference is that the learned model does not decode in a context sensitive manner but does it a word at a time. The main reason is because we wanted to release this soon and wanted the user's personal dictionary (i.e. contact names, etc... to show up correctly when swiped). It would have been nice if we could have followed through with the context sensitive decoding as described by the FUTO folks. It would really help with accuracy when dealing with words like:
1. (food, good, hood) 2. (you, toy, rot) 3. (our, or, it) etc...
(Disclaimer: I am one of the authors of the Grammarly swipe system as described in the linked blog post).
> The library also supports recognizing two-finger simultaneous swipe input through the SwipeEngine::recognize_multi method.
Integrated speak to text, good autocorrect typing, good autocorrect swiping.
This is such a massive deal. This is, as far as I can tell, the first useful free and open Swipe model. This paves the way for things like swipe typing on platforms other than iOS and Android, a major pain point to newcomer OSes.