Debunking a legend about languages and keyboard layouts

[Sigmoid]

So far when I have asked about keyboard layout suitability and non-English languages, most people usually just shrug, some say one should stick to QWERTY or an equally bad old modified QWERTY layout for that specific language. Thankfully I have found a really awesome online app:
http://patorjk.com/keyboard-layout-analyzer/#/main

This allows one to analyze keyboard layout efficiency on various inputs. So I did - I tested Java code from github, a long German text (the Communist Manifesto off of de.wikisource), and a long Hungarian text (a collection of novellas from Karinthy Frigyes, one of our most significant writers).

It turns out that QWERTY badly underperforms in all these tasks. The only thing shittier than QWERTY is AZERTY. For both German and Hungarian, Simplified Dvorak came out on top, with Colemak a very close second. For Java, understandably Programmer Dvorak wins, but Colemak performs well. The heatmaps show a very nice, home row centered usage profile in both layouts (and a horrid mess in QWERTY).

(I tested English too, with some HP Lovecraft from hplovecraft.com, and Colemak rules there, with a slightly more balanced heatmap compared to Dvorak.)

So no, Colemak isn't just for English. It's most definitely for German and Hungarian too. Now we just need to figure out the best way to add accented letters - dead keys are incredibly uncomfortable on long texts.

[Muirium]

Correct: QWERTY always sucks for any language with vowels (hint: all of them), and the 26 Roman letters of AZERTY and QWERTZ are no better.

Creating a custom Colemak with language specific accented keys is an exercise for the reader.

[Mal-2]

One shortcoming of that site is that it is not capable of generating one-handed layouts. I have three reasons for wanting to do this:

• Sheer curiosity
• Seeing what the loss of speed might be if one were to choose to leave one hand on the mouse at all times
• I've built adaptive devices for people missing a hand before, though not entire keyboards. The central character of my ongoing fiction work also lacks a left hand, and it would be informative to see what she might type on if given her choice.

I have searched for such a site in vain. Is this because it truly does not exist, or does Rule 34 apply to more than just porn?

Here is the right-hand layout I've come up with tentatively on my own:

RH7.png (29.93 KiB) Viewed 10342 times

And the corresponding left-hand version:

LH7.png (30.36 KiB) Viewed 10342 times

In both cases, I made a special effort to keep ZXCV together, even if it is on a vertical. The two columns of modifiers ensure that no key is too far to be reached with mods. I also didn't see any particular advantage to mirroring the entire layout for the left hand (I only flipped the alpha area), but it's not exactly a conceptual stretch to do so. I could not make a trackpoint nub render above all keys, so I have left it out, but it would most likely make it into a final design. The buttons will without question, as the targeted user (like one actual person I targeted in the past) has an arm good enough for operating a trackball, just no hand on the end of it.

M1 through M6 are there specifically to become dead keys for accents and other characters, making this inherently multi-language friendly while remaining centered on English. My character's primary language is Japanese, but since I don't speak/write/type it at all, I'm not ever going there. She also considers English a native language, however, and further along in the continuity she will have to tackle both French and Spanish.

[Eszett]

The devil is our habit. I would recommend any child to start with Dvorak, but to switch as an adult from QWERTY to Dvorak would need years of re-accustomization, since the body-coordination memory is pretty hard to overwrite. I decided to stick to QWERTY layout, and accustom myself to a new layout of modifier keys, which is abit easier to switch over to than to switch over to a changed alphanumerical layout. I don't understand why Dvorak forgot to change the modifier keys as well, to a more reasonable arrangement. We press enter less often than shift. Nevertheless is enter in a more prominent place (the home row) than shift. Which is counterproductive. Not to speak about CapsLock and Tabulator ...

[Mal-2]

The devil is our habit. I would recommend any child to start with Dvorak, but to switch as an adult from QWERTY to Dvorak would need years of re-accustomization, since the body-coordination memory is pretty hard to overwrite. I decided to stick to QWERTY layout, and accustom myself to a new layout of modifier keys, which is abit easier to switch over to than to switch over to a changed alphanumerical layout. I don't understand why Dvorak forgot to change the modifier keys as well, to a more reasonable arrangement. We press enter less often than shift. Nevertheless is enter in a more prominent place (the home row) than shift. Which is counterproductive. Not to speak about CapsLock and Tabulator ...

Because Dvorak was developed for typewriters, and for mechanical reasons, manual typewriters need the Shift keys to be at the sides so they can literally depress the entire type mechanism by a quarter inch or so, causing a different part of each type stem to contact the ribbon and paper. Caps Lock did not exist, only Shift Lock. Enter was a big, hooked metal bar on the left you had to reach up and pull to drag the carriage back into position for the next line (there was no Enter key). Often, there were no 1 or 0 keys either, lowercase L and capital O had to be substituted instead.

This is also why no concern whatsoever was given to the ZXCV control cluster — it did not yet exist.

Dvorak is a system devised for manual typewriters, with all the legacy issues inherent to such a device.

[Muirium]

Yeah, I'd advise against Dvorak. Colemak hit the sweet spot (popular enough to be built in to most OSes these days) and is different in the ways it needs to be, given the hardware and uses we have today.

But QWERTY has stuck about in ubiquity for long enough that I fear it'll last for as long as people use their fingers to enter text! Alas.

[Mal-2]

Yeah, I'd advise against Dvorak. Colemak hit the sweet spot (popular enough to be built in to most OSes these days) and is different in the ways it needs to be, given the hardware and uses we have today.

I'd advise against switching to Dvorak, because Colemak really does a good job of addressing several of its shortcomings without adding any major new ones. I gave it a spin, and if it were the first alternate layout I'd tried, I probably would have stuck with it.

However, if you've already taken the plunge (or jumped the shark) and learned Dvorak, everything I've seen seems to indicate that Colemak is not sufficiently better for typing to be worth the pain of switching. Personally I found it not to be as confusing as I expected considering it's designed for an easy transition from QWERTY, but then I did start on QWERTY and it's a bit like a bicycle. I haven't quite forgotten how to do it.

If your only problems with Dvorak lie in the scattered control keys, a programmable keypad (or a keyboard with programmable extra keys) set up in a QWERTY-like arrangement of control keys may solve the problem. There's no law that says your typing keys and your control keys have to match, or that you can't dedicate keys to the common sequences.

I wish all keyboards had Control Lock, similar to (or even in place of) Caps Lock. It would be enormously useful for certain pieces of software which are most efficiently operated with the Control modifier held down for minutes at a time.

[7bit]

Indeed it should be possible to remap a key if Control is down.

BTW, the CONTROL LOCK key will be available in Round 6.

[wlhlm]

Sigmoid, have you taken a look at the Neo layout? It aims to be a more ergonomic layout for the German language.

[Sigmoid]

Sigmoid, have you taken a look at the Neo layout? It aims to be a more ergonomic layout for the German language.

Yes, I had a look. It's pretty cool, but for now I feel like it's too big a switch... It really is a whole new paradigm.

[davkol]

These online layout comparing tools are cool, but…

• They're not capable of dealing with reasonably large corpora.
• There's usually very little—if any at all—support for characters outside ASCII.
• Furthermore, they don't allow multi-tap elements, esp. dead keys and layers, in the layout design.
• The scoring measures are completely arbitrary and poorly documented.

All in all, they're pretty much useless for any serious analysis. Unfortunately, the same is mostly true for offline tools as well.

BTW people add various random restrictions on layout rating/optimization. For example, the ZXCV mask. If you stick to one-handed Ctrl-[ZXCV] (presumably with a curled pinkie), why bother optimizing the layout at all? Alternatively, have you heard of the Dvorak-QWERTY layout distributed with OS X? It's easily implemented even elsewhere, and we have dedicated scan codes for copypasta and more!

Marsan's criteria adapted by Wagner et al. are the best thing I've found so far… Provided someone ports them to an extended version of carpalx or MTGAP.

[Sigmoid]

These online layout comparing tools are cool, but…

• They're not capable of dealing with reasonably large corpora.
• There's usually very little—if any at all—support for characters outside ASCII.
• Furthermore, they don't allow multi-tap elements, esp. dead keys and layers, in the layout design.
• The scoring measures are completely arbitrary and poorly documented.

All in all, they're pretty much useless for any serious analysis. Unfortunately, the same is mostly true for offline tools as well.

Of course they fall short of serious scientific criteria, but they are useful as pointers.
Also, I think due to the nature of language, a short book is perfectly enough for a test indicative of performance over the whole corpus. (Though I agree that a larger corpus would be necessary for actually optimizing a layout.)

I think several of these tools are open source, so improving them is entirely possible.

BTW people add various random restrictions on layout rating/optimization. For example, the ZXCV mask. If you stick to one-handed Ctrl-[ZXCV] (presumably with a curled pinkie), why bother optimizing the layout at all? Alternatively, have you heard of the Dvorak-QWERTY layout distributed with OS X? It's easily implemented even elsewhere, and we have dedicated scan codes for copypasta and more!

Yea I'm not in love with ZXCV or anything... It is better for mouse workflow than having these functions all over the place... but it's not the reason I like Colemak. It simply looks like a good layout, it was comfortable to use, and there does exist such a thing as "good enough". Optimize something too far, and the reward per unit effort continues to drop dramatically.

[davkol]

Of course they fall short of serious scientific criteria, but they are useful as pointers.

Scientific criteria? More like basic functionality, if they drop plenty of symbols or the input layout offers no way of typing the necessary symbols.

You can't use the patorjk.com analyzer to evaluate, say, Czech QWERTZ on a Czech corpus, because it simply drops plenty (if not all) of necessary accented symbols. Forget about proper typographic punctuation symbols as well. Maybe you could process the layout with some ad-hoc regular expressions, but that's just hitting nails with a screwdriver.

Also, I think due to the nature of language, a short book is perfectly enough for a test indicative of performance over the whole corpus. (Though I agree that a larger corpus would be necessary for actually optimizing a layout.)

A short book, typically from Project Gutenberg, contains neither grammar, nor vocabulary used in work-related reports (no-one sane will put that into a random web form), daily e-mail/IM communication, 4chan comments (eh, that escalated quickly), you name it. It's easily skewed by the names used.

I think several of these tools are open source, so improving them is entirely possible.

Unless you attempt to read the documentation… or the source.

I've reimplemented the carpalx typing effort model from scratch, because the upstream code wasn't exactly easily extensible in terms of Unicode and multi-tap support mentioned above. The default set of parameters needs a lot of adjustments too. There are inconsistencies between documentation and the actual implementation, and what's worse, possible mistakes (e.g., stroke path components are sorted awkwardly).

I like Colemak. It simply looks like a good layout, it was comfortable to use, and there does exist such a thing as "good enough". Optimize something too far, and the reward per unit effort continues to drop dramatically.

Colemak is an excellent layout for a very specific use case. The use case is typing in English on a "standard" (read asymmetrical, staggered) modern laptop (read: low-profile) keyboard, if you can appreciate Tarmak and don't care about maintaining your QWERTY skills.

I find single-hand runs frustrating, when typing in Czech on an actually ergonomic keyboard, and see PieterGen's analysis of Colemak on Dutch (and perhaps German too) corpus.

[Sigmoid]

Part of the problem is that nobody types in a single language anymore, except for native English speakers. Regardless of your nationality, you'll type English, and probably type as much or more English than any other language. If you wanted to optimize for multiple languages, you'd have to calculate character and bigram frequency for your specific use, which is, honestly, an exercise in futility (unless you were just collecting your own use data for a year).

Also, while Colemak may be inoptimal in German or Dutch, my point was that it almost certainly is better than QWERTY. The i18n indecisiveness around Colemak is like arguing whether mineral water with low or high mineral content is better, and drinking waste water out of indecision.

And to top it off, learning a keyboard layout carries cost, and using multiple fundamentally different keyboards slows most people down in general. Also, especially on OSX, nonstandard keyboard layouts also carry a software cost, as the only tool for editing layouts is outdated, badly maintained and unfriendly, and OSX tends to bug out when using user layouts.

[davkol]

Part of the problem is that nobody types in a single language anymore, except for native English speakers. Regardless of your nationality, you'll type English, and probably type as much or more English than any other language.

Not true for the vast majority of population.

If you wanted to optimize for multiple languages, you'd have to calculate character and bigram frequency for your specific use, which is, honestly, an exercise in futility (unless you were just collecting your own use data for a year).

Wait, you wrote that pasting snippets into patorjk analyzer provides a significant insight earlier.

The "exercise in futility" is more like "ignorance". It's only necessary to pick reasonably representative corpora. Archived personal e-mails or e-mails from mailing lists, technical articles (can be represented by contents of properly chosen Wikipedia portals), MUC archives. Very few people are special snowflakes with special vocabulary.

Also, while Colemak may be inoptimal in German or Dutch, my point was that it almost certainly is better than QWERTY. The i18n indecisiveness around Colemak is like arguing whether mineral water with low or high mineral content is better, and drinking waste water out of indecision.

What measures are you using? What weights? Otherwise, it's impossible to call a layout better or worse.

The Colemak-QWERTY thing is a false dilemma. There are many more different layouts, and different typing techniques as well. Colemak isn't a standard either, thus it doesn't have the advantage of convenience that QWERTY, DSK and some national layouts have.

And to top it off, learning a keyboard layout carries cost, and using multiple fundamentally different keyboards slows most people down in general.

[citation needed]

Arguably, typing in more very different layouts reduces the risks of mixing them up. The same applies to other skills as well.

[Mal-2]

BTW people add various random restrictions on layout rating/optimization. For example, the ZXCV mask. If you stick to one-handed Ctrl-[ZXCV] (presumably with a curled pinkie), why bother optimizing the layout at all? Alternatively, have you heard of the Dvorak-QWERTY layout distributed with OS X? It's easily implemented even elsewhere, and we have dedicated scan codes for copypasta and more!

I've posted two solutions for the ZXCV dilemma. One (the most obvious) is to have a set of permanently CTRL-enabled keys, of which ZXCV would be at lower left. This is my layout, for example. The purple keys aren't visually distinguished in reality, they're just tagged that way to show that AltGr + that key is a dead key (and a second one when shifted). That's what the green keys at upper left are mapping to. (I'm only using seven of the available eight dead keys, nothing is assigned to AltGr + ".)

dv-right.png (48.57 KiB) Viewed 10062 times

Another is to leave them clustered, though not necessarily horizonally. I made them a vertical column in each of the one-handed layouts posted earlier in the thread. C is still reasonably reachable (as befits its letter frequency in English) while X and V are less so, and Z is quite a stretch from the home position.

[Sigmoid]

Wait, you wrote that pasting snippets into patorjk analyzer provides a significant insight earlier.

The "exercise in futility" is more like "ignorance". It's only necessary to pick reasonably representative corpora. Archived personal e-mails or e-mails from mailing lists, technical articles (can be represented by contents of properly chosen Wikipedia portals), MUC archives. Very few people are special snowflakes with special vocabulary.

Now I've got the feeling you're just messing around. This isn't a verbal boxing match.

So here's the thing. You type in Dutch, German and English. What is your personal bigram frequency? That depends on how often do you type Dutch, how often German, and how often English. Also, what do you type in each? Are you writing a novel in Dutch and posting on forums in English? Etc.

It's not about being a "special snowflake".

And I am not being inconsistent as you're trying to make it out. Indeed, what I just explained means that pasting snippets into patorjk is "as good as it gets" unless you install a keylogger and collect actual statistics for a year. (And yes, your points on patorjk not handling unicode and dead keys is perfectly valid. This argument isn't about that.)

[kbdfr]

Is it possible to "tell" the keyboard layout analyzer that I am using an AZERTY layout hen entering a sample text?

[davkol]

So here's the thing. You type in Dutch, German and English. What is your personal bigram frequency? That depends on how often do you type Dutch, how often German, and how often English. Also, what do you type in each? Are you writing a novel in Dutch and posting on forums in English? Etc.

It's only necessary to pick reasonably representative corpora. Archived personal e-mails or e-mails from mailing lists, technical articles (can be represented by contents of properly chosen Wikipedia portals), MUC archives. Very few people are special snowflakes with special vocabulary.

-->corpora
a plural of corpus

Pick a corpus for each kind of use case. It's mostly just a matter of using `wget` or a web crawler, applying some regular expressions and extracting trigrams + frequency. Run the analysis for each. Assign each corpus/result a weight, and take it into account, when compromising. Done.

Is it possible to "tell" the keyboard layout analyzer that I am using an AZERTY layout hen entering a sample text?

[/quote]
patorjk? Open the Configuration tab, choose AZERTY from European layouts in the Preset select list, Load.

[kbdfr]

Is it possible to "tell" the keyboard layout analyzer that I am using an AZERTY layout hen entering a sample text?

Open the Configuration tab, choose AZERTY from European layouts in the Preset select list, Load.

That’s what I had done, but it seems not to work,
and accentuated characters of my sample text are not accepted.

So it is absolutely no wonder if, even when I type something like "Il l’a déjà aperçue dans son rêve.",
AZERTY is considered the worst possible layout.
Those accentuated characters which you type with a single key press in AZERTY,
and which would be considerably more difficult to generate with another layout,
are simply ignored

[davkol]

These online layout comparing tools are cool, but…

• There's usually very little—if any at all—support for characters outside ASCII.

[ryebread761]

Yeah, I'd advise against Dvorak. Colemak hit the sweet spot (popular enough to be built in to most OSes these days) and is different in the ways it needs to be, given the hardware and uses we have today.

But QWERTY has stuck about in ubiquity for long enough that I fear it'll last for as long as people use their fingers to enter text! Alas.

I have a difficult time with the statement about it being supported nearly everywhere. I can get Dvorak on the computers running Windows 7 at my school, but not Colemak as far as I am concerned. This creates an issue for me and anyone else who would like to have their preferred layout at school, work, the library, etc. where they do not use one computer all the time but instead move around frequently.

[Muirium]

It's been a factory default for a long time on the Mac, and for external keyboards in iOS. Can't say I follow Windows. Though I assumed 7 and up should have it.

Of course, swapping system keyboard layouts on machines beyond your own is a daunting task in itself. Last time I used Windows without admin rights, I remember that stuff was hell. I'd suggest doing it in the keyboard instead, such as using a keyboard with a programmable controller (like almost all of mine) that can appear handle all the layout tweaking you like, quite independent to the host. But then you're talking about toting around your own hardware of course, which isn't ideal either.

Meanwhile: QWERTY's always there from the get go. That is of course its one true strength.

[kbdfr]

"Debunking a legend about languages and keyboard layouts"?
By absolutely no means.

The keyboard layout "analyzer" in fact analyses only characters existing in English, omiting any language specific ones,
i.e. exactly the ones which clearly are an advantage when typing a "national" text on a "national" keyboard.
So it just cannot "conclude" anything about languages if it excludes precisely the feature it pretends to analyze.

That is not only utterly non-scientific, it is just sheer nonsense. Absolutely ridiculous.

[ryebread761]

It's been a factory default for a long time on the Mac, and for external keyboards in iOS. Can't say I follow Windows. Though I assumed 7 and up should have it.

Of course, swapping system keyboard layouts on machines beyond your own is a daunting task in itself. Last time I used Windows without admin rights, I remember that stuff was hell. I'd suggest doing it in the keyboard instead, such as using a keyboard with a programmable controller (like almost all of mine) that can appear handle all the layout tweaking you like, quite independent to the host. But then you're talking about toting around your own hardware of course, which isn't ideal either.

Meanwhile: QWERTY's always there from the get go. That is of course its one true strength.

I can see why you would think that, but I have yet to find it. Everything I've seen on google requires you to actually install it for use on Windows 7. I have never had any issues changing to Dvorak at school. It's a user-level property, nobody else has Dvorak as an option for them when they use the computer AFAIK but once I enable it I can select it from the taskbar. It's kind of like changing the desktop background or any other property that is dependent on the user which is logged in at the time.

[Sigmoid]

That is not only utterly non-scientific, it is just sheer nonsense. Absolutely ridiculous.

Have you just logged on and thought "wow man, what I really need right now is to flame someone"? Chill out.

Anyway, whatever. My point was pretty much that regardless of language, you can only win by not using QWERTY. People tend to be on about "Colemak / Dvorak was optimized for English, so it's inoptimal in other languages", to which my response is "NOT AS INOPTIMAL AS QWERTY!"

As for your defense of AZERTY before (which unlike the last post was quite civil, thank you), yes, I'm sure it's a lot harder to type French on a keyboard that doesn't have the accents you need for it... The question is, if you put the same accents on a Colemak variant, would that be better than AZERTY or worse... (Given that AZERTY has nothing BUT consonants on the home row, my educated guess is that it would be a LOT better.)

[kbdfr]

That is not only utterly non-scientific, it is just sheer nonsense. Absolutely ridiculous.

Have you just logged on and thought "wow man, what I really need right now is to flame someone"? Chill out. […]

Flame?
Not at all, it is a nothing but a fair comment if you consider the sentence preceding that appreciation:

[…] So it just cannot "conclude" anything about languages if it excludes precisely the feature it pretends to analyze.

That is not only utterly non-scientific, it is just sheer nonsense. Absolutely ridiculous.

May I recall you opened this thread with the following words?

So far when I have asked about keyboard layout suitability and non-English languages, most people usually just shrug, some say one should stick to QWERTY or an equally bad old modified QWERTY layout for that specific language. Thankfully I have found a really awesome online app:
http://patorjk.com/keyboard-layout-analyzer/#/main

This allows one to analyze keyboard layout efficiency on various inputs. […]

I still contend that it clearly doesn't.
Not counting diacritics like é, ñ, š, ł or ø, it analyzes non-English texts as if they were written only with English characters, and thus biases the analysis by depriving national layouts of the exact feature at which they are clearly at an advantage, i.e. their specific national characters.
So, yes, I stick to it: "It is not only utterly non-scientific, it is just sheer nonsense. Absolutely ridiculous."

Call that "flame" if you like.
Apparently you find it difficult to accept critical comments not even directed at yourself,
but not to call someone an "asshole" based on a sheer assumption.

[Halvar]

Kbdfr is clearly right: one cannot make a statement (or "debunk a legend") about "keyboard layout suitability and non-English languages" and ignore the language's special characters. It does have something ridiculous to even try that, as special characters are the whole point why Dvorak or Colemak aren't considered to be well suited for other languages.

To my understanding, what Sigmoid is suggesting at the end of his OP is that you would take e.g. Colemak as a base and develop a language-specific Colemak-based layout much in the same way as today's QWERTZ and AZERTY layouts are QWERTY-based. That might be interesting, and you might come out with something that is more efficient that QWERTZ or AZERTY, but going that way is not "debunking the legend" at all, and to test it properly you would again need a tool that supports characters beyond ASCII. And of course whatever comes out of that process would be yet another different layout that only one person on earth is using, unlike what is commonly understood by Colemak or Dvorak.

[Nuum]

There are versions of Dvorak for other languages, but they might be not ideal for the specific language as the main alpha cluster remains the same for all Dvorak versions.

[davkol]

There are versions of Dvorak for other languages, but they might be not ideal for the specific language as the main alpha cluster remains the same for all Dvorak versions.

There are many Dvorak-like layouts (Dvorak-fr, Bépo, German Dvorak type II) that are only based on the same principles.