F104+SSK+122+62+77+50+Ergo orders now open! New Kishsaver+Industrial Model F Keyboards

[webwit]

P.S. Ellipse uses "updated firmware":
https://www.modelfkeyboards.com/code/

Which should be the same I guess if it's based on the latest code (maybe Ellipse can elaborate). But the threshold level is compiled in there as well, I'm currently using this one with threshold 6: ibm_capsense_usb debounce threhold 6-recommended.hex
Can you check what you're using before making a new hex?

EDIT: Too late. Can you check what you used?

[pandrew]

My hex file is based on xwhatsit's latest 0.9.0 release with my changes applied.
On https://www.modelfkeyboards.com/code/
I don't see source code for the debounce-modded firmware.
I'm still trying to find source code for that so I can create a .hex that combines both.
Anyway, my .hex should work for you as far as testing goes, even if the keys are a little bit bouncier with it.

[webwit]

Test 1...2...3...

Only tested briefly, but at first glance it seems we have a WINNER! I can't reproduce the layer problem!

[pandrew]

That's great!
I found the patch file for the debounce in this thread. Now working to apply my changes on top of that.
Not so easy because it modifies the same code as I did.
Anyway, I'm a bit concerned that I see the patch file also modifies two more things:
1) the usb max current 100->500 -- shouldn't be needed, but fine, at least it's certainly more stable.
2) it changes the address of the bootloader -- that's a bit unsettling. So which one is correct? the official xwhatsit value of 0x1800, or the patched value of 0x3800? Looking at the datasheet I think that the official xwhatsit is wrong, and the patch is right. So it's possible that if you try to enter bootloader from the GUI, it won't work with my last .hex, and maybe you have to short the PROG pads again.

Can you try restoring to the official release and see if you succeed without needing to short the PROG pins again?

[webwit]

Yeah, I can restore it without problem.

Maybe Ellipse or others can answer you other questions.

[pandrew]

[EDIT: this didn't work. Please see follow-up post(s)]

Alright, attached are my hex files which are based on the alternative 6&&11 debounce firmwares.
For reference, these are the patches:
1) joc's patches on top of official xwhatsit 0.9.0:
https://github.com/purdeaandrei/ibm_cap ... faad659663
2) my patches on top of it all to fix the layering issue
https://github.com/purdeaandrei/ibm_cap ... 3258aac5ae

.hex files attached.

The only testing i've done is that it doesn't go crazy on a microcontroller devboard

Andrei

[EDIT: removed .hex files that don't work]

[webwit]

I tried threshold6, but that one doesn't work. It passes the flashing, but no keys are registering.

[pandrew]

Alright, I'll look at it again, to see why it isn't working. In the meantime a question, can you restore from threshold6 without shorting PROG?

[pandrew]

[EDIT: this didn't work. Please see follow-up post(s)]

Found another stupid mistake
So now the list of patches is:
1) joc's patches on top of official xwhatsit 0.9.0:
https://github.com/purdeaandrei/ibm_cap ... faad659663
2) my patches on top of it all to fix the layering issue
https://github.com/purdeaandrei/ibm_cap ... 3258aac5ae
3) fixing a stupid mistake in 2.
https://github.com/purdeaandrei/ibm_cap ... 9873ffe107

.hex files attached

Andrei

[EDIT: removed .hex files that don't work]

[webwit]

That one does register keys but far too many. I.e. if I type "t" it will output t plus another random character, sometimes two, sometimes capitalized or del or strange character (i.e. random scancode). I reverted to your original fix just to check and that one still works best. So there's still something wrong in the merge with joc's patches or the threshold settings.

EDIT: Have been typing a bit with your original fix before the merge and it really works fine so far.

[pandrew]

Alright, found another small bug, really hope this is the last one I have to fix
Complete list of patches is:
1) joc's patches on top of official xwhatsit 0.9.0:
https://github.com/purdeaandrei/ibm_cap ... faad659663
2) my patches on top of it all to fix the layering issue
https://github.com/purdeaandrei/ibm_cap ... 3258aac5ae
3) fixing a stupid mistake in 2.
https://github.com/purdeaandrei/ibm_cap ... 9873ffe107
4) fixing another mistake:
https://github.com/purdeaandrei/ibm_cap ... 9f0282fd7c

.hex files attached

cheers,
Andrei

[webwit]

Seems to be working well! Typing with this now without problems.

I'll be doing some programming with this tomorrow for some heavy duty testing. If anyone else can test it that would be great.

Ellipse, can you double check if this has the same patches as the latest firmware on your site plus pandrew's fix?

Just curious pandrew, I've been looking at the git changes, but it's beyond my expertise. But in what way was the original wrong and did you fix it?

[pandrew]

Just curious pandrew, I've been looking at the git changes, but it's beyond my expertise. But in what way was the original wrong and did you fix it?

Assuming original = joc's version (the problem is the same fundamental problem in pure xwhatsit release, but some of the details differ).

The software scans through all columns, and reads the status of the rows. Based on the status of each row, and the debouncing algorithm used, it tweaks the scanState[][] matrix, which holds the debounced status of each physical key. scanState[][] contains 1 byte of information for each physical key. If the value is >0 then it's pressed. If it's <0 it's not pressed.

This then needs to be transformed into a scanCode matrix (called kbdSCBmp[], which is 1 bit per scan code).

The layersMatrix[][] variable holds the physical key to scancode mapping of the _current_ layer only. Because of very little SRAM available, only a single layer's data can be held in memory at any single time. So any time you press a Function key, layersMatrix[][] is repopulated from eeprom.

The original code simply converts scanState[][] using information in layersMatrix[][] , to scan codes that are stored in kbdSCBmp[]. This is done in the "kbdUpdateSCBmp()" function. This code doesn't care about any other state. It doesn't care what layer was active when a physical key was depressed, it only cares about what layer is active now, and which keys are pressed right now. So if the only thing that changes at some point in time is which layer is active, then it will cause one scancode to go to zero, and another scancode to go to 1, so it effectively releases your original keypress, and will send a new keypress with the newly active layer.

To combat this I needed to memorize what layer was active for each currently held keypress. The idea is, that for each physical key you hold in a variable what the pressing-layer was. So that physical key will be ignored within the new layer, until it is released, (and when it's released it is released with the correct scancode).

This is problematic to implement because there wasn't enough memory to store another variable that has 1 byte for each physical key. The Atmega32u2 just isn't big enough to store this information in RAM as another variable. But I realized that the scanState[][] matrix doesn't need all of the bits of it's 1 byte per physical key. So I renamed the scanState[][] variable to scanStateAndAssociatedLayer[][], and I assigned the bits the following way:
* bits 4..0 hold the original scanState signed integer. (bit 4 is the sign bit)
* bit 7 holds the sticky layer status. If this bit is 1 then a layer is associated to this physical key, if it's 0 then there isn't a layer associated.
* bits 6..5 hold the number of the layer that is associated (0..3) (this is valid only if bit 7 is set).

With this solution I ended up not consuming any extra memory at all.

I implemented access to the subfields of this byte using macros in scan.h, and the main algorithm is in kbd.c, starting line 192.

Hope that's clear enough.

Cheers,
Andrei

[pandrew]

Ellipse, can you double check if this has the same patches as the latest firmware on your site plus pandrew's fix?

joc's patch is over here: download/file.php?id=31138

And this .zip file contains the same .hex files that are released on https://www.modelfkeyboards.com/code/

So I'm quite confident now that I'm using the same patches.
To be very clear, there is 1 patch, which is for debounce threshold of 11.
For the debounce threshold of 6, I went in and manually edited the value of the SCAN_DB_THRESH_TOP macro in scan.h, from 11 to 6.

PS, I just want to say, I now understand joc's debouncing algorithm, and it's much more robust then the original 0.9.0 xwhatsit release. joc's debouncing would probably perform better then the original 0.9.0 even with thresholds that are less then 6.

[webwit]

Damn. Admittedly I don't understand most of that, except that you know your trade. You're a boss, man!

[Twst]

The beauty of open source!
Great work pandrew.
Will test out on my new model fs, hopefully this week

[pandrew]

The previous explanation was very heavy on technical details, I also wrote out the differences in python-ish pseudocode to make it easier to understand:

original code:

Code: Select all

for each scanCode:
    scanCodePressed[scanCode] = 0
for each row, col:
    if physicalKeyPressed[row, col]:
        scanCode = convertRowColToScanCode(currentLayer, row, col)
        scancodePressed[scanCode] = 1

the generation of press/release events is handled somewhere else, by comparing previous and current values of scancodePressed. So this code is only responsible of maintaining an accurate image of what scancodes (virtual keys) are being actively pressed.

my code:

Code: Select all

for each scanCode:
    scanCodePressed[scanCode] = 0
for each row, col:
    if physicalKeyPressed[row, col]:
	scanCode = convertRowColToScanCode(currentLayer, row, col)
        if isThereAnAssociatedLayer[row, col]:
            if associatedLayer[row, col] != currentLayer:
                scanCode = convertRowColToScanCode(associatedLayer[row, col], row, col)
        else:
            isThereAnAssociatedLayer[row, col] = 1
            associatedLayer[row, col] = currentLayer
        scancodePressed[scanCode] = 1
    else: (if it's not pressed)
        isThereAnAssociatedLayer[row, col] = 0

Note currentLayer, is modified every time a function key is pressed or released.
So convertRowColToScanCode() will return different value when you press/release a function key.

So let's say you have a base layer where a key is mapped to the letter "A", and a function layer where it's mapped to "B".
If you do this, using original code:
1) press FN
2) press A/B key: at this point scan code for B press is sent
3) release FN: at this point two things are sent: B release, and A press
4) release A/B key at this point A release scan code is sent

With my code:
1) press FN
2) press A/B key: at this point scan code for B press is sent
3) release FN: nothing happens, because my code knows that A/B key belongs to the alternative layer.
4) release A/B key at this point B release scan code is sent

Also, if this is used in a context of a game for example, you could, with my code:
press FN, press A/B key, release FN, keep the A/B key pressed, and it will keep sending B scancode,
but if in the meantime you want to press other keys, they will come from the base layer as you might expect.
So for example you could keep using the arrow keys from the base layer, while B keeps being pressed from the FN
layer.

Note: the bug works the other way around as well with the original code:
1) press A/B key: at this point scan code for A press is sent
2) press FN: at this point two things are sent: B press, and A release
3) release A/B key: at this point B release is sent.

My code fixes this scenario too.

Cheers,
Andrei

[webwit]

Thanks! Yeah, that exactly matches what I experienced as a layman user.

[Dikkus]

Great that this was resolved. Thanks a lot, pandrew!

[tentator]

The previous explanation was very heavy on technical details, I also wrote out the differences in python-ish pseudocode to
[...omissis...]

Cheers,
Andrei

Cool Andrei!! Great contribution!
Great Open Source!!!

Now I'd be super curious to know your opinion on my secret wish: being able to implement in xwhatsit dual role keys (like "space-fn" if you know it or if you take a fn key like shift, I'd like to make it output arrow up if only pressed/tapped, while continue to be a shift when pressed and hold, because a predefined timeout has passed while holding that key, say 150ms). )))

Again thanks for the contribution, but it's a bit late now so will test it tomorrow,
tent

[consolation]

Arrived this Monday morning as I was heading out to work, for the last day before lock down, lock down came a day early I guess... mixed feelings about the plague today Kitted it out with some terminal keys for now. I will write a bigger comment with some thoughts, but, overall; it's blimin' amazing to type with and the build quality is excellent. I thought my Norbaforce was a chonk, the F77 is a mega-chonk.

Apologies for the potato camera:




Quick question, when entering capsense utility, the matrix indicates that 16,2 15,3 16,5 are pressed. Are they meant to be detected as such? They do not seem to map to anything on the keyboard.

PS. Just a brief fyi: When running on a USB 3.1 port, I have to set the voltage threshold to 127, USB 2 was happy with 123. 122 and lower had random keys detected. So far no other issues, once that was adjusted. To be clear, once set to 127, it seems to run fine on USB 2 as well.

PPS.

Alright, found another small bug, really hope this is the last one I have to fix
...

If I follow your fix, you are using two bits to set the layer? That puts a hard limit of layers at four?
Asking as I had vague plans to change the source code to allow more layers. I use multiple languages, so tend to just have different layouts on different layers, two layers per language - it adds up quickly. But, tbh, I can easily switch a keyboard at home and the F77 isn't something I'm going to carry around in the laptop bag; so if I read your change correctly, I'll save myself the hassle.

[pandrew]

Now I'd be super curious to know your opinion on my secret wish: being able to implement in xwhatsit dual role keys (like "space-fn" if you know it or if you take a fn key like shift, I'd like to make it output arrow up if only pressed/tapped, while continue to be a shift when pressed and hold, because a predefined timeout has passed while holding that key, say 150ms). )))

I think it's possible, if I understand your idea right. It would be possible to do something like:
1) When the key is pressed, nothing is sent to the host
2) If the key is released, before time T passes, a quick immediate sequence of key A press-release is sent to the host
3) When time T passes, and the key is still pressed, then key B press is sent to the host (or layer is changed, if it's a keyboard-side function key)
4) When the key is released AFTER time T passes, then key B release is sent to the host (or layer is restored, if it's a keyboard-side function key)

Is this what you thought about? I just want to say, that the quick-press key A might feel sluggish in this case, since it's only sent on key-up.

If I follow your fix, you are using two bits to set the layer? That puts a hard limit of layers at four?
Asking as I had vague plans to change the source code to allow more layers. I use multiple languages, so tend to just have different layouts on different layers, two layers per language - it adds up quickly. But, tbh, I can easily switch a keyboard at home and the F77 isn't something I'm going to carry around in the laptop bag; so if I read your change correctly, I'll save myself the hassle.

Yes, but you can always tweak it to be more to your liking. For example if debounce threshold can never be more then 7, then you can take a bit from the scanState part, and add it to the layer selection. You just modify the access macros in scan.h, and you can reshuffle storage as you whish.
Or -- I just double-checked, there is actually 379 bytes free in SRAM. An array of [cols][rows] size is 128 bytes. So I was wrong above about not being enough space for a separate sticky data structure. (My initial data structure idea didn't fit though, but that was different. So just modifying the acces macros, you can put the different fields in different variables.

The bigger problem that will make it harder for you to add more layers is not my change, but the lack of space in EEPROM.
As it is designed currently, each layer occupies 128 bytes of eeprom, and right now, eeprom has only 72 bytes of free space left.
It contains as follows:
1) Layer configuration for 4 layers: 4 * 128 bytes
2) Macro configuration: 424 bytes
3) Other smaller stuff: 16 bytes

Some of the things you could do to go around this limitation is to:
1) rip out macros (or to reduce macro capability)
-or-
2) move layer configuration into program space, where it won't be configurable anymore, but hardcoded. There's around 20K of program space left, so there's plenty of room there. But then you will have to rebuild your .hex file everytime you change layer layout, and it will be more complicated to set it all up manually without the aid of the GUI.
-or-
3) there may be other ways to shuffle things around, so you have more space for layers.

Saying all that I just picked up on what you said -- 2 layers per language -- what do you mean by language? Do you mean keyboard layout for special characters such as áéóúöőűíșâățî?
If yes, then there aren't any usb scancodes for those, languages like that can't be hardcoded into keyboards, those are always handled by the OS.

[tentator]

Hi Andrew,
yes correct, this is how dual action key's are supposed to work indeed. And if the time T is something quite low like 150ms max 200 it should not appear sluggish at all. I use this setup in many of my keyboards with Q/TMK (to implement "space-fn" which activates another FN layer when space bar is pressed and kept pressed; and RSHIFT is dual-action: shift when pressed and hold, instead it's up arrow when pressed and released within time T) and it never was sluggish for me at least..

It's documented here: https://docs.qmk.fm/#/feature_advanced_ ... id=mod-tap
And other interesting discussions about it here: https://thomasbaart.nl/2018/12/09/qmk-b ... d-actions/

But it definitely does not need to be implemented as fancy as that.. it's as you said and actually limited to modifiers only when held, normal key when tapped. What I see as of today in the macro section of xwhatsit does not have any similarity to this IMHO.

For the international/language specific characters I think the best is to work with ALTgr and OS layouts/Xresources or so.

Kr,
tent:wq

[webwit]

Did some heavy testing with Andrew's fix today. Not a single issue!

[consolation]

Saying all that I just picked up on what you said -- 2 layers per language -- what do you mean by language? Do you mean keyboard layout for special characters such as áéóúöőűíșâățî?
If yes, then there aren't any usb scancodes for those, languages like that can't be hardcoded into keyboards, those are always handled by the OS.

Yes they do, but the location of umlauts, macrons etc changes with each "keyboard" plus some may use different codes to produce same symbols etc. (not to mention the issue that some keyboards expect different layouts; the difference between ISO, ANSI and AZERTY being the most common) So, to not drive myself mad, I tend to keep my generic mostly ANSI layout and keep the modifiers on same keys - by having the keys send different scan codes that produce same modifiers. It's really arse-backwards. But there are couple pieces of legacy software that requires me to switch to a particular layout to work. OK, the best way to describe it is; I don't want the layout change in OS to change my keyboard layout, so I kludged my way around it.

Here is the thing, if I was more efficient, there could probably be one base layer, then the typical function layer, then the remaining two could probably hold all my changes with a some rare stuff sacrificed. Or, I could hard code the base layer into the firmware and get the extra layer that way? So, only the function layers are programmable?

Honestly with the amount of stuff on my to do list, it's not something I see myself tackling soon. There is a keyboard I am happy to use for those days that has everything setup - it's starting to look like a lot of work for an issue that happens couple times a week lol.

[pandrew]

Yes they do, but the location of umlauts, macrons etc changes with each "keyboard" plus some may use different codes to produce same symbols etc. (not to mention the issue that some keyboards expect different layouts; the difference between ISO, ANSI and AZERTY being the most common) So, to not drive myself mad, I tend to keep my generic mostly ANSI layout and keep the modifiers on same keys - by having the keys send different scan codes that produce same modifiers. It's really arse-backwards. But there are couple pieces of legacy software that requires me to switch to a particular layout to work. OK, the best way to describe it is; I don't want the layout change in OS to change my keyboard layout, so I kludged my way around it.

I still don't understand exactly what you are trying to do.
I mean I can imagine the following two scenarios:
1) Interchange Z and Y keys on some layers
2) In the standard windows hungarian layout the key ";/:" produces é/É. You could create a layout that maps some other key key to send the keycode for ";/:" but that would only work in combination with OS-selected hungarian layout.
You can't send any scancode for é/É directly, because there is no such scancode, it just doesn't exist. Please see page 53 from here for a reference of all the available scancodes: https://www.usb.org/sites/default/files ... 1_12v2.pdf
3) You could do something that is only windows-compatible. Windows has Alt-codes that you can use to input lots of different characters: https://tools.oratory.com/altcodes.html so you could re-play the keypress-combination: press ALT, press 1, release 1, press 3, release 3, press 0, release 0, release ALT. And that would send an "é". But there are a couple limitations with that. One is that it's Windows-only, The other is that you have to have numlock active for it to work consistently. There are alternatives for linux and mac, for example on linux you can do Shift-Ctrl-U followed by unicode code. On mac I think there is a completely different system, but there isn't a single standard.

Does what you want to do sound similar to the scenarios above? If not, could you clarify with a specific example? By saying exactly which scancodes you want to use and what should be the outcome.

[consolation]

I'll pm you detailed example tonight and why; last day before level 4 lock down here in NZ, off to harvest resources and keep kids entertained. But, essentially you got it exactly right in 2) When I have to switch to a kb in OS, different layers emulate that kb - but keep the actual layout same, as much same as possible. All because of software from c.2000 that I'm forced to use.

[pandrew]

I use this setup in many of my keyboards with Q/TMK

I'm wondering if it was possible to port QMK to the xwhatsit controller. Just taking a look, it does seem like they support some atmega32u2 based keyboards. So I think it might be possible. I don't know much about QMK though, so it could take a while, also it could be quite limiting that I don't have an xwhatsit keyboard at the moment to test on. (I ordered my model F with printed keys, so it's gonna be a while until I receive mine)

[tentator]

I use this setup in many of my keyboards with Q/TMK

I'm wondering if it was possible to port QMK to the xwhatsit controller. Just taking a look, it does seem like they support some atmega32u2 based keyboards. So I think it might be possible. I don't know much about QMK though, so it could take a while, also it could be quite limiting that I don't have an xwhatsit keyboard at the moment to test on. (I ordered my model F with printed keys, so it's gonna be a while until I receive mine)

Nah, I'd say forget it, many attempted and it was never possible so far.. i think something related to capacitive sensing missing in qmk... surely it's a wet dream of many here since qmk is the de facto standard.. do you know qmk already?

otherwise I think the best put effort would be if it would be possible to implement space-fn in current xwhatsit firmware.. especially since you seem to have already just fixed one of the most relevant bugs!

And i do really like your elegant way you did it by better making use of that byte per key! Really cool!.

What do you think?

[pandrew]

Nah, I'd say forget it, many attempted and it was never possible so far.. i think something related to capacitive sensing missing in qmk... surely it's a wet dream of many here since qmk is the de facto standard.. do you know qmk already?

I just started reading a bit about QMK. I actually think QMK is quite possible. So far I don't see a reason why it wouldn't work. Capsense code would have to be reimplemented from scratch, because QMK is GPL2(+) and xwhatsit is GPL3(+). Yes, I know, they are compatible, but if we want main QMK to take the contribution, and it to show up in the main configurator tool, it's likely they won't accept a GPL3-only contribution. But I don't think implementing capsense is all that hard. Talking to the 74595 shift registers is simple. Talking to the DAC that generates the voltage threshold can't be that hard either. But it wold take at least a few days to implement. Testing it and tuning it could also take a while.