This is a controller that doesn't do much.
It doesn't have settings that require reflashing.
It doesn't have any trouble with "non-US backslash/pipe" key. Or hash/tilda of the same origin.
It doesn't have thresholds that drift.
It doesn't have any trouble with BIOS or UEFI, on windows or mac. Nor with anything else that knows what an USB keyboard is.
It doesn't have tap dance macros - and will never will, I give you my word.
It doesn't have NKRO - only 62 keys at once. Be careful testing that - 62 standard 65-gram keys is four kilograms of force, never mind the strain on your tendons.
As you can see - no fun at all.
It doesn't come pre-packaged either - you'll need to roll your own. So, you'll need $15-ish, some wires, a soldering iron, curiosity (as in "being curious about how things work"), and, for a couple hours, a windows machine (VM's fine).
Oh, and one last thing. You probably guessed it by now. It doesn't have warranty of any kind, expressed or implied. You might be lucky enough to burn down whole city with it. That's not my fault.
It reads capacitive, inductive and ohmic switches (no deghosting), and probably SD microswitch by now.
The Repo's README.md contains bootstrapping instructions. this topic has a guide with pictures for the illiterate (which is another bootstrapping problem, not covered here).
FlightController (the controllers' controller), precompiled: Windows and OS X. Linux users need to compile (but at least ubuntu has all dependencies pre-built. OSS has it's benefits, too).
Hardware used is CY8CKIT-059 prototyping kit. Since the kit is $10 - there's no GB for building the controller. It's hard to beat $10 price. If, however, project MF is revived or Ellipse will decide to switch controller to CommonSense - there will be custom hardware, too.
Theory of operation:
If you want to make you want to claw your eyes out know more about capacitors, read it there, in all it's Comic Sans glory. There probably other guides, but it's hard to find one that explains energy storage. It even has pictures - but I have something better. Pr0n.
Spoiler:
This is the circuit simulator for one capacitive key. Click it. Look at the current going back and forth. Press "Any" key, see how the signal changes on the 'scope. You will notice that when the key is pressed, the line returns to zero much slower. It does so, because larger capacitor can transfer more energy from one pulse. But it still does.
"What attracts is to zero?" one might ask. The answer is "load resistance". Increase the resistance and you'll have less decay, making pulses more square. So, if you want to know the key state and you want to use comparators - eventually, instead of measuring just that the peak has occured (it happens every time in the ideal world, btw - the comparator just misses it if it's short enough.), you'll soon find yourself measuring the time signal spends above the threshold, and in the end of it even that doesn't help because the waveform you're looking at exactly matches the driving waveform.
Drag the "Load" slider all the way to the right to observe that firsthand.
You will notice that the wave slowly drifts down. It does so to reduce DC component of the signal - because capacitors effectively block DC. If you click the generator and set duty cycle to 50% - you'll get the perfect symmetry eventually
But The Real World is a bit more complex than that. Because in the real world there is a capacitance between the ground and the sense line. It is called "parasitic capacitance".
And this is where things get interesting. NOW THROW THE REALITY SWITCH. If you are going to do that in a simulator already open - Please drag the "load" slider all the way to the left before that, or you spoil the surprise.
Now try to press any key.
See the difference? That's right, peak height depends on capacitance now. That's because the key charges the parasitic capacitance now (charging capacitor has zero* resistance), and the load discharges it. But the key capacitance limits the amount of energy that can be transferred thru it. Because charge equals capacitance times voltage, and after capacitor is fully charged it doesn't conduct anymore current.
BUT.
If the parasitic capacitance is larger than the key capacitance - key cannot fully charge it. So the voltage on the parasitic capacitance will be lower than driving voltage. Or equal to it, if the key capacity is more than parasitic.
So if you ramp up the load resistance again - you will see nice and square signal, proportional to the key capacity. With large enough load resistance this voltage can be measured at leisure, not involving sample-and-hold circuits or any other complicated stuff. All you need is an ADC and an muliplexor (or a separate ADC for every channel if you want to go baroque).
So yes, essentially I'm using the keyboard as capacitance to voltage converter.
Spoiler:
Now, if you press the key in simulator at exactly the right moment, you'll get huge DC offset. To prevent that I'm tying the inputs to the ground when not used. PSoC allows the pin to be analog in and digital out at the same time - but on other devices this can be achieved by connecting 2 pins together, provided the MCU can have the pin configured as "open drain, drives low".
In theory, there's no difference between theory and practice.
You would expect that it would work, wouldn't you? Everything will be nice and smooth, like it is on the XT keyboard (if you cut the row trace and connect oscilloscope to the PCB end of the cut):
xt_rows.jpg (40.87 KiB) Viewed 27802 times
Nope. Reality is much, much worse.
You discover it gradually. First, there's this:
monitor-released-explained.png (18.13 KiB) Viewed 27802 times
4 on the real key is pure luck. It rarely happens. Unless you start scanning the matrix all the time, about 1k times per second. Then you'll notice that it happens about 5 times a second.
Then you'll notice the amazing signal level connsistency.
monitor-pressed-explained.png (21.52 KiB) Viewed 27802 times
The "?" key pressed gives off less singal than the lucky key from previous example. Enter gives about as much - but there's a twist, it is noisy as hell. It pretends to be 1 in resting state, but if you're lucky, couple times a second it's 3.
Backspace gives a healthy strong 9. But it has a downside - sometimes it lights up half of it's row. Including F10 and especially F11, which will toggle your browser's fullscreen mode. What a nice surprise.
Then you get out the scope (cheap, USB one) and see that signal levels, indeed, different and nicely square.
key_released.jpg (57.87 KiB) Viewed 27802 times
Well, almost. They have those Burj Dubais at the start of the pulse and at the end of it. If you zoom them in, they have classical decaying sinusoidal form. This is called "ringing". RF designers go a looong way to avoid that.
OK, not a problem, just wait a couple microseconds for it to settle and then measure!
key_pressed_explained.jpg (65.77 KiB) Viewed 27802 times
Nope.
And then you zoom out and see...
teh_horror.jpg (70.01 KiB) Viewed 27802 times
Teh horror. Those things are everywhere. EVERYWHERE!! But they're short.
Also is looks like they sometimes happen on their own. Look, that's how driving pulses positioned relative to each other in neighboring columns:
adjacent_columns.jpg (55.5 KiB) Viewed 27802 times
(here I remembered that I actually try to scan even/odd lines 4 columns apart.)
Last edited by DMA on 26 Jun 2020, 21:16, edited 19 times in total.
HuBandiT wrote: ↑Just a quick drive-by suggestion without really understanding the situation: would the Microchip PIC CTMU perhaps be of use?
Thank you for pointing me there. AN1375 finally explained to me how all those capacitive touchscreens work and what limits number of fingers recognized at the same time. There's easter egg in there - look at items 47 and 48.
CTMU measures capacity by pumping known charge (known current * known time) into the external circuit and measuring resulting voltage between pin and the ground. Doesn't matter how many capacitors are there - the bigger the capacitance, the lower the voltage.
this one pushes as much charge into the matrix column and measures how much of it got out of the row line. And compares "how much I usually get for this column" vs "how much I've got this time". Doesn't care about absolute values - don't know if it even can figure those out. Most probably not.
But it's simpler. ADC is one of the components of CTMU. Not a central one
Managed to transfer config to the device. No saving to EEPROM yet. That's for tomorrow.
Almost had to resort to external programmer - changed commands enum and forgot which command code is "go to firmware update mode". Was able to bruteforce it back to the bootloader though.
In other news switched Qt IDE to FakeVim (Yes, I'm a connoseur of ViM) - but PSoC creator doesn't have that and Oh Hilarity. Switching to different editor every 5 minutes is a headache. Staves off Alzheimer, though. At least they say so.
Wireshark sucks. Spend almost an hour staring at the MAGIC - the host sends an empty packet to the keyboard AND IT TURNS ON THE LEDS.
First I thought that the keyboard somehow remembers you pressed caps lock and lights it up when instructed by a host to do so.
But then I connected a second keyboard to my machine and the same empty packets were sent to that keyboard and it recognized the LED status.
And then I tried to search Teh Internetz for this and found ONE post. Luckily the author was kind enough to go back to it after a _year_, literally, and post an update that yes, wireshark indeed completely misses the data stage of control transfers.
And this kind of described at usbpcap site, but in a very vague way. It says it doesn't support split transactions - and it seems that control transfers consist exclusively of those.
It is a first time wireshark fails me. I feel betrayed!
Luckily, HID reports seem to be dumped intact. (I'm doing all that because I've accidentally noticed that LEDs on my controller use interrupt transfers, and on the apple keyboard (and unicomp, as later testing shows) it uses control transfers. Trying to figure out where on which stage exactly I fucked up - and if it's me or cypress).
Suddenly there's lots of ringing - pretty high-voltage ringing at that. Keypress generates sparks across the whole row somehow. Luckily they are very short, but debouncing is definitely needed.
Probably has something to do with USB report frequency - time to get the scope out.
Also found out that "?" key has a very weak signal. What's worse, the dot key has very strong one, and it creates quite strong signal on the former. Don't know yet what to do and what causes this. May be scope will shed some light on that too..
[Nerd note: I've never heard of a console report interval measured in a unit for electrical conductance.]
That's weird about different keys being weaker or stronger than each other. Can you estimate the SNR for the '?' key, given the noise from the '.' key and its own weak signal? I'm just wondering how weak/how much noise we're talking about here.
Turns out I accidentally disabled row discharge logic, so those sparks were the charge left over from the key press.
Fixed now. Will leave non-interlaced scan there in config anyway - beam springs should not have that much crosstalk but have more columns, this will shorten the scan time little bit.
Ratfink wrote: ↑[Nerd note: I've never heard of a console report interval measured in a unit for electrical conductance.]
What would you expect at 5 am after 2 hours of fruitless attempts to understand what's going on?
At least it's not mSv!
Ratfink wrote: ↑
That's weird about different keys being weaker or stronger than each other. Can you estimate the SNR for the '?' key, given the noise from the '.' key and its own weak signal? I'm just wondering how weak/how much noise we're talking about here.
SNR is literally negative. Noise floor is 4-5, Pressing '?' raises voltage to 13-15, pressing '.' raises it to 16-18. In interlaced scan mode, though, noise floor is 2-3, pressing '?' gives 14-16, pressing '.' is still 2-3.
Units are 2mV, if my math is correct. 2.048V/10 bit.
I've noticed something interesting, btw. Next column to '.' there's something that looks like a button which is constantly pressed, but with weakest signal of them all. 10-11 units.
This can be used for automatic calibration methinks. Just scan the matrix couple of times, determine the average voltage level, determine what keys stand out, scan again and select maximum from the keys that are below average and minimum from the keys above it. Can even do this at powerup. Even in worst case - keys pressed at boot time - this will allow for reliable sensing.
Will try to do that today.
Any reason why we don't just charge all capacitors to a nominal value while idle, and then just keep measuring voltage across them to sense when capacitance changed due to actuation, topping them off as needed occasionally?
HuBandiT wrote: ↑Any reason why we don't just charge all capacitors to a nominal value while idle, and then just keep measuring voltage across them to sense when capacitance changed due to actuation, topping them off as needed occasionally?
We'll need to measure time very precisely, as opposed to measure voltage pretty coarsely (by today's standards, at least).
Also I cannot figure out how to measure separate capacitors in pretty vast network. I'm shorting everything to ground, then I untie rows from the ground and drive _one_ of the columns up. So there's only 8 capacitors which are measured at any time, and every one of them is connected to a different pair of pins. Trying to measure the whole matrix at the same time will require significant math - like MRT or x-ray crystallography
HuBandiT wrote: ↑Any reason why we don't just charge all capacitors to a nominal value while idle, and then just keep measuring voltage across them to sense when capacitance changed due to actuation, topping them off as needed occasionally?
We'll need to measure time very precisely, as opposed to measure voltage pretty coarsely (by today's standards, at least).
Also I cannot figure out how to measure separate capacitors in pretty vast network. I'm shorting everything to ground, then I untie rows from the ground and drive _one_ of the columns up. So there's only 8 capacitors which are measured at any time, and every one of them is connected to a different pair of pins. Trying to measure the whole matrix at the same time will require significant math - like MRT or x-ray crystallography
It's completely possible I'm fundamentally misunderstanding the situation. But what I gather is that we have an N×M matrix of capacitors; assuming we can connect to ground one pin of a row of capacitors at a time, as well as we can connect the other pin of a column of capacitors one column a time, we can address each single capacitor individually, and then measure its voltage, and charge/discharge it individually.
For simplicity, let's assume the keyboard starts up with all keys released. Now the microcontroller can address each capacitor individually and charge it up to a nominal value. After they are charged, the microcontroller keeps scanning the capacitors: addressing each of them individually in turn, and measuring the voltage. The moment one of the keys is pressed and the hammer is starting to approach the capacitor electrodes on the PCB, the capacitance will start to change, and, since there is no transfer of energy, this will cause the voltage across the capacitor to change as well. The microcontroller, during scanning, can sense the changed voltage (debounce if necessary). When the key is released, the capacitance - and together with it the voltage - will return to its resting value.
Of course since this is not a theoretical, perfect circuit, there will be non-zero current flowing, so the capacitors will gradually discharge over time. So the microcontroller, after measuring the voltage on the capacitor, can also make a comparison: if the voltage has fallen below a threshold, top up the capacitor to the nominal voltage again (some logic will have to be added to recharge long held-down keys to a proper value as well). Or, just characterize/calibrate the rate of charge/energy per capacitor, and instead of charging to a target voltage (and taking into account whether the key is supposed to be up or down at the moment) recharge each capacitor just as much as it actually needs to replenish its energy (this would be particularly easy to do with the constant current source of the CTMU and a timer/PWM).
With this arrangement, energy flow in the system would be minimized (since we will try to keep all capacitors in a steady state electrically) - which not only means reduced EMI and reduced current consumption, but also enables a much faster scanning frequency, which in turn enables a very clean and reliable debouncing as we would be scanning fast enough to observe the hammer approaching and bouncing in "slow motion".
Hmm. Testing is definitely needed.
EMI would definitely not be a problem tho - PCB sits in a metal cage, after all
Need to find a way to get the current consumption accurately.. I suspect we're talking about uAmps here.
Scanning rates above 1kHz is impractical, as I found out - you can only send 1k reports per second over USB, is seems.
DMA wrote: ↑Hmm. Testing is definitely needed.
EMI would definitely not be a problem tho - PCB sits in a metal cage, after all
Need to find a way to get the current consumption accurately.. I suspect we're talking about uAmps here.
Scanning rates above 1kHz is impractical, as I found out - you can only send 1k reports per second over USB, is seems.
Technically, you could send 8k reports per second over high-speed (480Mbit/s) USB (USB 2.0).
But that's not where the higher scanning rate comes into play - but instead for accurately detecting order of keypresses as well as debouncing. If we accept the 0,4 m/s finger speed figure Cherry tests their mechanical switches with, an 1kHz scanning rate results in 0,4 mm mechanical resolution, one tenth of key travel - if you touch type quickly, and roll keys in a way that they are following each other closer than this distance (which I think can happen for fast typists), you risk transposing the order of keys merely due to your low scanning rate. And then let's not begin on debouncing mechanical keys. (As a tangent, if a keyboard scans keys at 400 Hz, the mechanical resolution becomes 1 mm, with a 100 Hz scanning rate it becomes 4 mm, with 50 Hz it's 8 mm. Think of this the next time you fault yourself for being a bad typist for apparently transposing letters when typing fast.)
So while a reporting rate of 1 kHz or faster is probably only interesting to gamers (and researchers in neurology!), key scanning itself could very well benefit from a faster scan rate (if otherwise technically feasible).
HuBandiT wrote: ↑Technically, you could send 8k reports per second over high-speed (480Mbit/s) USB (USB 2.0).
Well, after I get this one working, it would probably make sense to move to another controller which understands USB 2.0 high speed.
So while a reporting rate of 1 kHz or faster is probably only interesting to gamers (and researchers in neurology!), key scanning itself could very well benefit from a faster scan rate (if otherwise technically feasible).
I think.
Yeah, hit that landmine today.
Turns out there's a lot of noise in the signal. I mean if you don't even touch the keyboard, some keys shoot up to 50 units for a very brief time - which is like 3x the signal "?" key gives off when pressed.
4x oversampling mostly takes care of that, but the row with BkSp in it is especially noisy for some reason and, if you're lucky, quickly hitting BkSp several times produces up to 5 keypresses which break through the averaging momentarily.
Because of this I'm rewriting this post from the notebook's keyboard - BkSp triggered developer console + full screen + something else in Chrome
Per-key calibration and debouncing are still needed, it seems.
DMA wrote: ↑Turns out there's a lot of noise in the signal. I mean if you don't even touch the keyboard, some keys shoot up to 50 units for a very brief time - which is like 3x the signal "?" key gives off when pressed.
Since you yourself said the electronics is sitting in a metal cage, you should not be experiencing that level of noise (as in EMI). I suspect you have a bug: either an actual software bug, or you did not correctly account for the dynamic behaviour of the electronics in your model. Since it only happens sporadically, that smells to me like some kind of a timing inconsistency issue.
In theory, there's no difference between theory and practice.
You would expect that it would work, wouldn't you? Everything will be nice and smooth, like it is on the XT keyboard (if you cut the row trace and connect oscilloscope to the PCB end of the cut)?
I'm too lazy to reupload all the pictures - look in the original post, under the second spoiler, it's nicely illustrated even.
I remembered that I actually try to scan even/odd lines 4 columns apart, so it's a bit wrong though - the pulses are much closer.
Will check tomorrow with different delay configurations - though I know from today's experiments that putting 200 us delay after each column scan does not help. Will probably also post less JPEGy pictures, though don't count on that.
Right now I think about double scanning on the spot and discarding the higher value. Or push the current results to github and abandon this whole thing.
Or just implement plain old debouncing. It doesn't seem to be needed though - looks like the sheer mass of the flipper, force of the spring and relatively large detection distance do the debouncing trick in hardware.
I plan on setting up ADC to the maximum sampling frequency it can (they promise 1Msps, but I doubt I'll be able to record that fast) and see if I can detect the ongoing flipper in flight. Would be interesting. Single-channel scan should get 5us resolution, I think - more if I'm lucky, though I don't believe in less than 3us cycle.
Lovely illustrations. However, they only reinforce my conviction that there is simply too much dynamic behaviour going on, and you'd be better off by the more static arrangement I suggested. (You might need a different measurement architecture for that though.)
Don't give up. Simplify the problem. Try "scanning" just one single key. Do the issues (noise) still happen then?
Also, do you have schematics up somewhere? Maybe someone better than me with analog could take a look. More eyes squash more bugs.
HuBandiT wrote: ↑
Don't give up. Simplify the problem. Try "scanning" just one single key. Do the issues (noise) still happen then?
Also, do you have schematics up somewhere? Maybe someone better than me with analog could take a look. More eyes squash more bugs.
No, of course I wouldn't give up. It's almost working and I have workaround(s).
You can't scan a single key - or rather you can, but you'll make whole thing 4 times slower. I'm already doing what can be done - that is, grounding the alternating row lines to prevent crosstalk (which is working extremely well, btw).
schematics is "take a prototype kit, connect to 122key model F". This is a very early stage.
Ringing is a result of impedance mismatch, I know that. I also know that matching it would require several full-size prototypes and will need to be done on every PCB design anew. So more universal solution is needed (also this problem would surface no matter the measuring method).
I think I'll just implement the glitch filter (double scan, get minimum value) and be done with it. Or add a very low ESR small capacitor near the input (low-pass filter) to short out those transients - but I want to keep BOM as low as possible. Also all those ribbon cables and other stuff protruding out of Faraday's cage will only add noise - so countermeasures are needed anyway.
I also plan moving to custom schematics for ADCs instead of using the scanning ADC component. I have 2 SAR ADCs on a chip, after all - and in the custom case I have a bit more influence on mux design.
getting xwhatsit's controller to poke it with the scope would be beneficial at this stage. I suspect that it's the same hell - but need to confirm.
DMA wrote: ↑You can't scan a single key - or rather you can, but you'll make whole thing 4 times slower.
I meant to suggest, instead of checking/scanning all the hundredsome keys, just for purposes of simplifying the problem, scan only a single key, and observe behaviour.
DMA wrote: ↑I'm already doing what can be done - that is, grounding the alternating row lines to prevent crosstalk (which is working extremely well, btw).
schematics is "take a prototype kit, connect to 122key model F". This is a very early stage.
Ringing is a result of impedance mismatch, I know that. I also know that matching it would require several full-size prototypes and will need to be done on every PCB design anew. So more universal solution is needed (also this problem would surface no matter the measuring method).
I beg to disagree. If you keep the caps charged and topped up all the time (my suggestion above), instead of charging and discharging them during each scan cycle (as you do currently), you would have much much fewer transients to worry about.
On another note, could the ringing you observe maybe caused by the parasitic LC circuit of the C of the switch (and parasitics) and L being the parasitic inductance of your PCB traces and cables? I don't think you can have ringing without some kind of L.
I wish I had more time for this - is it difficult to get a copy of the KB hardware to play with?
First off ... i'm enthusiastically following this. i would love something simpler, smaller, and cheaper than xwhatsit. Remember that the UI is important. Xwhatsit did a pretty good job, and Aikon too.
i know it is tempting to call mutual-capacitance electrodes "rows" and "columns" but we should probably try to avoid that. Design-wise, the two electrodes are often not interchangeable. Also, i believe that IBM and Topre implemented them differently - IBM went with "rows detect" but i think that Topre went with "columns detect". Calling the electrodes rows/columns is just too ambiguous.
The different documentation sources call them different things ... "Tx", "signal", or "drive" for the electrodes that do that - "Rx", "sense", or "detect" for the other.
Last edited by wcass on 22 Jun 2016, 01:11, edited 1 time in total.
wcass wrote: ↑i know it is tempting to call mutual-capacitance electrodes "rows" and "columns" but we should probably try to avoid that. Design-wise, the two electrodes are often not interchangeable. Also, i believe that IBM and Topre implemented them differently - IBM went with "rows detect" but i think that Topre went with "columns detect". Calling the electrodes rows/columns is just too ambiguous.
The different documentation sources call them different things ... "Tx", "signal", or "drive" for the electrodes that do that - "Rx", "sense", or "detect" for the other.
Interesting! Do you maybe have any pointers to documentation on this? (Or is it already in the IBM patent(s)?) And/or do you have a suggestion as to what DMA should change or do differently?
wcass valid point.
drive/sense makes more sense, that's true. Don't have much interest in topre though - and IBM seem to always have driving columns
Anyway - will need to stick with software workarounds - hardware design will be too expensive. QFN packages are mighty hard to hand-solder, so every iteration will not only require a new PCB, but also assembly. I can do etching at home, but soldering SMDs that small is outside of my skill.
HuBandiT wrote: ↑Interesting! Do you maybe have any pointers to documentation on this? (Or is it already in the IBM patent(s)?) And/or do you have a suggestion as to what DMA should change or do differently?
I can't program so i can't advise on that. I'm OK at hardware design though - capacitive switch matrix and controller PCB layout. If you give me a controller schematic, i'll draft it up.
DMA wrote: ↑Anyway - will need to stick with software workarounds - hardware design will be too expensive. QFN packages are mighty hard to hand-solder, so every iteration will not only require a new PCB, but also assembly. I can do etching at home, but soldering SMDs that small is outside of my skill.
I used to etch at home but for small PCB prototypes, OshPark does a great job; 3x your design for $5 per square inch including world-wide delivery (though slow delivery outside of USA). Your controller should be under 3 square inches, so about $5 per controller for the PCB. I get solder paste stencils from OshStencil (about $5) and use a hotplate for reflow. Pick-n-place is the hardest part of assembly now. I have a rework station, but prefer the hotplate.
HuBandiT wrote: ↑Interesting! Do you maybe have any pointers to documentation on this? (Or is it already in the IBM patent(s)?) And/or do you have a suggestion as to what DMA should change or do differently?
I can't program so i can't advise on that. I'm OK at hardware design though - capacitive switch matrix and controller PCB layout. If you give me a controller schematic, i'll draft it up.
Sorry, I meant do you have info on the electrical difference between using the capacitors in capacitive keyboards one polarity vs the other polarity.
I am having a strong inkling to have a go at this with Microchip PIC microcontrollers, but I don't have a distilled design yet, and I wouldn't want to piss with other people's private parts onto the high voltage lines so to say. If however you guys feel experimentative enough, I could be interested in having a go.
But first off, do we know the capacitance figures (on/off, or at least a range) we are working with?
