CommonSense quick-start / noob's guide

snacksthecat · 24 May 2019, 03:09

Part 0: Intro

Foreword

I wanted to make this guide because CommonSense is an amazing piece of software but it has a steep learning curve (at least in my opinion). I think it would be awesome to see more people try it out. So I'm putting together the bits and pieces that work for me. And I'll also try to explain why we're doing xyz. I apologize if I get anything wrong; please let me know if you see something that isn't true (I'm definitely not an expert but I think I understand the basics fairly well).

The firmware offers lots of flexibility. Too much to go into in this guide. Just be aware that this is intended to be a streamlined guide aimed at getting someone up and running.

Since you need Windows to build the firmware anyways, this is going to be Windows focused. Also know that it's possible to fry the board.

Other resources you'll want to reference during this process are...

The CommonSense thread on Deskthority viewtopic.php?f=7&t=13988
The CommonSense github repo https://github.com/dmaone/CommonSense

Down the Rabbit Hole

If you're ready to go, the first step is easy enough. Just download the github repo and extract it somewhere on your pc. We're going to start on the hardware side of things but you'll need these files soon enough.

https://github.com/dmaone/CommonSense

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Part 1: Kit and Base Wiring

CY8CKIT-059

The controller is built on a development board from Cypress called the CY8CKIT-059 (similar to how you would build a TMK/QMK keyboard on a teensy or pro micro). The kit is actually really cheap at ~$10. You can purchase one of these boards from the usual suspects like Mouser or DigiKey. Here's the link to the product page on the manufacturer's website:
https://www.cypress.com/documentation/d ... rammer-and

Maybe it's obvious but I feel it's worth mentioning; The USB-A side (side you plug directly in) is used when you're flashing the board with firmware. The micro USB is what you' plug a cable into when you want to use or configure the keyboard.

Base Wiring

So the next step will involve some soldering. Certain pins on the board are used as references so they need to be connected the power / Gnd. This is all explained in the CommonSense github repo but it took me a while to digest it so I wanted to simplify it as much as possible. Oh and the pin notation is also worth mentioning. On the board they are printed as (for example) 1.0. On the computer, we'll refer to them in a different notation. The pin labeled 1.0 would be referenced on the computer as 1[0].

These are the base connections you need to make:

0[2] connected to the nearest Gnd
0[3] connected to the nearest Gnd (BY A SEPARATE WIRE)
0[4] connected to the nearest Vdd

So it should look like this:

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Part 2: Sodering rows and columns

Planning

Now that the base wiring is done, it's time to connect the controller to the rows and columns of the keyboard. The way that CommonSense works is by strobing the columns. When a key is pressed, it's detected by the rows, which are set up to sense this strobing.

The current version of CommonSense has a limitation of 8 rows and 24 columns. If I'm not mistaken, the code can be tweaked to support more rows, but that's a bit beyond what I know how to do so I'll not be going into that.

You do have flexibility to shuffle around the pin assignments, but there are some rules about which pins can be assigned for which purpose and some pins you might be planning to use may already be assigned and that might cause your firmware to fail the build step. I found it easiest to use the default pin assignments if possible. Those assignments are:

Rows: 2[0], 2[3]-2[7], 12[7], 12[6]
Cols: 1[0]-1[7], 3[0], 3[1], 3[3]-3[7], 15[0]-15[5], 0[0], 0[1], 0[5]

DMA notes in the readme that if you need to use more than 17 columns, you must desolder a few components from the dev board. These components are C41, C42, and R5. DMA elaborated on this that it's possible to do without desoldering by simply substituting for pins which do not have external components attached to them. Again, this is only if you need more than 17 columns.

Keyboard

So now we know which pins on the board we're interested in, but what do these need to connect to on the keyboard? Well first, you'll need to take the keyboard apart in order to get to the PCB. CommonSense should work on most if not all capacitive keyboards, but most frequently people seem to be using it on Beamspring or Model F so I will use those as an example.

Once you've got your keyboard apart, you need to trace the rows and columns of the matrix to some point on the PCB where they terminate. So for example on an IBM 5121, the place where these lines connect is the big edge connector. Usually it's easy enough to trace the lines just by visually inspecting the PCB. You don't need to map out every key in the matrix, you just need to figure out which lines are rows and which lines are columns.

You will also find some connections to Gnd. These must be connected to Gnd on the controller. Otherwise, you will get wacky results.

Soldering

If your rows/cols totals fit within the 8x17 (or 8x24 if you did the necessary modifications), that's great; it's time to hook everything up. Connect the row lines from the keyboard to the row pins on the dev board. Same for columns. Don't forget Gnd (must be connected to PCB gnd as well as chassis). Solder everything up and you should be good to go. Now you can put the soldering iron away, crack you knuckles, and park yourself in front of the computer for the firmware side of things.

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Part 3: Configuring, building, and flashing firmware

PSoC Creator

CommonSense is distributed as source files which need to be compiled. For compiling and flashing the firmware, the tool used is PSoC Creator from the dev board manufacturer. Download and install this piece of software. The current version and typical installation should work fine.

https://www.cypress.com/products/psoc-c ... onment-ide

Once it's installed, Launch PSoC creator and go to File > Open Project

Navigate to the CommonSense folder we extracted earlier. Open the PSoC project file "CommonSense.cywrk" (located in the top level folder) named.

Once the project is opened, make sure to set the dropdown in the upper right hand corner to "Release". (would say "debug" by default usually)

Building the bootloader

In the project explorer on the left side of the screen, find the Bootloader project.

Right click the project name and select "Set as active project".
Right click again and choose "Device Selector". In the popup pick "CY8C5888LTI-LP097"
Press Shift + F6 to build the bootloader

Building the firmware (dry run)

This will be similar to what you did for the bootloader. This is just a dry run, but don't skip it! Make sure to do this before making any changes!
In the project explorer, find the Firmware project.

Right click the project name and select "Set as active project".
Right click again and choose "Device Selector". In the popup pick "CY8C5888LTI-LP097"
Press Shift + F6 to build the firmware

If it builds, that's great! If not, something went wrong somewhere. In that case, try redownloading CommonSense from the github repo and repeating the build steps. What we want to make sure is that the build environment is capabile of compiling the vanilla firmware. That way if the configured project fails to build, we know the issue is probably something in our configurations, rather than with the build environment. Now we're going to hit the brakes while we do a few configurations.

Config.h

In the project explorer, find the file called config.h (located in Firmware project under "Header Files"). There are just a few things to set in here.

Set the number of rows/cols based on your plan
Set the switch type -- BUCKING_SPRING or BEAMSPRING
If your keyboard is something different (e.g. Digitran leaf springs) select the choice that makes sense (i.e. is the default state of the switch low (bucking spring) or high (beamspring)

The other configurations in here can be left as default as long as you're powering the board the standard way (i.e. USB micro cable). Make sure to save your changes.

Pins Assignments

Note: This part is only relevant if you deviated from the default pins. You can skip this section if you went with the default assignments. Though, I would recommend stepping through it just to confirm everything looks good.

We need to tell the firmware which pins to use. Still with your active project set to Firmware, locate and double click the node in the tree called "Pins" (it will be under "Design Wide Resources"). A view will open showing all of the pins on the chip. We'll be making our changed in the pane on the right ride.

Go through each of the "cols" and choose the corresponding pins from the drop down lists
Do the same for each of the "rows"

There are other changes we can make here but I'm not going to be going into those.

Building and Flashing the firmware

With the configs out of the way, it is time to build and flash the firmware for real.

Plug the USB-A side of the board into your PC.
Press Ctrl + F5

A popup should appear. This is where you select the board to flash. Click the top-level item that appears in the picker window. Then click the "Port Acquire" button. A sub item should appear. This is your target, select it and click the "OK / Connect" button.

The first time you do this, it will most likely say you need to update the kitprog. To do that, open the application called "PSoC Programmer". Go to the "Utilities" tab and click the "Update Firmware" button. When it finished, pop back over to PSoC Creator, and proceed with the build/flash as previously described.

Hopefully it builds and loads onto the board with no issue. If you have a build issue at this point, you may want to start with a fresh pull from the github repo to get those sweet sweet defaults back. Then try changing one setting at a time, rebuilding, etc. until you zero in on the issue.

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Part 4 - FlightController

Preface

FlightController is the piece of software DMA wrote to configure the controller. It is built using Qt. Qt is a framework for building cross-platform applications. This means that FlightController can be built on / run on Windows, OSX, and Linux.

So there are two paths that you can take for the next part:

You can download the compiled binaries from the Releases section of the CommonSense github repo
You can compile the app your self

For now, I'm just going to link to the compiled app.

https://github.com/dmaone/CommonSense/releases

Download the exe and you're good to go.

Getting Started

Plug the controller into your pc this time with a usb cable via the micro usb connector on the dev board. Make sure not to ever plug both ends of the controller in at once. I don't know the risk is but you've made it this far, don't fry your board!

Then launch FlightController. The application will immediately begin looking for your device and automatically connecting to it. If it fails to find / connect to your controller, hit the reconnect button. If it still doesn't connect, you may need to go back to section #3.

After connecting, all of the buttons on the left side will become available. Click the status button, the text area on the right should output something about CommonSense. Hey! That's a good sign!

So we know with 100% confidence that the firmware is flashed to the board. But before it will work, you need to initialize it with a config. DMA has kindly packaged some configs for you to start with. Go to Config > Load from file. Then browse to the CommonSense folder you extracted earlier. Next into the folder named "misc". Inside there's a config file for two keyboards. Pick the one that matches your keyboard. Don't worry if your keyboard is something way different; this is just a starting point.

Pick one of the files and load it. Now FlightController has a bunch of settings set. Next you will upload it to the controller by going to Config > Upload. This uploads the config, but does not "commit" it to the eprom (which is a separate step). The config you just uploaded will only last as long as your FlightController "session". But that's perfect because for now we're still configuring.

Thresholds

This is the part of the application where you define what the controller considers a keypress. And you do it PER KEY. Okay, it's actually not that bad. Hopefully your keyboard is not wildly different from key to key. Keep the thresholds window open while we do the next part.

Key monitor
Open key monitor and click the "Start!" button. So when you press a key on the keyboard, the number corresponding to that position in the matrix should change. Your job is to find a value where keypresses are actually detected and false keypresses are not triggered when key is just idling. As you figure out the values, set them in "Thresholds".

I'm not going to go very indepth about this part because I've not done this with enough keyboards to know what advice I should give. What I'll say is (a) you can probably figure it out after a little bit of playing and (b) read DMAs instructions in the repo

Layout, Layers, Macros

These sections behave just like you'd think and are pretty easy to navigate.

Delays & Hardware
I don't yet have a good grasp on these sections. As I learn more I will fill in this post. What I'll say for now is that if you have odd behavior coming from your keyboard, I'd suspect something in one of these two sections needs to be tweaked.

Oh and you can select what the expansion header on the board does (the little part that can snap off). You can choose from (1) nothing (2) solenoid (3) led lock lights

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Part 5 - Log

Spoiler:

snacksthecat · 24 May 2019, 03:10

Reserved

PlacaFromHell · 24 May 2019, 03:58

Cool. I didn't understand anything about this until now.

kmnov2017 · 24 May 2019, 10:45

So this basically means I could use this as a replacement for Xwhatsit. Correct?

snacksthecat · 24 May 2019, 13:26

kmnov2017 wrote: ↑
24 May 2019, 10:45
So this basically means I could use this as a replacement for Xwhatsit. Correct?

Yep, exactly right.

And echos my point about why I wanted to put something like this together. Many people are not aware of this project or if they are aware, they don't know where to get started.

I'm hoping this guide can help ease the barrier to entry a little bit and get this project in the hands of non-technical / less-technical people. Once the details are put in context, the process of building one of these controllers is actually not very difficult at all.

swampangel · 24 May 2019, 14:13

snacksthecat wrote: ↑
24 May 2019, 13:26
I'm hoping this guide can help ease the barrier to entry a little bit and get this project in the hands of non-technical / less-technical people. Once the details are put in context, the process of building one of these controllers is actually not very difficult at all.

This guide is absolutely fantastic. Thank you for taking the time to assemble and share it

JP! · 24 May 2019, 14:48

Wow thanks for breaking all this down. It's nice to see it all in one place. Where do you find the time to dive into all these rabbit holes?

What about lock lights for a Model F or support for a solenoid driver for beamsprings?

SneakyRobb · 25 May 2019, 00:19

This is a good guide.The xwhatsit was very good and excellent for its time and we should be appreciative of it. The DMA controller builds on that and is in many ways very much so improved. The DMA controller really does make the process of setting up a capacitive board much much easier. As well it makes it much more stable. People should know that it is the best current controller for these purposes and have an easy guide to follow. Good job snacksthecat

kmnov2017 · 17 Jun 2019, 22:49

How does one connect the solenoid driver?

PancakeMSTR · 16 Jul 2019, 19:19

This is a great start, and of course I think you should continue working on and refining this guide. I do second the question about the solenoid driver, though, can you add a how-to on that to the guide?

DMA · 16 Jul 2019, 21:10

Thanks for the pictures.

Errata:

CY8C5888LTI-LP079 => CY8C5888LTI-LP097

"Switch type if neither" - the real difference is "what's default level - low or high?". So, for beamspring the plate is laying on the PCB by default and on model F it's lifted. And there's no other options

Ground must be connected to both PCB ground and the chassis (metal cage PCB is in).

>17 columns is a bit trickier to explain. There are pins that you won't use - usually last 4 rows (because beamspring can be 23x4 and others don't have >16 columns.). So if you want to avoid desoldering - you usually can. Just swap 3 pins (which have external components) with those unused pins.

4978 with it's 8x22 matrix is a bit harder as you only have 2 unused pins. You can get third pin by remapping one of the rows to use P12 (can't use it for columns) and using the now-free pin for the remaining column.

kmnov2017 · 23 Jul 2019, 15:48

When opening Project and after selecting CommonSense.cywrk

I get the following error:

"For security reasons DTD is prohibited in this XML document. To enable DTD processing set the DtdProcessing property on XmlReaderSettings to Parse and pass the settings into XmlReader.Create method.). Address the error and try again."

snacksthecat · 23 Jul 2019, 16:08

CommonSense 1.0 was just released a few days ago. Maybe the project file was generated on an older version of PSoC Creator and the latest version doesn't like it (?) Just a guess.

I would suggest downloading the previous source code package off the releases page in the repo and seeing if it works:
https://github.com/dmaone/CommonSense/releases

Or try an older version of PSoC Creator (though I'm not sure if they make those available).

kmnov2017 · 23 Jul 2019, 17:28

What version of PSOC created did you use ? Cypress does have the past versions for download.

DMA · 23 Jul 2019, 19:06

Generated by 4.2 which is supposed to be latest.

kmnov2017 · 23 Jul 2019, 20:26

Finally managed to open the file. It seems some files didn't extract to the folder.

snacksthecat · 24 Jul 2019, 00:01

That’s why winzip’s logo is a vice. You’ve got to squeeze the archive to get all the files out.

kmnov2017 · 24 Jul 2019, 00:11

So the new version just doesn’t compile. I can get version 0.2 to build the firmware though, but not v1.0

DMA · 24 Jul 2019, 00:36

kmnov2017 wrote: ↑
24 Jul 2019, 00:11
So the new version just doesn’t compile. I can get version 0.2 to build the firmware though, but not v1.0

Don't cosplay an accountant, please. Obviously it compiles for me, otherwise I wouldn't push it. So "it doesn't compile" is not something helpful.

Where are the error messages? What part you were trying to compile?

snacksthecat · 24 Jul 2019, 00:47

Hi DMA,
First of all thank you for the feedback, I need to fold those things into the guide.

I tried building the bootloader project for the 1.0 package and got the following errors:

Seems like i'm missing cy_boot_v5_80

Steps to reproduce

Download and extract v1.0 release
Open in PSoC Creator (I'm using 4.2)
Set active project on Bootloader
Device selector: CY8C5888LTI-LP097
Build (shift + F6)

Same issue / similar errors building Firmware project

DMA · 24 Jul 2019, 01:08

Damn it. My habit of updating the software to reasonably-latest versions will get me killed one day.

Right-click on top-left item in the left tree editor ("Workpace 'CommonSense' (7 Projects)"), click "Update Components". It should download the missing parts.

If it doesn't - I'll have to do a fresh install of all the dev environment to get a repro

snacksthecat · 24 Jul 2019, 01:23

DMA wrote: ↑
24 Jul 2019, 01:08
Damn it. My habit of updating the software to reasonably-latest versions will get me killed one day.

Right-click on top-left item in the left tree editor ("Workpace 'CommonSense' (7 Projects)"), click "Update Components". It should download the missing parts.

If it doesn't - I'll have to do a fresh install of all the dev environment to get a repro

Followed your instructions and was able to build the bootloader.

I continued on to the next step:

Set active project to Firmware
Build (shift + F6)

and unfortunately received these errors:

This one is probably equally as obvious. Unfortunately I have no clue. Hope this makes sense to you.

I tried to locate the file it's looking for but there is no "Compiled" directory inside of Bootloader.cydsn

Thanks again!

DMA · 24 Jul 2019, 01:29

Make extra sure dropdown near "help" menu says "Release". Sometimes it just resets to "Debug" and everything explodes.

snacksthecat · 24 Jul 2019, 01:49

My bad! That was exactly the problem. Builds with no errors.

kmnov2017 · 24 Jul 2019, 02:07

That’s great...

I need some help changing the Pin configuration for a beamspring.

I quite like DMAs suggestion to remap the unused rows for columns and the expansion headers for the remaining columns.

kmnov2017 · 24 Jul 2019, 13:54

I did compile and flashed the cypress board.

When trying to configure Flight Controller, this is the error I get

: FC.jpg (77.78 KiB) Viewed 17042 times

DMA · 24 Jul 2019, 17:52

Please read the "Configuring your keyboard" section.

DMA · 24 Jul 2019, 20:43

OK, I changed the default pinout (and added a picture of it), so that you only need to desolder things if you want to use column 24.
I made a different project for it though - so this guide is still valid.

Added "default firmware" - which is, by definition, half-assed. I don't know what's the matrix size of your keyboard, and making it configurable via FlightController makes scanning considerably slower (I tried 2 years ago).

Explained the ExpHdr thing (it's not on the breakaway part) - and also why I don't want people who don't know what they're doing to touch it.

Also made contract a bit clearer in README.md, because 2 cases of "I want to use this thing you made, I don't want to read instructions, YOU OWE ME TECH SUPPORT" warrants clarifications.

kmnov2017 · 25 Jul 2019, 16:15

So here's a quick update:

After wiring by the Cypress board, flashing the firmware and wiring up the keyboard and after successfully connecting to the keyboard via Flight controller; I did the following:

1. Set the thresholds - This is straightforward. I pressed every key, all the keys registered and a threshold value was obtained. Note: Not every key press registers a yellow highlight. I have no clue why, but the numbers do show-up. Initially getting the scan to work took a while. Even after loading a template file, I would always get the scan = false status. Eventually, just with playing around and repeatedly pressing the "Scan" button (yes, it's clickable!) I got it to start scanning.

2. Once I got the threshold values, I went to threshold editor and per DMAs guide on Github, added an incremental value of 5. Most of my thresholds were under 7, so adding 5 roughly doubled the values.

3. I then opened up the Layout editor. When you a press a button, YOU NEED TO HOLD IT DOWN FOR A FEW SECONDS before the pressed key shows up. Assign your value based on the pressed key. If you have another keyboard connected (or if you are doing this on a laptop), just press the desired key on the other keyboard (or laptop) - This way you can set up the layout fairly quickly.

4. I did a commit. You'll get a message if it successfully flashed on to the EEPROM.

Now this is the part I am stuck. After the commit and closing Flight Controller, I was assuming the keyboard is configured and ready to use. But no, I can't get the keyboard to work. Pressing keys doesn't register anything. I read the Github guide all over again, I am not sure what I missed....

DMA · 25 Jul 2019, 20:29

kmnov2017 wrote: ↑
25 Jul 2019, 16:15
So here's a quick update:

After wiring by the Cypress board, flashing the firmware and wiring up the keyboard and after successfully connecting to the keyboard via Flight controller; I did the following:

Dafuq, man. I post here and there because it's a change to software so is a material change affecting everyone. What is your motivation for doing so?