CommonSense controller for digital piano project?

[DiodeHead]

I like a lot the aluminum extrusion idea with the strange tape to make it a bearing surface, that way you don't have to use a separate part made of brass or something similar.

it's a pity that the web and the forum are down I would like to see more about the internal mechanics if those mechanics are good and adjustable to the player then you could reuse them in case you would like to go the production route. I've heard that in the extrusion manufacturing the expensive part is the tooling, so if you contact the same manufacturer you could probably get that same extrusion very cheap since the tooling is already made.

I think it was in hackaday where I read that, that tooling could cost you in the thousands but the actual extrusion would go for ten dollars each meter or so, not completely sure I'll have to check.

My point being, if it was a really open source project with issues it is not a bad idea at all to fork the project resolving those issues and with better electronics ( CS comes to mind )

Also, I offer myself as kicad free labor since I want to step up my skill. Right now I'm doing a nvrom nes cart for a friend of mine, if everything goes ok I'll unlock the first level of PCB manufacturing

I hope this keeps advancing

[deskman]

wooting texts are a bit bulshitty. like "we use analog input so we can't have matrix". Also they don't need debouncing because IR switches don't bounce. What they need is EMI rejection. Which they can do with jacking up scanning speed to, say, 5kHz and have sub-1ms latencies (which are not that useful because USB limits latency to 1ms, but still).
4ms is kinda long. I was able to make 2ms latency with 4-deep debouncing buffer.

I'm surprised those wooting guys are not on DT. With ideas from CS they could make their keyboard much cheaper and better.

I'm on the fence about the Wooting blog. They are pretty transparent about the architecture and boundaries of skills. The Wooting v1 was popular enough to permit a v2, which has some good features. On the other-hand, as you note, Wooting missed opportunities to boost performance significantly. Not sure why they didn't drop by here. . .

I suspect ironing out the pretty software user interface sucked up a large percentage of their development resources, so that is a factor.

I like a lot the aluminum extrusion idea with the strange tape to make it a bearing surface, that way you don't have to use a separate part made of brass or something similar.

it's a pity that the web and the forum are down I would like to see more about the internal mechanics if those mechanics are good and adjustable to the player then you could reuse them in case you would like to go the production route. I've heard that in the extrusion manufacturing the expensive part is the tooling, so if you contact the same manufacturer you could probably get that same extrusion very cheap since the tooling is already made.

I think it was in hackaday where I read that, that tooling could cost you in the thousands but the actual extrusion would go for ten dollars each meter or so, not completely sure I'll have to check.

My point being, if it was a really open source project with issues it is not a bad idea at all to fork the project resolving those issues and with better electronics ( CS comes to mind )

Also, I offer myself as kicad free labor since I want to step up my skill. Right now I'm doing a nvrom nes cart for a friend of mine, if everything goes ok I'll unlock the first level of PCB manufacturing

I hope this keeps advancing

I know the VAX was marketed as open-source but don't see much released to the public. Certainly haven't found any of the code although one buyer threatened to release it.

You can access a lot of the old VAX website via archive.org. The assembly guide I linked above gives you a nice view of inside components. Archive.org takes a bit of massaging as snapshots from some days work but others don't. You can use the links in my post above to help. For sure, I could not get any detail in the VAX forum (beyond just some thread titles).

A guy offered me his partially assembled VAX today. Unfortunately, CyberGene noted, and the datasheet confirmed, the tcpt1350 is not really ideal for grand piano system. Might still be worth examining for fun but probably not.

EDIT - One advance VAX user told me he thought the firmware and sensors were fine; he thought issue was the physical keys/chassis, etc. prevented the keys from working or calibrating correctly


My idea was to use an old wooden grand piano action. So plastic & alu extrusion is out of the scope for now. That said, for future projects, I have a colleague in Europe who has been designing and 3D printing plastics for toy companies for a long time. His firm runs top-tier industrial equipment and he does that work in his sleep. If we ever need tooling, that route is expensive but he can help with design & his industry friends.

Thanks for the kicad offer! Let me get a bit further then contact you. I watched some kicad videos from Australian Dave at EEVBLOG.

[Jampu]

Please forgive my ignorance here - from what I understand, you need to know three things:

1. When the damper is released (key pressed partially)
2. When the hammer is moving
3. When the hammer strikes the stop

It sounds like this could be accomplished with three optical digital switches positioned such that the damper and hammer (or other parts that may be closer/easier to put instrumentation on) will make or break the beam at these three points. Measuring the time elapsed between 2 and 3 gives you the velocity, which could be measured with a free-running timer. 264 switches can be scanned with as little as 33 pins, though it may make more logical sense to arrange it in a 22x12 matrix (34 pins).

That said, as originally stated I am probably just ignorant

[Muirium]

The way most midi keyboards work, as opposed to actual pianos, is simply with two sensors per key. You have to activate both of them to trigger a note, and its “velocity” (amplitude) is tied to the timing between the two events. The harder you hit the key, the higher speed and so the closer in time between the first and second triggers. This isn’t perfect but it does suffice for the great majority of midi keyboards and electric pianos.

I’ve got a Roland that has a nice weighted keybed and triple triggers. It’s a little subtler, letting you do tricks a regular two sensor won’t. The essential principle is still the same, just with extra conditions.

[HuBandiT]

Here however the concept is to sense the position of the hammers continuously. And that allows for some quite nice things (like negative latency).

An extra idea would be to also sense the position of the dampers, as that would be another currently missing piece of information that is available on a real piano.

[Muirium]

Right. Analog sensing lets you do much more subtle things like determine velocity and acceleration at arbitrary points instead of fixed triggers along the key stroke. But midi itself is only designed to take velocity and timing, which as far as I can tell does limit things a bit.

Where capsense could really shine is in a mechanical, weighted piano with hammers (like my Roland or indeed a real piano) and feeding its output to a software sound engine unlimited by midi’s 1980s constraints.

[DiodeHead]

Really good point Mu, midi is very constrained and at the end of the day you only send noteON(velocity) and that velocity only have 128 points of resolution if I remember correctly, so the more sensor you use the only thing you get in return is more accuracy between lets say velocity = 67 and velocity = 68 in relation to the mechanical key movement, which brings me to what HuBandiT said " (like negative latency) "

Does that mean that some sensors are used to anticipate keystrokes and reduce latency? that is very interesting.

my point being, I think grand piano midi keyboard companies spend more time in the physical emulation of things ( Kawai internals are impressive as I can see on pictures ) and he already has that, so he should probably start with the less complicated electronic design and let his ear decide if he needs the extra bells and whistles.

the three switch options sound like what a nice expensive keyboard would have so would be a good starting point. As Mu said I would leave the analog sensors unless you want to make your own sound module (which is a cool idea, raspberry pi samplerbox comes to mind )

[DMA]

Mu, what tricks are possible with 3 sensors that are not possible with 2? Whole new world to me, interesting what's there

[HuBandiT]

Where capsense could really shine is in a mechanical, weighted piano with hammers (like my Roland or indeed a real piano)

... which is indeed the case above (real piano mechanism) ...

and feeding its output to a software sound engine unlimited by midi’s 1980s constraints.

... which is where PianoTeq (and probably others) come into play.

Look at the Lachnit keyboards; they, cooperating with PianoTeq (and maybe others), are already pushing the boundaries of MIDI.

The internal musical intent interface of most digital audio workstations and notation software is already way more sophisticated than '80s MIDI, and they continue to develop, because of the musical expressiveness demanded.

MIDI is flexible enough to allow passing through pretty much anything; the only thing is for the endpoints to agree on how to represent those things. Which is how most of the advanced MIDI features were born I assume.

Worst case there is polyphonic aftertouch already, which we could (ab)use for communicating key/hammer/damper position. (In a pinch there is also key-off velocity.) So I do not think MIDI itself is a bottleneck (aside from speed/latency/throughput of serial MIDI - but there is USB and Ethernet for that).

"If you build it, they will come."

[HuBandiT]

Mu, what tricks are possible with 3 sensors that are not possible with 2? Whole new world to me, interesting what's there

My theories:

• with 3 points, you can afford to make the release point (note off) to be at a different point of key travel, than where the (imagined) hammer starts to accelerate (=where you start to measure speed)
• hitting the key transfers mechanical energy from the fingers to the key and key mechanism, partially elastically and partially non-elastically. with 3 points, you can measure one extra point on the time-position curve, so you can get a better idea of what the playing dynamics is, what intent of the player was, what would happen on a real piano, etc.

[DiodeHead]

then from there what I get is that the easier route is to send a very polite a loving email to pianoteq support asking if they have some documentation about what midi commands they accept and start designing from there, if we know what info to send its easier to see what sensors are more adequate, the other route is to download the demo and make a midi sniffer and try to reverse the protocol which not impossible but probably a pain in the ass.

Look at the Lachnit keyboards; they, cooperating with PianoTeq (and maybe others), are already pushing the boundaries of MIDI.

from this video, I get that lachnit is using the dual channel optical sensor like the one that deskman said in his post about vaxmidi keyboard. I don't know a word of (German ? Dutch ?) but by the images looks like that.

[HuBandiT]

Really good point Mu, midi is very constrained and at the end of the day you only send noteON(velocity) and that velocity only have 128 points of resolution if I remember correctly

Don't limit our possibilities by this kind of thinking. These representational limits were true decades ago, but I have a hunch they are no longer (e.g. Lachnit mentioned above already uses high resolution velocity - I assume 14 bits), as there is continuously demand for more musical expressiveness. Instead let's focus on capturing musical expressiveness. Then communicating what was captured should be only secondary. MIDI is quite flexible and extensible.

negative latency:

Does that mean that some sensors are used to anticipate keystrokes and reduce latency? that is very interesting.

Yes, exactly. The concept of the original poster is to use optical sensing to sense the position of the mechanism (hammers currently). Since the travel of the hammer can be relatively well predicted - once it left escapement -, and we continuously watch the hammers, we will follow them during their approach, and even during their approach we will have a pretty good idea of where they are on their way and with what speed they are approaching; so we will know, when the hammer will hit and with what force way before they hit.

Consequently, we can report arrival before the hammer actually arrived, achieving negative latency of our choosing (up to, say, half of the hammer travel time), thereby giving other parts of the system (MIDI communication, operating system latency, virtual instrument latency, audio output latency, etc.) some relief.

Someone should measure the travel time of hammers (great opportunity to utilize the high speed camera function of your phone/camcorder/camera), but my assumption is that it is on the order of 10-200 milliseconds. Which is not little.

my point being, I think grand piano midi keyboard companies spend more time in the physical emulation of things ( Kawai internals are impressive as I can see on pictures ) and he already has that, so he should probably start with the less complicated electronic design and let his ear decide if he needs the extra bells and whistles.

True. However, that would leave him with a boring average mediocre-performing (= discrete 2 or 3 point sensed) instrument that he could already buy ready-made in stores, and it would totally waste/ignore the potential offered by the real piano mechanism and the DIY/experimental setting.

the three switch options sound like what a nice expensive keyboard would have so would be a good starting point.

... except where and how do you mount three (or even just two) switches/sensors per key on 88 keys of a preexisting wooden real piano action/mechanism? And in a way that provides good playability by being mechanically consistent?

As Mu said I would leave the analog sensors unless you want to make your own sound module (which is a cool idea, raspberry pi samplerbox comes to mind )

• There are no digital sensors. Every sensor is analog (even resistive computer keyboard keys) - you're just ignoring most of the information during processing. (Or the manufacturer ignores it for you.)
• I would agree with you more if the original goal was not to have piano-like playability and musicality, since the OP specifically mentioned PianoTeq, who are open to making their instrument more expressive.
• Just because we can.

[HuBandiT]

then from there what I get is that the easier route is to send a very polite a loving email to pianoteq support asking if they have some documentation about what midi commands they accept and start designing from there, if we know what info to send its easier to see what sensors are more adequate, the other route is to download the demo and make a midi sniffer and try to reverse the protocol which not impossible but probably a pain in the ass.

I disagree. I think the details of representation/communication is completely secondary in this setting. With that said I am sure PianoTeq would be open to releasing this information. They model almost the entire piano mechanism, including dampers. Their MIDI implementation already does partial pedaling as the model allows that to arbitrary resolution - floating point), as well as arbitrary resolution (floating point) velocity/dynamics - used by Lachnit, as well as note off velocity. Since damper position during key release is not currently available in MIDI, I assume they have to fake it for most cases - but their model works with a continuous damper position as well, so even that would be possible, we just have to see whether that is already offered through MIDI (e.g. polyphonic aftertouch maybe), or work with them to come up with a way to encode that into MIDI.

Representation is secondary and is easy to change, so it should not be the driving factor during design. The inherent possibilities of expression capture facilitated by the instrument is what should be the driving force. I am sure PianoTeq would be interested in working towards exposing more and more of the expressive capabilities of their model, as it would give more enjoyment for their players.

from this video, I get that lachnit is using the dual channel optical sensor like the one that deskman said in his post about vaxmidi keyboard.

I see a single optical interruption sensor with a pattern of flags and slits of the mechanism. I count two initial (near key up) slits and - although it cannot be seen clearly on my end - one or two end slits (near key down); plus the sensing starts from beyond the edge of the metal, so that can be counted as an extra transition. So schematically it looks like this:

o XX__XX__XXXXXXXXXXXX____XX

o is the beam, X is metal, _ is gap

this arrangement gives us 5 transition points near the key up position, and one at the key down position (or if there are two slits near key down, then 3 there). A total of 6-8 position points - if they only process the optical signal as digital signal. But since the flags can cover the sensor completely, one could sense more precise position (and thereby speed) by sensing not just light on-off but partial light levels. You can do this at every light/dark transition. So that would then give you 6-8 combined speed and position measurements along the path of the key - which is quite a rich information. (And that is with just one sensor per key.)

[DiodeHead]

since it's a dual optical sensor wouldn't be more like:

r r XX__XX__XXXXXXXXXXXX____XX <-- r = reciever

that with a timer could give you the time difference between 2 transitions and with that velocity of sorts ??

tran2 - tran1 / time = velocity ??

[HuBandiT]

since it's a dual optical sensor wouldn't be more like:

r r XX__XX__XXXXXXXXXXXX____XX <-- r = reciever

that with a timer could give you the time difference between 2 transitions and with that velocity of sorts ??

tran2 - tran1 / time = velocity ??

Oh yes, that is true. In essence you would double the number of transitions (and thereby, data points). (I was looking at the single-channel part you referenced in the message about the VAXmidi). In that case, Lachnit probably does not measure partial light, but instead does on-off only. Easier to manufacture/calibrate, and lower performance MCU is enough.

[deskman]

Advantage of triple sensor is elegantly shown in the orange & grey diagram

http://www.kawaimp.com/mp7/detail/touch/

[HuBandiT]

Fun with numbers:

The sensor apparently used in the Lachnit has 0.8 mm between the two light beams (not much, right?). Modern microcontrollers routinely have a 200 MHz clock. Let's put these two into perspective.

Let's conservatively assume that the metal hammer in the Lachnit keyboard does not fly faster than 1 Mach (the speed of sound - around 340 m/s). We can realistically assume this, since effectively dampening a desktop keyboard against sonic booms would add a lot of weight and size, not to mention thermal issues. 340 m/s is 340 000 millimeters per second; when detected by a pair of slits 0.8 mm apart, it would give two signals 0.8 / 340 000 = 1 / 425 000 seconds apart. A 200 MHz counter during this time would count to 200 000 000 / 425 000 = 470.588... 1/470 is approximately 0.2 / 100, so this arrangement can roughly detect 1 Mach speed with a 0.2% (or 2000 ppm - parts per million) accuracy.

The fastest human sprinters peak at around 45 km/h, about 12.5 meters/second. Let's - very unscientifically - assume this speed is representative of the fastest human limb speed that can be expected to hit the musical keyboard key. So we have 12 500 mm/s; a 0.8 mm slit pair arrangement would result in a time difference of 12 500 / 0.8 = 1 / 15 625 seconds. A 200 MHz counter during this time would count to 200 000 000 / 15 625 = 12 800. So this arrangement can detect "fastest human speed" (don't take that seriously) with about 0.0078125% or 78.125 ppm accuracy.

Cherry MX literature quotes 400 mm/s as assumed finger speed when testing their computer keyboards. Doing the math again: 400 mm/s detected by a 0.8 mm slit pair is 0.8 / 400 = 1 / 500 seconds. A 200 MHz counter would count to 200 000 000 / 500 = 400 000 (four hundred thousand) during this time; so this arrangement could detect this speed with a 2 ppm accuracy.

The maximum speed during piano/music playing probably lies between the second and third examples. But even if actual finger/hammer speed would be near 1 Mach (first example) there would be still 470 counts in the counter, which is already more than enough to provide a very fine gradation of playing dynamics.

So we can reasonably assume, that this measurement arrangement is (orders of magnitudes) more than sufficient to provide a very good capture of playing dynamics. Even at Mach 1.

[CyberGene]

Here's a little update:

Haven't been very active on the project taking in mind I am a father of a 1 year old daughter, so basically the only spare time I find is some sleepless nights...

[deskman]

Awesome progress CyberGene.

Now that you have got 10 keys calibrated and playable, last major hurdle is getting the multiplex performing fast enough with multiple simultaneous signals.

[deskman]

@CyberGene has the grand piano fully working now. He said it is fantastic and there is upside with some calibration and regulation.

Video and updated comments are here

http://forum.pianoworld.com/ubbthreads. ... ost2951761

www.youtube.com/watch?v=UFMgjKCsDu0