This post is motivated by another thread
from user magicpiano, and his attempts to customize the sound of his Kawai CN37 beyond the normal range of options.Speculation about how pianist mode works
About 2 years ago I read about a technical limitation of the pianist mode in the NV10, which appears to have a polyphony of only 64 (vs 256 in sound mode). Maybe I'm mistaken, but I mentally archived this as a hint of how pianist mode is implemented, and as a pathway to potentially achieve a similar effect on older models that do not offer pianist mode.
My perception was that pianist mode either requires more CPU/DSP power than soundmode (and cannot process more than 64 channels). Or, and this is more interesting, it employs 4 independent audio streams per note played. I'm refering to 4 of 256 hardware DMA channels that fetch sample data from memory and make it available to the mixer algorithm.
In the following I'm speculating only about the latter possibility, because the former one is a dead end for doing DIY enhancements.Why would pianist mode use 4 channels at once?
This only makes sense if the 4 streams are chosen, or modulated, dynamically at runtime. Otherwise the mixing could be done at the factory and be stored as 1 audio stream.What type of dynamic behavior could useful?
It must be something that uses only little CPU and off-loads the main work to the 3 extra hardware channels. I see two possibilities.
What sample material would be played on the extra channels?
- One stream is the main note and the other 3 streams are exaggerated resonance that is associated with the note. The algorithm could dynamically insert more resonance when an isolated melody is played, and less of it when a lot of notes are active anyway (avoiding muddiness).
- This is a more refined version of the above: Complement the main note with 3 additional samples, choosing both pitch and volume dynamically based on all currently active notes. I imagine a mathematical model that calculates the interaction of them and how this energy can excite other strings (maybe even damped ones to some degree). The algorithm would sort the results by magnitude and play the top 3 on the extra channels.
Certainly not just another piano voice. The hardware is capable of fetching multiple piano voices at once (e.g. with the layering feature in sound mode). Therefore it is feasable to store and reproduce specially crafted complementary samples along with the main voice, samples that do not resemble a stand-alone piano voice on their own. Maybe it's a faint sound without attack, something like when you do a silent key press on an acoustic and then shake the body of the piano. Or maybe the output of a digital filter, the delta between a richer main note and a less rich main note, something that sounds unnatural when played alone.
Either way, it must be something that *sums* to the already existing state. Something that can be added over and over (using different dynamic parameters) with each additional attack. Because, if it was just one single aspect per each of the 88 string(-triples), then the huge reduction in polyphony didn't make sense. A reduction to 256-88 = 168 would suffice.Suggestion for a DIY pianist mode on CN37
Based on this wild speculation, here's my suggestion for a DIY cheap-skate pianist mode:
- Connect the CN37 to a computer over MIDI and disable local-control.
- Receive the MIDI events in software and loop them back into the CN37 to get sound.
- Rather than faithfully forwarding note by note, insert additional (soft) notes to complement the ones that are played by the pianist.
- Make sure that all simultanous MIDI events encode into a single USB datagram to avoid timing issues introduced by USB congestion.
At the very least, this can achieve different sounding timbres (as user magicpiano seeks in his thread). But if one comes up with an algorithm that choses the extra notes based on dynamic interaction of the already active notes, then the result could even be somewhat similar to pianist mode.
The main limitations are that the extra notes come from the same piano voice and that they contain an attack phase.
To attenuate the attack phase, I can think of a few tricks:
- Use only low velocities, considerably lower than the "main" note that is attacking at the same time. That will hide the attack psycho-acoustically.
- Study the brightness and velocity layers of the piano voice and establish an upper limit (per pitch).
- Study the MIDI behavior. Maybe the attack phase is attenuated (or skipped?) if one sends a (medium fast) note-on, immediately followed by a (slow) note-off. Notes that the pianist is already holding down need to be excluded, of course. This method probably cannot be applied when the pedal is down (except possibly encoding as pedal-up + note-on + note-off + pedal-down).
- Play all/some extra notes as silent notes (velocity 0 or 1), triggering the sympathetic resonance feature rather than a note with attack. The magnitude and duration can be controlled by following up with a note-off shortly after.
I suppose you can see the potential. If you choose to explore this rabbit hole, please share your results on github.