Welcome to the Piano World Piano Forums Over 2.7 million posts about pianos, digital pianos, and all types of keyboard instruments
Join the World's Largest Community of Piano Lovers
(it's free)
It's Fun to Play the Piano ... Please Pass It On!

Apologies if this question has been asked before and I have not picked it up in the list of threads.

Digital piano manufacturers increasingly talk about sound generation by the use of modelling. Can anyone explain what this is ? I am familiar with mathematical modelling and I would be very surprised if they mean modelling from first principles, in other words starting with a wave equation, but then modifying the solutions to account for anharmonicity and variation of amplitudes of different overtones. In any case I would view the perturbation of the wave forms so that it sounds like a piano as a numerical approach and would have thought it required sampling in the first place to see what the wave form actually looks like.

A complete modelling from the differential equation need multiple hours of computation for each second.

Then we have something simpler. Pianoteq patent describes the weighted sum of sinus function. Then we just need to store for some velocities the weights and the frequencies. This is a quite compact « modelisation », but it doesn’t describe how the piano works.

Yamaha CLP150, Bechstein Digital Grand, Garritan CFX, Ivory II pianos, Galaxy pianos, EWQL Pianos, Native-Instrument The Definitive Piano Collection, Soniccouture Hammersmith, Truekeys, Pianoteq

This is just the first two videos of the series (see link above for the rest):

across the stone, deathless piano performances

"Discipline is more reliable than motivation." -by a contributor on Reddit r/piano "Success is 10% inspiration, and 90% perspiration." -by some other wise person "Pianoteq manages to keep it all together yet simultaneously also go in all directions; like a quantum particle entangled with an unknown and spooky parallel universe simply waiting to be discovered." -by Pete14

Thank you for the responses. So if the weighted sum of sinusoidal waves are being stored this does imply use of sampling to provide knowledge of the waveform being aimed at.

keff: There has been discussion here for years about this subject. And virtually none of it is anything close to authoritative. It's mostly just guesswork and inference based upon inventive ideas.

If you want real information you'll have to find it elsewhere. It aint here.

Thank you for the responses. So if the weighted sum of sinusoidal waves are being stored this does imply use of sampling to provide knowledge of the waveform being aimed at.

Yes, that's been assumed. Since the creators won't answer that question, the closest answer is what we can deduce from their patents. Based on that the answer is "yes".

IMHO, often digital piano manufacturers talk about modeling because it's trendy, just like the "hybrid" word. Apart from Roland (in their top models) and a few others (Viscount with their Physis pianos) all the main manufacturers use the same sample-based technology from the '80-'90. It's just a little better because now we have more memory (to store longer and better samples) and more computational speed, so manufacturers can apply some effects on top of those samples that enrich their sound to make it more realistic (or, at least, they try to make that). Then they call those added post-processing effects "modeling". To me, modeling a piano sound is a completely different thing, but of course opinions may differ. In my opinion, modeling is when the engine generates the waveform (from scratch) based only on some initial parameters (like string length, thickness, number of strings for a single note, strike point of the hammer on the string, etc.), and then it changes this waveform in real-time to create the main effects that we have in a real piano, like the attack, the beating (continuous amplitude variations in the overtones), the dispersion (inharmonicity of higher frequencies), the decay of the sound (the high freqs decay faster), etc...

Thank you for the responses. So if the weighted sum of sinusoidal waves are being stored this does imply use of sampling to provide knowledge of the waveform being aimed at.

I suppose yes : tuning the parameters of the sum of decaying sinus needs a sound reference.

One of the modelled pianos is based on Bechstein Digital Grand which is a sampled piano. I suppose Modartt has to sample other pianos (or use an already sampled piano).

Yamaha CLP150, Bechstein Digital Grand, Garritan CFX, Ivory II pianos, Galaxy pianos, EWQL Pianos, Native-Instrument The Definitive Piano Collection, Soniccouture Hammersmith, Truekeys, Pianoteq

Modeling from the first principles today is less effective than building something, which works well, but is inexplicable by a human. Machine Learning experience in recent decades proves this (SVM regression is an example).

Piano combines too many phenomena, which, albeit well understood in pure physics, as a whole cannot be described by simple equations so that we could reproduce the piano behavior and discriminate it from the behavior of other pianos.

However, given samples of a specific piano, we can define a problem, which is similar to dimensionality reduction: Describe these samples using a reduced set of vectors (which are generally related to some amplitude and frequency changes in time).

There exist numerous ways to specify and resolve this problem.

[...]However, given samples of a specific piano, we can define a problem, which is similar to dimensionality reduction: Describe these samples using a reduced set of vectors (which are generally related to some amplitude and frequency changes in time).

What you describe is more like a compression algorithm for piano samples. Modeling is much more complicated, because you should be able to change (possibly in real-time) the generated sound timbre based on parameters that should be somehow related to the real physics of the acoustic instrument. For example, the point in which the hammer strikes the strings, the hardness of the hammer, the length of the strings, the number of strings for each key, the size of the soundboard, etc... You don't have necessarily to implement all the equations from physics. That would be impossible to do in real-time with current hardware. The target is to get a result similar to an acoustic piano sound. Final users care more for the quality of the sound, not much for the "how" the sound is generated, so it's sufficient a good "emulation" more than an accurate "simulation".

The learning model simply randomly generates new models that successively get closer and closer to an exact match of the target waveforms. The trick is combining multiple models together rather than the 'one model fits all parts of the waveforms' common mistake.

A complete modelling from the differential equation need multiple hours of computation for each second.

Then we have something simpler. Pianoteq patent describes the weighted sum of sinus function. Then we just need to store for some velocities the weights and the frequencies. This is a quite compact « modelisation », but it doesn’t describe how the piano works.

This is essentially a Fourier sine series for the waveform. One certainly could produce a power series approximation for the solution to the boundary value problem for the modeled wave equation as well. I think the wave equation, at least for modeling the string vibrating in a plane, is inadequate because at high enough amplitude, the string actually vibrates in three dimensions.

Login name is a tribute to Jan Pieterszoon Sweelinck, arguably the historically first great keyboard virtuoso.

The learning model simply randomly generates new models that successively get closer and closer to an exact match of the target waveforms. The trick is combining multiple models together rather than the 'one model fits all parts of the waveforms' common mistake.

It's very interesting, and I don't know much about evolutionary computing, but I guess that to create an evolutionary algorithm to "search" the best way to model a piano sound would be much more difficult than creating a single piano model.

From what I understand, usually those evolutionary algorithms starts from a big population of things. Then they make some random selections and combinations of those things until the output get very close to the desired target. The best combination would be the best algorithm. The problem here is that the "things" in our case should be "parts" of piano models, so you could combine them. But, often, piano models from academic literature are so complicated that you cannot so easily dissect them and use in other models in such an automatic way... For this type of matters I think the human brain is still far better than a machine.

Do not forget that the goal of said modeling or sampling is creating (preserving in case of sampling) beautiful piano sounds.

All optimization methods need numerical error function (fitness function in case of evolutionary algorithms) for evaluating the "degree of beauty" of the generated objects.

"Degree of beauty" of samples, extracted from a known $200-500K grand, is maximal by definition. It is guaranteed by the history of piano industry, by the art of the tuner, and by the acoustics and public image of the specific grand. So, we only need to evaluate the similarity of generated objects to what is beautiful by definition.

"Degree of beauty" of a model, based on initial principles without sampling an existing beautiful object, is something weird. Indeed, how can we evaluate the musical beauty of a complex construction? By hearing it? Who is hearing it in the development phase? Who can tell it sounds good or not?

That's why, I think, piano modeling based on initial physics-mechanical principles is less perspective than that based on sampling. But maybe I am missing something related to acoustics here.

One purpose for modeling the sound rather than just sampling is to provide the user with a way to customize the tone. Those pianists that are looking for the most accurate recreation of their particular favorite acoustic will probably prefer a sampled implementation. But others are looking for a unique tone that either stirs their creativity or fills a specific sonic space that they have imagined. For them, a modeling implementation is probably preferable. Degree of beauty? It's in the eye of the beholder, of course.

One purpose for modeling the sound rather than just sampling is to provide the user with a way to customize the tone. Those pianists that are looking for the most accurate recreation of their particular favorite acoustic will probably prefer a sampled implementation. But others are looking for a unique tone that either stirs their creativity or fills a specific sonic space that they have imagined. For them, a modeling implementation is probably preferable. Degree of beauty? It's in the eye of the beholder, of course.

I totally agree with this view. Usually the amateur pianist needs to find the perfect tone for his ears never bothering about anything else. The first goal of playing piano is to thrill your mind and your body with happiness and emotion! There are people who seek for their perfect non existent acoustic sound

Mathematically, the distinction between sampling and modeling is much less than the prevailing view. Once you collect samples, the samples can be used to reconstruct the original waveform. It will generate the waveform precisely for all frequencies up to the Nyquist limit (half of the sample rate). The Shannon-Nyquist theorem is actually mathematically related to Fourier analysis. Samples are a time domain representation of the waveform and a Fourier expansion is a frequency domain representation of the waveform. Both are mathematical models for the waveform. Both models can be created by using sampling as the basis.

The difference in the techniques can be viewed through the lens of a classic space-time tradeoff in computing. Sampled-based models need less computing power to recreate the waveforms, but require more space to store the model. So-called physical modeling requires less space to store the model, but requires more computational power to recreate the waveforms in real-time.

But both are mathematical models of the waveforms.

Login name is a tribute to Jan Pieterszoon Sweelinck, arguably the historically first great keyboard virtuoso.

. . . The difference in the techniques can be viewed through the lens of a classic space-time tradeoff in computing. Sampled-based models need less computing power to recreate the waveforms, but require more space to store the model. So-called physical modeling requires less space to store the model, but requires more computational power to recreate the waveforms in real-time.

But both are mathematical models of the waveforms.

Two comments:

a) It's interesting that the lightest "footprint" (in both CPU time, and storage) of the common piano VST's is Pianoteq -- which does real-time build-up of its output waveforms, and has a small dataset that forms the basis for its "piano models".

. . . That small dataset may be why some people find it unsatisfying.

b) A sampled-based VST doesn't have a "mathematical model" of the waveforms -- it has _real recordings_ of the waveforms! [I know, you could argue that that's a "degenerate case" of a "model". But I think it's genuinely different in philosophy and computational approach.]

. Charles --------------------------- PX-350 / microKorg XL+ / Pianoteq / Lounge Lizard / EV ZXA1 speaker