Piano World Home Page Forums Our Most Popular Forums Digital Pianos - Electronic Pianos - Synths & Keyboards The DPBSD Project!

In my opinion, dewster's test is similar to that of a performance benchmark for computers.

In such benchmark tests my little netbook scores rather poorly when compared to a top of the range desktop PC. Yet it still allows me to happily browse the PianoWorld forums, watch clips on YouTube, and check my email etc.

Similarly, a P85 may score rather poorly when compared to a multi-gigabyte VST such as Ivory, however I'm sure it's perfectly fine for most individuals - regardless of their playing ability.

Benchmarks are fun for geeks, but they simply do not tell the full story.

Cheers,
James
x

Good point. And a painful reminder that Pianoteq 3.5 will not run on my notebook...

From what I can tell, no one is making the argument that the DP BSD project is telling " the whole story "; that is simply a straw man of your own fabrication.

If something is " perfectly fine " depends of course on what each individual's expectations are.

I, for one, would have very high expectations on both noticeable sound quality / controllability and on improved supporting technology if I were being asked to spend many thousands of dollars on a CP1 versus a P-85. I am sure that I am not alone.

Perhaps James, but there may be quite a few geeky individuals new to digital pianos and perhaps looking for an instrument, that might be stupid enough to believe a test that might possibly be inaccurate, and done by only one individual using samples that may or may not be even be from the instrument stated.

Maybe you and I might not choose or decide to pass on an instrument (or a laptop) based on an alleged "benchmark" test, but maybe there are people who might be interested in a Kawai or Roland, who might look elsewhere because of such posted findings or they may even pass on this information to their friends.

It's one thing to say we may or may not like a particular sound or piano based on our own criteria, but to post a test who's raw material may or may not even be from the instrument in question, on a public forum, is, in my opinion, more than a tad risky.

No doubt Dewster thinks he's doing everyone a service, but I'm afraid that sometimes even a well intentioned action can turn around and bite that person in the ass.

My withdrawal (finally) from this thread has more to do with not wanting to encourage and/or support this alleged "benchmark" test, than it has to do with any slight I may feel about a favored instrument or manufacturer...I've always bought what works for my needs, regardless of the name on the panel and certainly not based on an amateur tester's results and/or opinions.

But, as I said in my post's opening paragraph, there may be some who take these tests quite seriously, and if for some reason, this alleged "benchmark" test is found to have serious mistakes, the integrity of this forum may suffer.

Some test results from this thread have already been posted on another forum, so it's already starting to spread.

That's all from me...I've said what I needed to say, and it is my own opinion.

I'm staying out of this from now on.

Snazzy

I fully agree that benchmarks alone can't be used to judge an instrument. In the same way engine/car measurements alone can't be used to judge a car. CPU speed alone can't be used to judge a PC. And brightness and contrast measurements alone can't be used to judge a beamer.

But still, if I do buy a new car, I sure as heck want to know *before* the purchase, how the engine performs. If I buy a PC, I definitely want to know how fast the CPU is. And if I buy a beamer, I want to know how bright and contrasty it is. DON'T YOU ALL, GUYS?

This thread was only ever meant to reveal some aspects of the underlying technology of current DPs. If some of you guys (not you, James) misunderstood the purpose of this thread to be more than that, then I think that's a simple misinterpretation of this thread's purpose.

@dewster, maybe you could put a disclaimer on the first post of this thread, explaining the purpose (and the limits!) of these tests?


So because some people new to DPs might misinterpret the test results to mean more than they do, we should stop doing technical analysis? Is that what you're saying? That's a weird point of view IMHO... Instead let's be more constructive: Your worries might be (partly) justified, so let's try to make the purpose and limits of this thread more clear. But there's no reason to avoid doing technical analysis, just because some people might misinterpret it!

As for the test being possibly inaccurate and dony by only one person: The MP3s are publically available. If you want to refute the accurateness of the tests, you are *very* welcome to double check!!

As for the samples maybe not being from the right instrument: The CP1 sample comes from Lawrence (dewster said that and Lawrence - who also participated in this thread - didn't refute, so I take that as sufficiently proven). So let me ask: Why would you doubt Lawrence's CP1 sample? And if you do, why did I not see you post doubts when Lawrence posted his personal CP1 preview?

FWIW, personally, I find this thread very informative and I hope it will continue. I planned to buy a CP5, and I may still do so. I will also test play Kawai and Roland DPs to find the best action and sound for my taste. The DP BSD tests will not be the one and only factor that decides my purchase (far from it), but I do find the tests very useful as an additional source of information, which is not available anywhere else.

I agree with madshi totally. To even put forward the whiff of possibility that the instruments assessed may not actually be the instruments they are claimed to be casts the most serious doubt over Dewster's integrity and that is a shameful thing to do without evidence. And ridiculous bearing in mind the mp3s are available to anyone In Dewster's shared folder.

And Lawrence has also withdrawn because he doesn't like the message of the thread. We all want our choices to be reinforced by positive opinion from others, this is human nature. But Lawrence, if you are still reading, is what Dewster has done really any different to your constant criticism of the V-Piano's midrange? (not that I think you were being unreasonable in any way by airing your opinion (which I agree with to some extent)). I agree with your point that there may be other considerations with the CP1 with the interaction of the new sample replay system they are using and some of the modelled elements that Dewster's test does not examine. But the test is just one way of assessing certain aspects of a DP. Would you be taking your bat home had the test shown the CP1 in a good light?

I'm looking forward to trying the CP1 and 5 and do expect to be impressed but Dewster has revealed they are using note stretching! I mean, come on...2010 on a flagship instrument and they are using note-stretching...regardless of what else Yamaha's technology offers, this is disappointing in my view. Wouldn't be a deal-breaker for me but leaves a slightly bad taste in the mouth in a four-and-a-half-grand DP.

Steve

He hasn't revealed anything at all.

The only place you will hear any note stretching or looping when playing a CP1 is in your imagination.

Ok well that's what his test results say. Would I be right in thinking Yamaha have given you a CP1 or at least given you very favourable terms? If I remember rightly you were getting a 5 as well and had pre-release access to these instruments. Just how impartial would you claim to be? What is your side of the deal? That you shout from the rooftops on forums about how good these pianos are?

Huh!? I'm not angry and I'm not trying to kill any messengers or anybody else for that matter, just offering an alternative viewpoint! Actually I was really reacting to Madshi's post about stretching in the CP1/5. I'm guessing that if Yamaha are including stretching and looping in their flagship stage piano it's probably for a good reason, not because of ignorance, laziness or greed. You may well believe differently as is your right.

I'm going to wait until I can get my sweaty little paws on one before I cast judgement and since the only people I know of that have actually played the thing have raved about it I'm guessing I'll probably quite like it!

This is also an excellent point.

The proof of the pudding is in the eating.
Humans hear in their brain and if our brain doesn't register the difference, then the question is if the difference is important. After all, a good part of playing the piano well is knowing how to take advantage of human's hearing deficiencies and using techniques to give the illusion of a singing tone.

If the player or audience or recording engineer can't hear it and they all agree that it sounds good, then the underlying technology and sound wave patterns may be moot.

On the other hand, whether or not we are consciously aware of a sound wave that is inferior in its behavior does not mean that our unconscious is not affected by it.

Kids can watch cheap Chinese cartoons with jumpy coarse animation frames and still get into the story; they can listen to super compressed Mp3's and still enjoy the music. As fewer and fewer grow up experiencing high fidelity, fewer will demand it.

I still don't see the point. Pianoteq is completely virtual modeling, but I don't like the sound signature at all. Even Truepianos is (in my ears!) way better just because of the sound signature. Same applies to my CA-63. From a technical point of view I'm sure that Pianoteq will "outperform" CA-63, but still I like the sound signature better. Am I going to condemn dewster because of what he might reveal? Most probably not...

I'm completely impartial I am by no means tied to Yamaha.
I have Roland boards, I have Yamaha boards, I have Kurzweil and Korg boards too. I do not get a different or better deal from Yamaha then I would get from any other manufacturer. The test prove nothing the only proof is in the playing...... and the CP1 has no stretching or looping when played. It might show it on technical software but that's not a real world situation the fact is you can't hear any and that's all that counts.

I do believe to hear a small change of tone after every some notes in the lower parts of the chromatic passage in the CP1 MP3. Is it my imagination? I don't think so, but who knows for sure...

Roland says this:

"The decaying sound of an acoustic piano not only decreases in volume, but also gradually changes in color to a softer tone."

You can't reproduce this with looping - unless you post process the data to make the looped samples appear softer over time. I guess that Roland is doing exactly that with their SuperNATURAL implementation (because I don't think that full length samples fit on an SRX card). Does Yamaha do that, too? @dewster, would you see such a post processing in your pan/spectral view?

hmmm Being a computer geek myself I am reminded of the time when nVidia was found to be cheating in the popular 3DMark benchmark test. People were upset because they felt mislead by nVidia because it would perform better in benchmarks then the competition. However the counter argument was it was simply a benchmark and not indicative of game performance which is the only thing that matters of course because you are not playing a benchmark.

This benchmark is simply a good way to determine on a strictly technical level how a digital piano performs compared to another digital piano. Just like in video card benchmarks just because you cannot see the difference between 150fps and 180fps does not make the results useless.

So thanks dewster for taking on this task and please don't be dissuaded.

Hi Madshi,

This test is generally at the first part of the opening page, so it will be viewed by many newcomers, including my students.

I know Pete (Snazzy) well enough to trust his integrity, but I believe where he is coming from is that someone could easily upload a sample from a different instrument than stated.

I did not see Pete make any reference to Lawrence's CP-1 sample, nor did he single out any others; he just made a point that the tests could easily be bogus, if an individual wanted it that way.

Dewster is merely the tester; how does he know that every sample MP3 is from what the donor says it is?

I believe that Pete's concerns are more than "(partly) justified", and if we all go blindly into totally trusting the accuracy of the results, we may lose out on a perfectly nice piano.

Sometimes planting the seeds of sonic artifacts that appear in the bogus sample, can make an unskilled listener hear them in the actual piano, sort of the "power of suggestion".

I'm certainly not proposing that the testing be stopped or be totally not trusted, but we should at least recognize the possibility of errors, in the way the test is carried out.

I guess what I am trying to say, is that for this analysis to be entirely accurate, the tester alone should be recording the MP3's of each tested piano, using the same equipment for gathering each sample, and while this doesn't wholly eliminate the possibility the tester could be biased (which I doubt, as Dewster seems to have no allegiance to any company) it would give the results far more consistency.

Since the above is not realistic, the tests as they are will have to suffice; but I will be taking Pete's statements into consideration, and hopefully, you and others will as well.

Best regards,

Colleen

Well, as far as that goes, anyone who takes the advice or observations of anonymous people posting anonymously behind cutesy avatars on a public forum that exists primarily for its entertainment value as being more important than their own shopping, auditioning and due diligence probably deserves to miss out on a good piano.

Thank you. That was exactly my point, although I do find some of your posts rather informative as well as entertaining.

Regards,

Colleen

There have been lots of great input in this thread.

Ah, I ought to perhaps point out that I too consider myself a digital piano geek - and there's absolutely nothing wrong with that!

I believe I mentioned this point a little while ago, however a few months ago I was chatting to a Yamaha DP engineer about sampling, and why even the top-end Clavinovas and stage pianos didn't utilise 88-key sampling. He argued that 88-key sampling (i.e. not stretching samples over multiple keys) wasn't the most efficient way of capturing an acoustic piano, and that devoting more memory to longer samples (and thus reduced looping) results in greater authenticity.

Interesting - he really got me thinking.

Indeed, I wonder what other forum users would prefer?

88-key sampling, with no note stretching but maximum 4 second samples, or
44-key sampling, with 2 note stretching, but maximum 8 second samples.

Cheers,
James
x

@Colleen_500,

I totally welcome any constructive suggestions, and I think (hope) dewster will, as well.

You're right, there is a chance that someone uploads an incorrect MP3 to make a competitor look bad. But the buzz on the last few pages was about the CP1 BSD "review". And, given the circumstances, IMHO that MP3 is unlikely to be incorrect. Anyway. What can we do? I don't think dewster will have a reasonable chance to create test MP3s for all DPs on the market. Maybe a workaround would be to ask for MP3 uploads for each DP from two different people, where at least one must be a long time forum member? As long as these requirements are not met, the "BSD review" could be classified as "preliminary / not confirmed yet".

Thoughts?

P.S: How difficult would it be to setup a mobile DP BSD test environment? With such a mobile test lab (ideally just a good notebook with some USB devices?) one could go into a DP should and simply run the DP BSD test on all DPs in the shop!

Your post really does have me thinking, James, and to get this out of the way at the very first, may I ask what type of sampling Kawai uses?

Snazzy

Interesting! What I'd really like to know is what stops him from going 88-key sampling *and* using longer samples on top of that? I mean why does he seem to be concerned about using memory efficiently? Memory prices have fallen so much! They still do, every year...


I don't want to choose. I want it all. 88-key sampling and full length samples. And I'm willing to pay a premium, if that's necessary.

Anyway, if we *have* to choose, then probably an intelligent method is the way to go. No stretching for mid range, some stretching for bottom/top, just like the CP1 seems to do. But still, I don't like the whole idea of compromising.