Piano World Home Page Forums Our Most Popular Forums Digital Pianos - Electronic Pianos - Synths & Keyboards Postprocessing binaural sound for normal stereo output piano

As promised in some other threads, this weekend, I tried using the Creative SX-Fi DSP to create binaural sound using a custom HRTF for a normal stereo headphone piano sound - in particular I tried using the Bösendorfer VI sound on my N1X.

This post is to summarise what I tried and the results.

TL;DR: Works but doesn't work. I could create very realistic 3D binaural sound, but the latency is too great to be usable for piano.

The test configuration was:

• Raspberry Pi Zero W kit with Raspbian loaded from a NOOBS microSD card
• Behringer U-Control UCA222 2-input / 2-output USB Audio Interface
• 3.5mm to 2-male RCA adapter stereo audio cable
• Micro-USB HUB adapter with power
• Creative SXFI Amp with headphones

Not going to go through the details of the initial Raspberry Pi Zero W installation, but basically I connected the Raspberry Pi Zero W to a display and keyboard, installed the Raspbian OS from the NOOBS microSD card, and used the Linux ALSA command, 'alsaloop,' to set up a loopback between the Behringer UCA222 and the Linux-compatible SXFI Amp device, set to a 2.0 millisecond maximum latency.

Physical connections were Micro-USB HUB adapter with power connected to the micro-USB port on the Raspberry Pi Zero W. The Behringer UCA222 connected to the HUB adapter, and the SFXI Amp also connected to both the HUB adapter and headphones.

The idea was the VI's sound from the N1X would pass out through the 3.5mm stereo jack on the N1X, through the 3.5mm to 2-male RCA adapter stereo audio cable and into the UCA222 which is connected to the Raspberry Pi Zero W via USB. Then on the output side, also connected to the Raspberry Pi Zero W via USB is the SFXI Amp, which in turn is connected to the headphones. In theory, headphone stereo from the N1X is converted to digital which passes through the Raspberry Pi Zero W and is sent on to the SXFI DSP processor and turned into binaural sound for the headphones. The Raspberry Pi Zero W, itself, was set up with SSH so that it could be managed remotely without a display or keyboard after the initial setup, and the 'alsaloop' command was set up to be started using init so that it would execute immediately on power-up.

Results: The Raspberry Pi Zero W worked as a USB host/master for the Creative SXFI Amp, as intended. Stereo piano VI sound passed out of the piano and into and then through the UCA222 to the Raspberry Pi Zero W to the Creative SXFI Amp and finally to the headphones as planned. However, the latency was too high. The total latency was accurately measured using an oscilloscope and it was found that of the total latency, the most significant portion was 30 milliseconds due to processing by the Creative SXFI Amp's binaural processing. This is likely due to the convolutions being performed on very large bufferfuls of audio sample data within the SFXI DSP processor.

So while the sound was very good (spacious and 3D), the entire apparatus is not suited to real-time piano playing. The play would not feel connected to the piano.

However, if anyone has any idea of creating their own HRTF convolution engine using smaller bufferfuls of audio samples, this overall approach seems to be workable, even if the Creative SXFI Amp is not useable for low-latency digital piano playing in particular.

Any idea of how would it fare with a regular X86 PC or notebook?

EDIT: elaborating a little more, say you take a light Linux distribution (Lubuntu or Xubuntu to avoid bloatware) in a USB stick and configure it similarly to how you did with Raspberry Pi. How would it perform? If performance is acceptable, then it is a limitation of the ARM architecture.

OTOH, it may be that the software is not optimized to use SIMD capabilities built into ARM coprocessors (NEON or whatever they call it nowadays).

I’m waiting for mine to arrive. It’s currently in London and will be delivered to the Bulgarian forwarder soon, so I guess at least a week or two... But I’m going to use it for my everyday music listening at the office. You may be surprised but I listen to symphonic music more than 80% of the time, apparently much more than piano music. I really love classical symphonies not the least because of the sound so I hope this device can make that gorgeous experience even more realistic and 3D. Thanks for letting me know about that device. Hopefully I will like it. And unfortunately I’ll have to ask a colleague to configure it for me since the app only works with Android phones But they say it’s a one off process. And I ordered the kit with the Aurvana SE phones which should be very well matched by Creative.

Tyrone, in plain English, I guess that means it will not connect to a digital piano? I wouldn't want to use all of that stuff. I would just like to connect this device between my piano and my headphones.

Tyrone, latency aside, does it make the Bosie sound "as good" binaurally as the CFX binaural sound?

Is there a plain (non-binaural) CFX sound somewhere that you can put through your setup to compared it against the actual Yamaha-made CFX binaural sound? I'm interested in seeing if the Super XFI processor is up-to-par with what Yamaha did to the CFX sound binaurally. It'd be more apple to apple to compare CFX to CFX rather than Bosendorfer on SXFI to CFX on N1X headphones, if it can be done.

I guess it also makes sense now why the SXFI amp doesn't accept analog audio and only accepts the USB digital data. They probably already have enough latency processing the USB digital audio from the computer as it is, let alone wanting to deal with another phase of accepting analog audio through an A/D converter and add on even more latency on top.

As a follow-up to the Vol-meister ... If it sounds good (or even if it doesn't) could you be persuaded to post the audio?
We all have headphones. We'll experience a full 24 hour latency! But that doesn't matter.

The audio is optimized only for a specific pair of ears. You need to make pictures of your ears and face from different angles so that the software calculates your ear geometry and then produces corresponding binaural sound. That’s the crucial part and the reviewers praise it. Also, it depends on headphones and there’s a list of supported ones. Which is why even if an audio is posted, it won’t just sound good with your ears and headphones.

Based on the oscilloscope measurement, the SXFI's HRTF-based convolutions are taking 30 milliseconds by themselves. 30 milliseconds by itself, without any other delays, is unacceptable in my book, for piano playing. The pianist would feel disconnected from the instrument.

Therefore having a regular X86 PC or notebook instead of a Raspberry PI Zero might decrease the latency a bit, but not to an acceptable level as you would never get the latency below the 30 milliseconds that the Creative SXFI uses/causes.


There is certainly a limitation of the ARM architecture, and especially as the Raspberry PI Zero W is very minimal, but I was still able to get the 'alsaloop' latency down to 2 milliseconds taking only 50% of the CPU. The hard limitation will be the SXFI DSP processing as mentioned above.


The software that really matters is in the Creative SXFI Amp device itself, and we can't do anything with that or optimise further. Clearly it was meant for applications which are not latency-sensitive, such as listening to your music library, vs. playing piano and hearing the sound which is very latency sensitive.


I think you will find it very enjoyable in that use case, as I do. You can listen to piano music too from your music library or streaming service. You just can't play piano and listen to the sound. That is too latency sensitive for this device as my testing this weekend has shown.


In plain English, this is not the absolute minimum necessary to play sound from a digital piano through the SXFI, but it is pretty darn close. And the latency through the SXFI device itself is too high for you to have a satisfying piano playing experience. As mentioned to CyberGene above, of course you can listen to piano playing of others out of your music library or streaming service, and it will be fine and satisfying, but not if you are the one playing and hearing the sound. In that case, the latency will be unacceptable.


It sounds very good to me and definitely binaural. A lot more spatial.

But when the latency was too great for normal piano playing, I didn't think it would be worth recording.


I was testing a setup where the binaural mode on the SXFI Amp could be turned on or off. I just didn't have it set up to record. Let me see if I can recreate that set up. For recording, I might need to find another cable.


OK, will keep this test scenario in mind. Just keep in mind, no matter how great (or poor) it sounds, it is not practical for actual piano playing since the pianist will feel detached from the instrument when there is latency.


My guess is the processing power is enough, they are just using too large a buffer of data for the HRTF convolutions. Just my semi-educated "guess!" My educated guess is that using the same algorithms but smaller buffers might decrease the latency enough to make this usable. We can't do that though without the source code to the SXFI Amp.


Yes, I will see about recreating the scenario so that I can record with and without the SFXI. Keep in mind what I said above about unacceptable latency though.


True. The only one who will be truly satisfied with the recording "I" make will be "me!"

The fact that the bulk of the latency is in the SXFI processing is unexpected to me, because I've seen review videos on YouTube of the SXFI amp and the reviewer said something like he now won't play games without using the SXFI amp anymore because he's been spoiled by it. But I thought gamers are one of the MOST sensitive bunch on latency, whether it be video latency or audio latency.

While I did forget about the SXFI's need to customize the sound to give the most optimal binaural experience, I also think that it's kind of gimmicky on the other hand because while there may be differences in people's hearing profiles based on their facial features, I don't really think it'd make THAT much difference in the first place. But I don't own the SXFI Amp or Air so who am I to say this?

BUT, if you look at how Yamaha has been able to make its single (non-customized) binaural sound very pleasing to ANYONE's ears, you'd have to wonder why does SXFI need to have your facial profile to give you a customized-only binaural experience? That's why I think it's a marketing gimmick to hype up the value of the SXFI technology even more.

Well, I called 30 milliseconds the most "significant" portion of the latency because it is the part that is non-compressible without access to the source code for the SXFI Amp. The Creative engineers might be able to drive this down, but not us "consumers."

Now, note that I am taking an analog audio signal, converting it to digital via an ADC inside the UCA222, and then there is a 2.0 millisecond sync before it is put out via USB to the SXFI. I mention this because the user you mention probably plugs his SXFI directly into a USB port on his gaming computer, which could, if his gaming rig is fast enough, reduce his latency to little more than the 30 milliseconds for the SXFI's HRTF convolution processing.

Also, it's possible that if sound is only a supporting element of gaming (video being the primary element), a 30 millisecond delay could be acceptable since the average human reaction time to audio stimulus is 170 milliseconds, which is almost 6 times as long as the 30 millisecond delay that was measured. However, for something like piano, I think it would result in the pianist feeling detached from the instrument and the quality of playing and the pianist's enjoyment might both suffer.

Lastly, I only tested the SXFI Amp. It's possible the SXFI Air and Air C have different latency than the SXFI Amp.


People who have tested this find that it is noticeable. Research into HRTF also shows this really matters in how individuals hear. If you and I swapped ears (and ear canals), both of us would likely be shocked at how our brains interpret the sounds differently after the swap.


If you read up on HRTF, you'll see It's more than just a gimmick. It's backed up by some acoustical science.

I'm just asking that if the CFX binaural sound can fit and work just fine and well for everybody/anybody, why would the SXFI sound need individualized HRTF to sound good? One (the CFX binaural) is totally independent from HRTF, while the other (the SXFI binaural) is dependent on HRTF, and both incorporate/create binaural sound.

That’s an easy one. The uncustomized binaural suit is a one-size-fits-all outfit one buys off-the-rack. You put it on, it more or less fits, you won’t be naked when you go out. The custom HRTF convolutions is the suit tailored to your specific measurements. Fits perfectly everywhere and doesn’t make you look fat. Sure CFX binaural sounds good, but in theory, HRTF sounds better, more realistic, etc

I think you're hitting the nail on the head there by putting in a disclaimer yourself in bold -> "In theory". We don't know how much better the HRTF CFX binaural sound would be compared to the one-size-fits-all Yamaha CFX binaural sound. All we know is that the Yamaha CFX binaural sound is already perceived to be VERY good by almost everyone. It blows other non-binaural piano sound out of the water, like the also much-touted Bosendorfer sound, simply because it's binaural and the Bosie sound is not.

I would surmise that the big difference is whether it's binaural or not binaural. Whether it's one-size-fits-all or HRTF is a much smaller difference, maybe almost miniscule compared to binaural vs sans-binaural. Using your analogy, it's being naked vs being clothed that makes the huge difference. It makes being clothed vs being tailored clothed become a miniscule difference compared to being naked. That's why I think binaural is big, but HRTF binaural is just gimmicky on top of it.

I agree overall concerning degree. I just don't agree it is a "gimmick," per se, since they are trying to reach the point of "being there" and not wearing headphones. Further improvements in HRTF take into account the position of the user's head. For example, if you turn your head toward the sound, shouldn't it hit one ear before the other? Shouldn't it be louder in that ear? Would that also be a "gimmick?"

Also, keep in mind that those that try SXFI uncustomized for different people do say it sounds different. I haven't tried to customise it for someone else and then listen myself though.

EDIT: I personally would not be surprised if the difference was greater than that one would experience with different models of headphones. But again, this should be tested.

Hi Tyrone,
I wonder if you have explored other options for creating a binaural effect with headphones. This plugin looks promising,

https://goodhertz.co/canopener-studio

It came up on the pianoteq forum as a feature request to beef up their rather weak binaural setting.

Paul H

Yes.

To be less terse it's a good plug-in but by my ear, doesn't work as well as SXFI (and doesn't work with SXFI for the same reason that SXFI doesn't work with the CFX binaural sound). It also takes a computer and I was trying to set things up to be as minimal as possible on my headphone chain this weekend. However, the reality is that if I want to use SonicWorks, I may need to involve my computer anyways. The latency through CanOpener should be tested though to even see if it is a possibility with the piano. There is a free trial version.

Sounds like an invitation to me😀 I had no idea you had already tried those approaches. I suppose I had hoped they would provide a binaural experience equivalent to Yamaha CFX but alas not by the sound of it. I will have a go with the trial version. You never know your luck. Thanks Tyrone for all your hard work.

Paul H

Paul, I did try CanOpener with CFX on the N1X those few weeks my pianocomputer was still alive after I got the N1X (having a replacement DAW workstation built for me now). The results were very poor. CFX, CanOpener, and SXFI can't be mixed-and-matched, I think because each do convolution processing and convolutions on convolutions, oh my!

I have used CanOpener with my stereo system and it sounds fine. I do think SXFI sounds better, so if you have an application (like a stereo system) which is not latency-sensitive, I would recommend you consider SXFI as it is definitely better than CanOpener, although about $100 more. For obvious reasons, neither CanOpener, nor SXFI sound right when using speakers.

Now for latency-sensitive applications, such as playing your piano and hearing the sound, well, CanOpener should be measured for latency. PW member, RobR, has reported that he is using it in combo with Reference 4 for recordings. Don't know if he is also trying to use it for monitoring as he is playing his piano though.

Speaking of Reference 4, there is a free trial of that also, so if you are in the mode of trying out some free software, I recommend that for headphones too. Both CyberGene and I have had good results. I was using it for my piano too before my pianocomputer broke. If you use Reference 4 instead of SonarWorks' TrueFi version of Reference 4, the latency is quite acceptable for piano once you turn off phase correction (the problem with TrueFi used to be that one couldn't turn the phase correction off, which makes the latency too great for playing the piano - SonarWorks told me they were not going to correct this )