Piano World Home Page Forums Our Most Popular Forums Digital Pianos - Electronic Pianos - Synths & Keyboards A festival of numbers -- DPC Latency question

Hi friends,

Warning: this is going to get computery, and so those who are looking for musical content, click out of this thread.

Do I still have a crowd? I hope so.

OK, I wanted to knock down my buffer size from 64 samples to 32. However, as expected, there are clicks and pops, particularly with Ivory American Concert D. Knowing that 32 samples might not be stable, I checked my DPC latency using LatencyMon. I will post my entire report text in a separate post on this thread.

I would like you to help me understand what I should do next. As I'm not that computer efficient, I'm looking for easy step-by-step guidance to what I should do.

Note that I can only deal with things on my system. As Bios / UEFI is not equipped with text-to-speech, I can't work with that interface on my own.

Thank you!

David

-Report on next post-

_________________________________________________________________________________________________________
CONCLUSION
_________________________________________________________________________________________________________
Your system appears to be having trouble handling real-time audio and other tasks. You are likely to experience buffer underruns appearing as drop outs, clicks or pops. One or more DPC routines that belong to a driver running in your system appear to be executing for too long. One problem may be related to power management, disable CPU throttling settings in Control Panel and BIOS setup. Check for BIOS updates.
LatencyMon has been analyzing your system for 0:05:43 (h:mm:ss) on all processors.


_________________________________________________________________________________________________________
SYSTEM INFORMATION
_________________________________________________________________________________________________________
Computer name: DESKTOP-JDKJUKS
OS version: Windows 11, 10.0, version 2009, build: 22000 (x64)
Hardware: Precision 3540, Dell Inc.
BIOS: 1.13.0
CPU: GenuineIntel Intel(R) Core(TM) i7-8565U CPU @ 1.80GHz
Logical processors: 8
Processor groups: 1
Processor group size: 8
RAM: 32615 MB total


_________________________________________________________________________________________________________
CPU SPEED
_________________________________________________________________________________________________________
Reported CPU speed (WMI): 1792 MHz
Reported CPU speed (registry): 1992 MHz

Note: reported execution times may be calculated based on a fixed reported CPU speed. Disable variable speed settings like Intel Speed Step and AMD Cool N Quiet in the BIOS setup for more accurate results.


_________________________________________________________________________________________________________
MEASURED INTERRUPT TO USER PROCESS LATENCIES
_________________________________________________________________________________________________________
The interrupt to process latency reflects the measured interval that a usermode process needed to respond to a hardware request from the moment the interrupt service routine started execution. This includes the scheduling and execution of a DPC routine, the signaling of an event and the waking up of a usermode thread from an idle wait state in response to that event.

Highest measured interrupt to process latency (µs): 2935.50
Average measured interrupt to process latency (µs): 17.37410

Highest measured interrupt to DPC latency (µs): 2918.90
Average measured interrupt to DPC latency (µs): 5.978809


_________________________________________________________________________________________________________
REPORTED ISRs
_________________________________________________________________________________________________________
Interrupt service routines are routines installed by the OS and device drivers that execute in response to a hardware interrupt signal.

Highest ISR routine execution time (µs): 292.230422
Driver with highest ISR routine execution time: ACPI.sys - ACPI Driver for NT, Microsoft Corporation

Highest reported total ISR routine time (%): 0.033418
Driver with highest ISR total time: Wdf01000.sys - Kernel Mode Driver Framework Runtime, Microsoft Corporation

Total time spent in ISRs (%) 0.039936

ISR count (execution time <250 µs): 532519
ISR count (execution time 250-500 µs): 0
ISR count (execution time 500-1000 µs): 1
ISR count (execution time 1000-2000 µs): 0
ISR count (execution time 2000-4000 µs): 0
ISR count (execution time >=4000 µs): 0


_________________________________________________________________________________________________________
REPORTED DPCs
_________________________________________________________________________________________________________
DPC routines are part of the interrupt servicing dispatch mechanism and disable the possibility for a process to utilize the CPU while it is interrupted until the DPC has finished execution.

Highest DPC routine execution time (µs): 3941.769076
Driver with highest DPC routine execution time: dxgkrnl.sys - DirectX Graphics Kernel, Microsoft Corporation

Highest reported total DPC routine time (%): 0.544390
Driver with highest DPC total execution time: Wdf01000.sys - Kernel Mode Driver Framework Runtime, Microsoft Corporation

Total time spent in DPCs (%) 0.587175

DPC count (execution time <250 µs): 734813
DPC count (execution time 250-500 µs): 0
DPC count (execution time 500-10000 µs): 633
DPC count (execution time 1000-2000 µs): 22
DPC count (execution time 2000-4000 µs): 1
DPC count (execution time >=4000 µs): 0


_________________________________________________________________________________________________________
REPORTED HARD PAGEFAULTS
_________________________________________________________________________________________________________
Hard pagefaults are events that get triggered by making use of virtual memory that is not resident in RAM but backed by a memory mapped file on disk. The process of resolving the hard pagefault requires reading in the memory from disk while the process is interrupted and blocked from execution.

NOTE: some processes were hit by hard pagefaults. If these were programs producing audio, they are likely to interrupt the audio stream resulting in dropouts, clicks and pops. Check the Processes tab to see which programs were hit.

Process with highest pagefault count: msedgewebview2.exe

Total number of hard pagefaults 177
Hard pagefault count of hardest hit process: 129
Number of processes hit: 7


_________________________________________________________________________________________________________
PER CPU DATA
_________________________________________________________________________________________________________
CPU 0 Interrupt cycle time (s): 21.336155
CPU 0 ISR highest execution time (µs): 292.230422
CPU 0 ISR total execution time (s): 1.097061
CPU 0 ISR count: 531879
CPU 0 DPC highest execution time (µs): 3941.769076
CPU 0 DPC total execution time (s): 16.040092
CPU 0 DPC count: 721964
_________________________________________________________________________________________________________
CPU 1 Interrupt cycle time (s): 5.254928
CPU 1 ISR highest execution time (µs): 35.723394
CPU 1 ISR total execution time (s): 0.001182
CPU 1 ISR count: 640
CPU 1 DPC highest execution time (µs): 111.531627
CPU 1 DPC total execution time (s): 0.057565
CPU 1 DPC count: 4951
_________________________________________________________________________________________________________
CPU 2 Interrupt cycle time (s): 2.072181
CPU 2 ISR highest execution time (µs): 38.157129
CPU 2 ISR total execution time (s): 0.000038
CPU 2 ISR count: 1
CPU 2 DPC highest execution time (µs): 327.285643
CPU 2 DPC total execution time (s): 0.021288
CPU 2 DPC count: 4281
_________________________________________________________________________________________________________
CPU 3 Interrupt cycle time (s): 2.958747
CPU 3 ISR highest execution time (µs): 0.0
CPU 3 ISR total execution time (s): 0.0
CPU 3 ISR count: 0
CPU 3 DPC highest execution time (µs): 89.887550
CPU 3 DPC total execution time (s): 0.010741
CPU 3 DPC count: 1093
_________________________________________________________________________________________________________
CPU 4 Interrupt cycle time (s): 1.283756
CPU 4 ISR highest execution time (µs): 0.0
CPU 4 ISR total execution time (s): 0.0
CPU 4 ISR count: 0
CPU 4 DPC highest execution time (µs): 79.119980
CPU 4 DPC total execution time (s): 0.009705
CPU 4 DPC count: 1836
_________________________________________________________________________________________________________
CPU 5 Interrupt cycle time (s): 1.319452
CPU 5 ISR highest execution time (µs): 0.0
CPU 5 ISR total execution time (s): 0.0
CPU 5 ISR count: 0
CPU 5 DPC highest execution time (µs): 62.745984
CPU 5 DPC total execution time (s): 0.001415
CPU 5 DPC count: 250
_________________________________________________________________________________________________________
CPU 6 Interrupt cycle time (s): 1.791934
CPU 6 ISR highest execution time (µs): 0.0
CPU 6 ISR total execution time (s): 0.0
CPU 6 ISR count: 0
CPU 6 DPC highest execution time (µs): 50.354920
CPU 6 DPC total execution time (s): 0.002453
CPU 6 DPC count: 565
_________________________________________________________________________________________________________
CPU 7 Interrupt cycle time (s): 1.897621
CPU 7 ISR highest execution time (µs): 0.0
CPU 7 ISR total execution time (s): 0.0
CPU 7 ISR count: 0
CPU 7 DPC highest execution time (µs): 96.845382
CPU 7 DPC total execution time (s): 0.004772
CPU 7 DPC count: 529
_________________________________________________________________________________________________________

I wish I could help you with a detailed analysis. Hopefully others can.

But I'm wondering if you've measured true end to end latency?

From what I've experienced, digital latency is typically faster (objectively and audibly) than playing acoustically. Playing a silent piano, I always hear the digital sound before the acoustic sound. If your VST is anywhere near well-optimized, and running at 64 buffer on most systems would seem to qualify, I would expect this may not be quite the bottleneck you think it is?

On my Mac system, I could play perfect output using Garritan CFX at 64 buffer, 44khz. I also tried to stretch that down to 32 buffer, but started getting cracks and pops, and it honestly wasn't audibly lower latency to me. In general, getting down to 32 buffer isn't an easy or possible thing for many PC setups, most of the ones I've heard of here end up at 64 to 128 for stable play.

Also, I'm a bit suspicious that a tool like latencymon is really that useful. It can only see what it can see, which is a small segment of the audio pipeline, which starts from the keypress on your controller, to the MIDI transmission, to the USB polling to driver engagement and routing. And the same on the output side going to the DAC and ultimate signal output to speakers or headphones.

All of this is to say you might want to try setting up a rig to measure true e2e latency--comparing a note played through your built in sound engine if your DP has one to the note output through your VST. That would tell you how much slower your VST is, and you can also compare that to measurements taken with acoustic pianos to see if your VST output is in line with expectations.

Actually 64 is good on my end, but I was thinking, if I can slash the buffer size down just a little bit? I really enjoy the 32 buffer size, and I find the response more snappy than 64. That said, I don't know the tools to measure this end-to-end latency, and I just read an article on solving DPC-latency issues from Sweetwater. That's how LatencyMon entered the picture.

I would not do any changes. 64 is not bad.
DELL laptops are known for having inefficient ACPI, and Windows is not a real-time system, its task scheduler sucks as you can see by comparing load on different cores.

I totally trust that your ears are telling you the right story between 64 and 32 (I strongly believe people who are trained/attentive are able to tell sub-5 to 10ms differences latency in piano playing).

Measuring end to end is a chore, and requires work outside the PC (microphon setups, recording, etc.). The cleanest way I heard was articulated by (former?) forum member MacMacMac, who described splitting a stereo cable to take one channel output from his DP, and one output from his VST, both piped into the stereo line-in input on his PC. That way, when he pressed a note, he could use the DP sound as a baseline, and see how many milliseconds longer it took for the VST to register. Then he could change buffer size and sample rate and other settings and actually quantify the differences against the same baseline.

Well, I use a Kawai VPC1, a midi keyboard controller that's quite popular around here. But I could feel a difference in how fast the VST's responded to my key presses from 64 to 32. If I could make my experience better, I'm willing to modify my system.

Thank you for your opinion, Vlad! I'm sure your experience has also been similar to mine. If this modification effort is more work and less successful than expected, I'll stay on 64.

several problems with laptops, the u-series cpu means its highest turbo clock is not sustained, so dpc will go down, then back up. you are also stuck with asio4all drivers which have higher latency than pcie based soundcards with its own native asio drivers. this driver issue alone has higher impact on overall latency than dpc or selected buffer size.

for windows laptop, you need a pcie enabled (thunderbolt) usbc port to get the lowest latency via a thunderbolt based sound card, or a breakout board to use a desktop card.

for all these reasons, people use macs, because of their native core audio, mac's equivalent of asio, plug and play, no need to mess around.

if you're married to windows, the cheapest route without all the hassle is to buy a desktop with a fast cpu, and a pcie soundcard that has native asio drivers.

Well, I have a FocusRite 4I4 audio interface connected to the one and only USB-C port on my laptop. Hope this helps.

David

The problem lies in Windows design. It always treats hardware interrupts as tasks of highest priority than anything else, that in theory must be executed quickly and in a timely manner to not miss information from the hardware and to not interrupt normal processing for too long, but on practice its speed and efficiency depends on driver implementations, including 3rd party ones.

On seeing any interrupt signal, CPU stops all cores no matter what it was doing at the moment, and calls a special piece
of code called an Interrupt Service Routine (ISR). This ISR either processes this hardware event immediately, or puts the data from this interrupt in a queue where it can be processed later.

ISR itself must finish very fast, to let CPU return to whatever it was doing before the interrupt occurred.
This is why most of the time, because processing of the interrupt requires longer time than ISR can reserve, ISR will just capture information about the hardware event, and schedule a Deferred Procedure Call (DPC) to finish the processing of the event later.

Now, all DPC are stored in a queue, and can have one of 3 priority levels. If a poorly written driver has higher or the same level DPC code that can run relatively long, it will delay execution of other DPC waiting in the queue, including DPC that handles your real time audio streaming data. As a result, audio buffer can become overfilled before it is processed by your DPC, and this will result in drop-out.

ACPI is one of those drivers that are written very poorly in regards to real time processing.

As you can see from LatencyMon report, the highest DPC execution time was almost 4 milliseconds, and this happened during a very short 5 min analysis timeframe. This is terribly long execution time for a DPC, and you can do nothing about this. I am positive that if you ran the report for longer, you would see an even longer maximum DPC execution time.

CPU with higher clock will reduce the DPC execution time, but only if you disable all power management features that reduce CPU clock speed when it is not under heavy load. And many of these configuration options are hidden by default and can be seen only if you tweak the Windows registry.
Audio processing does not load CPU much which means that by default the CPU core speed will be low.
Laptops are configured as power efficient as possible, and run at lower CPU speed pretty much all the time. Add to this that different cores can also run at different speeds, and the bad DPC can happen to be scheduled for execution by a slow core.

Thank you for a detailed discussion that is not hard to understand, Vlad! Basically I'm concluding that

1. actually doing something about DPC latency is very hard on a laptop;

2. The design of Microsoft Windows will complicate this matter even more.

And 3 Not all drivers can respond very well, and the process of resolving DPC issues can be tedious.

So in conclusion, staying with 64 sample buffer size is the best to go.

Am I concluding this correctly?

Thanks for your replis!

David

Yep.

If you haven't yet, disable Dell services, especially supportassist and data vault ones.

These programs are not on my computer, so don't worry about that, haha.

Hi David,

I also spent a lot of time trying to optimise latency on my old Dell XPS laptop. 64 samples at 44.1khz is excellent performance for any computer, especially a laptop.

With the RME BabyFace Pro interface, I can run Garritan pretty well at 48 samples but for a serious piano player 64 is probably more appropriate to minimize dropouts. The "cheap" BabyFace Pro does not offer anything below 48 samples, but some more expensive RME interfaces unlock those lower buffers.

32 samples is very low, albeit sometimes doable with some pro-audio custom desktops and good pcie audio interfaces. Pro audio producers might be running 64 or 128 samples, or more with big projects, but their use cases vary from practicing a piano VST live. These computers are very expensive. I thought someone at GearSpace was running 24 and sometimes 16, but that was exceptional.

Note that the Apple latency figures include safety buffers, so 32 samples reported is "optimistic" but I don't know how big the safety buffers are.

"Overall latency" from your finger hitting the key to the sound hitting your eardrum is really what counts (it is not reported on your computer). "Overall latency" has several components that effectively are fixed. For example, the time it takes the VPC to process your playing and send the MIDI note information. Or the speed of sound from the speakers to your ear drums.

Importantly, as the buffer size drops, the decrease in latency becomes a smaller and smaller percentage of "Overall latency" due to so many fixed components in the chain. By the time you are at 64 samples, dropping to 32 samples makes very little difference in the whole system's "Overall Latency". With a microphone, I measured the "overall latency", the time it takes between striking the key on my Kawaii es100 and sound hitting my eardrum to be roughly 9ms with a buffer of 64 (using loudspeakers about 1 metre from my head). A buffer of 32 drops that delay by about 1 second to about 8ms, so less than a 15% decrease, not 50%.

Headphones are a good trick to reducing latency; by substituting the speakers for headphones, you might save about 3 ms (sound travels about 3 milliseconds per 1 meter). So I measured "overall latency" with headphones dropping to 6ms with a buffer of 64 and 5ms with a buffer of 32. These are all very rough estimates but they aren't that far off.

Part of the latency may be simply calculated as "number of samples / sample rate" follows:
32 / 44100 = 0.7ms
64 / 44100 = 1.5ms
128 / 44100 = 2.9ms

You also could lower latency by increasing the sample rate to say 88200. However, that requires the CPU to work more; that may cause clipping, just as lowering the number of samples will. You could experment with different sample rates but don't expect miracles. For technical reasons, use the sample rate the original samples are stored at; in any case don't change the sample rate by odd multipliers (e.g. it may be ok to go from 44100 to 88200, but not 44100 to 96000)

On another note, there is some latency in a real piano so maybe at really low buffers, there is a risk of the VI playing too fast, a negative latency. I suppose a Yamaha acoustic with MIDI output (and a mic at the player's position with some spakers for the VI) could be used to measure if this "negative latency" is real on that system.

Finally, a few performance tweaks could help slightly but I'm not sure you can run such low latency on a typical laptop:

-You could select "High Performance" in Control Panel\All Control Panel Items\Power Options

-You could install Dell Command | Power manager and select "Ultra Performance"

-You could shut off the wifi and bluetooth and close all programs you are not using

-Your Dell laptop has a 15 watt CPU so has limited processing power (and limited cooling). I think you might be able to boost CPU performance with ThrottleStop software (e.g. Enable SpeedShift and select EPP=0 for max CPU performance, change the CPU limit to 28 watts, undervolt so the CPU to reduce heat, etc.). Undervolting may have been disabled by recent BIOS updates.

Good luck! I really enjoy your playing and the performance videos you recently posted.

Interesting info in this thread. I can add two considerations.

First, like Gombessa I play a silent system (Kawai K300 ATX3 in my case) and both the on board sound generator and VST with 32 or 64 audio buffer can have lower latency than the acoustically generated sound. At 128 buffer size it is on average similar. Of course that may be different on different computers/audio hardware/acoustic pianos, but that is what it is in my case.

Even if your system can handle lower buffer sizes without audio problems (mine can, e.g. Pianoteq buffer size 32) that may cause other problems, and thus not always be optimal. Besides latency in digital audio generation->sound output there is also key press->digital audio generation latency (processing midi events etc). Perceptually you can't distinguish between the two, they just add up to overall latency between action (your finger moving a key) and sound reaching your ear. Using a VST, on my system the key press event detection->digital audio generation latency is probably very low on average (few ms?) but it is variable and sometimes very noticeably so. And the shorter you make your audio buffer, the more your computer is busy with that, and the more latency variation may be introduced in the other part. And larger latency variation around a short mean latency may be much more detrimental to your perception of control than small variation around a longer mean latency. Perceptual studies on instrument playing have shown this (albeit not on a piano).

Hi newerplayer, I posted before I saw you post. As you can see my experience is different with the rest of the latency being effectively fixed. I haven't been able to measure it, but when my computer is under heavy load (windows doing something like download updates) the audio is never affected (no crackles or pops, just continuous sound) indicating audio is still getting sufficiently high priority, but the action -> sound reaching the ear drum is very much so. Somehow there will be very variable and sometimes very long delays in overall latency, but no audio problems.

I forgot to add that jitter might be more important than latency. For example, if you played a 4 note chord on a digital piano, reproduction of the 4 notes would be staggered, even if you played the 4 notes at exactly the same time. Effectively, parallel information becomes serial. This is more of a problem for big chords and fast playing.

I don't think there is anything a user could do for this. Maybe the premium Yamaha & Kawai hybrids boost performance for internally generated sounds but that is just speculation.

A couple of years ago, a board member of the MIDI Association wrote on their comments section that he never saw a keyboard, even modern ones, output MIDI data faster than the old 1980s norm of 31.25 kbit/s. USB cables could run the data much faster than the old DIN cables could, and there is no MIDI speed limit. He speculated that the keyboard makers just wanted to maximize compatibility. So it is possible the AvantGrand, for example, communicates notes to the internal sound engine at a (much) faster speed than 31.25 kbit/s (whilst slowing the data output to the DIN and USB MIDI plugs to maximize compatibility).

I suppose Yamana & Kawai could have plenty of tricks up their sleeves to improve performance . . .