You are not logged in.

#1 2020-04-30 20:45:25

LukeLR
Member
Registered: 2016-03-18
Posts: 6

Monitor memory performance

I'm running Arch on a AMD Ryzen 7 3900x (12 Cores, 24 Threads) and 40GB RAM. However, my CPU maxes out at 80% when using `ffmpeg`, even with two processes simultaneously. The disk is mostly idle, so might the memory speed be the bottleneck?

I'd like to debug this issue further. Is it possible to monitor memory activity, like reads/writes/sec, % of active time, or whether the CPU is waiting for the memory?

Thanks in advance!

Offline

#2 2020-05-01 09:38:36

sabroad
Member
Registered: 2015-05-24
Posts: 139

Re: Monitor memory performance

LukeLR wrote:

Ryzen 7 3900x (12 Cores, 24 Threads) [...] CPU maxes out at 80%

CPU utilization is not a useful metric walks through several tools to measure this, then explains:

“But wait; doesn’t hyperthreading double the processing power of my CPU?” you may ask.
Well, no, it doesn’t.

You're getting 80% of a theoretical 24-core machine.


--
saint_abroad

Offline

#3 2020-05-21 11:28:55

LukeLR
Member
Registered: 2016-03-18
Posts: 6

Re: Monitor memory performance

Hi sabroad, thanks for your reply! Of course, hyperthreading does not double the computing power of my CPU. But as the utilization percentage is relative to the computing power available - no matter much that is - shouldn't it max out at 100% (full available computing power used) for computation-intensive workloads as long as no bottlenecks occur? So if I only see ~80%, doesn't that mean I have some kind of bottleneck somewhere?

Offline

#4 2020-05-21 11:39:59

Ropid
Member
Registered: 2015-03-09
Posts: 793

Re: Monitor memory performance

When the CPU has to wait for the memory, it will basically lock up. That kind of "waiting" will then show as 100% CPU usage. You will not be able to diagnose a bottleneck caused by slow RAM through the CPU usage. You can only see it by comparing benchmarks of two different RAM speeds.

This means when you see 80% usage, there's something else going on. This is something about how your program is organizing the distribution of your task over the threads. It might not be able to do this in a way that you get to 100% usage.

Offline

#5 2020-05-21 13:27:59

LukeLR
Member
Registered: 2016-03-18
Posts: 6

Re: Monitor memory performance

Okay, thanks for clarifying. So even with a memory bottleneck the CPU utilization should still be at 100%. But what could be going on that is utilizing the CPU but not showing up in the CPU utilization? I mean, if the remaining 20% were used by some other task, then CPU usage would be at 100%, wouldn't it?

And of course, it could be possible that ffmpeg is not able to utilize 100% of my CPU. But if one h.265 ffmpeg encode uses 80%, adding a second one should increase the CPU utilization to 100%, or at least increase the CPU utilization a bit. But that's not happening.

Could it be some problem with scheduling?

Offline

#6 2020-05-21 14:29:52

sabroad
Member
Registered: 2015-05-24
Posts: 139

Re: Monitor memory performance

LukeLR wrote:

So if I only see ~80%, doesn't that mean I have some kind of bottleneck somewhere?

Let's consider a simplified 1 physical, 2 logical core (HT) system:

Suppose reasonably that ffmpeg has well-optimized loops that can completely saturate a processor's pipeline with vectored instructions;
Then 1 thread will always be running on logical core A (100%) and logical core B will never get chance to run (0%).
Thus overall utilisation of 50% (under aforementioned conditions).

Last edited by sabroad (2020-05-21 14:35:38)


--
saint_abroad

Offline

#7 2020-05-21 14:39:07

LukeLR
Member
Registered: 2016-03-18
Posts: 6

Re: Monitor memory performance

Yes, but in that single-threaded ffmpeg-example, wouldn't a second completely separate ffmpeg encode then use the other logical core and bring CPU utilization to 100%? Probably this wouldn't give me twice the encoding performace (in total over both processes) but it would use the CPU as much as possible, wouldn't it?

Last edited by LukeLR (2020-05-21 14:39:49)

Offline

#8 2020-05-21 14:54:56

sabroad
Member
Registered: 2015-05-24
Posts: 139

Re: Monitor memory performance

LukeLR wrote:

single-threaded ffmpeg-example

The example did not feature "single-threaded ffmpeg".

Let's rephrase to simplify:
Then logical core A will always be running (100%) and logical core B will never get chance to run (0%).


--
saint_abroad

Offline

#9 2020-05-21 14:58:12

loqs
Member
Registered: 2014-03-06
Posts: 10,644

Re: Monitor memory performance

http://www.brendangregg.com/blog/2017-0 … wrong.html covers using perf to gain an idea of if a CPU is waiting on memory.

Offline

#10 2020-05-21 15:16:19

sabroad
Member
Registered: 2015-05-24
Posts: 139

Re: Monitor memory performance

Quick test to determine if HT or something else: reboot with kernel parameter nosmt to disable HT.
I'd expect ~100% aggregate cpu utilization with HT disabled.


--
saint_abroad

Offline

#11 2020-05-21 15:18:41

Ropid
Member
Registered: 2015-03-09
Posts: 793

Re: Monitor memory performance

sabroad wrote:
LukeLR wrote:

single-threaded ffmpeg-example

The example did not feature "single-threaded ffmpeg".

Let's rephrase to simplify:
Then logical core A will always be running (100%) and logical core B will never get chance to run (0%).


In your example, you will see 100% usage on both A and B. You will not be able to see that the two logical cores are getting in each other's way by looking at the usage. You will only be able to know that they have problems sharing the real core by measuring the time to complete the tasks. The logical core will take a longer time to finish the task if they are not running alone on the real core.

Offline

Board footer

Powered by FluxBB