Appalling NVME speeds

[Sptz87]

So, I mentioned this in another thread I made but was off-topic from the original question.

 

Basically, I'm having appalling speeds with my dedicated nvme drive for downloads. I used to have both NVMEs as a BTRFS pool but decided to separate them, one for appdata another for downloads literally due to more space.

 

Now I tried ASPM (L0+L1) on my motherboard to attempt to lower energy usage and found out that it was taking 30 minutes to unpack a download. Even SABNZBD's 10GB test took over a minute. Which for my setup isn't normal at all (5800x3d,32GB@3200,Gigabyte B550i).

This is what the benchmark showed:

 

And the devices with enabled ASPM:

 

Device: 00:01.1 PCI bridge (AMD Starship/Matisse GPP Bridge)

ASPM Level: L1 Enabled

Device: 00:03.1 PCI bridge (AMD Starship/Matisse GPP Bridge)

ASPM Level: L1 Enabled

Device: 01:00.0 Non-Volatile memory controller (Micron/Crucial Technology NVMe PCIe SSD) -> Download NVME

ASPM Level: L1 Enabled

Device: 07:00.0 VGA compatible controller (NVIDIA GeForce GTX 1660 Ti)

ASPM Level: L1 Enabled

Device: 07:00.1 Audio device (NVIDIA High Definition Audio Controller)

ASPM Level: L0s L1 Enabled

Device: 07:00.2 USB controller (NVIDIA USB 3.1 Host Controller)

ASPM Level: L1 Enabled

Device: 07:00.3 Serial bus controller (NVIDIA USB Type-C UCSI Controller)

ASPM Level: L1 Enabled

 

So I decided to head into the bios and noticed that even shutting down took around 5 minutes. It used to take seconds.

Went into the BIOS and turned off ASPM and now ASPM is reported as disabled except for Nvidia's Audio Controller apparently... regardless, ASPM is disabled for the drive:

 

 

So I benchmarked again and got 800-900MB/s in folder speed. That's good news right? So I tried another 10GB test and unpacking took a little long, a few more seconds than I hoped, after that was done I decided to benchmark it again and got this:

 

 

So... yeah back to square one. It seems to tank soon after boot looks like. And I can see there's no activity on the drive... I'm a bit at a loss here. No idea where to go from here?

 

Edited February 27 by Sptz87

[wuudogg]

Have you run a trim on them lately? 

[Sptz87]

53 minutes ago, wuudogg said:

Have you run a trim on them lately? 

 

No, actually. Gonna do that, but surely that shouldn't create this type of impact right? 

[Sptz87]

2 hours ago, wuudogg said:

Have you run a trim on them lately? 

 

After trimming:

 

This is bizarre right? Pystone seems ok for this CPU but the disk speeds should be at least 2000MB/s each

Edited February 27 by Sptz87

[wuudogg]

Boom.  Working as designed.  I set a trim to run weekly or monthly on my heavy hitter NVMEs.  Set to run after mover has had time time to execute.  Use the Scheduler app. My download nvme take the most abuse and files aren't actually deleted from the storage cells when mover runs.   They are only marked for deletion.   Trim forces those marked cells available again. If your nvme workload is more static, you can set it to run less often.

[Sptz87]

19 minutes ago, wuudogg said:

Boom.  Working as designed.  I set a trim to run weekly or monthly on my heavy hitter NVMEs.  Set to run after mover has had time time to execute.  Use the Scheduler app. My download nvme take the most abuse and files aren't actually deleted from the storage cells when mover runs.   They are only marked for deletion.   Trim forces those marked cells available again. If your nvme workload is more static, you can set it to run less often.

 

More or less, did you read the post? It's still pretty slow right? This drive should be writing at around 3000-4000 MB/s... 

[wuudogg]

Sab wrench tests are meh. You arent going to get lab theoretical performance using sab wrench test behind an unraid docker.   Try directly copying some files between your nvme drives using the unraid gui and observe performance.  Don't use SMB as it adds overhead.