I replaced 15 1TB drives with 5 new-ish larger capacity drives (all precleared successfully) and the parity sync is going at most 20 MBps., but averaging 16 or so.  I've attached diags, I don't see anything glaringly obvious but was hoping someone else might.

 

Thanks!

catan-diagnostics-20221211-2336.zip

There's something reading from at least disks 6 and 12.

3 hours ago, JorgeB said:

There's something reading from at least disks 6 and 12.

Something more than the parity sync reads?  The read counts all look around the same for the array drives in the GUI.

Your system share has files on disk1 and you have it set to be moved to the array. Ideally, appdata, domains, system would have no files on the array. Docker/VM performance will be impacted by parity, and array drives can't spin down since these files are always open. Your appdata and domains are setup correctly, but not system. The system share is the one share that really should not be on the array. It isn't large, and its files are really always open whenever Docker or VM Manager is enabled in Settings regardless if any Dockers or VMs are running.

43 minutes ago, shwa87 said:

Something more than the parity sync reads?

Yes, enable read/write speeds top right, counter is mostly useless.

49 minutes ago, trurl said:

Your system share has files on disk1 and you have it set to be moved to the array. Ideally, appdata, domains, system would have no files on the array. Docker/VM performance will be impacted by parity, and array drives can't spin down since these files are always open. Your appdata and domains are setup correctly, but not system. The system share is the one share that really should not be on the array. It isn't large, and its files are really always open whenever Docker or VM Manager is enabled in Settings regardless if any Dockers or VMs are running.

Thanks for pointing that out - I'm not sure how that happened.  system is now ONLY cache, appdata is ONLY cache, and domains is prefer cache.  All data matches that now (after moving the system share to only be on cache)

 

27 minutes ago, JorgeB said:

Yes, enable read/write speeds top right, counter is mostly useless.

 

Speeds are also the same, averaging around 18-20.  No disk is more or less than the other, and when I stop the sync (to restart it after fixing my shares per trurl's post) it drops to 0 on all disks, upon starting the sync again, the speeds average the same.

Post new diags please.

16 minutes ago, JorgeB said:

Post new diags please.

Attached. Thank you!

catan-diagnostics-20221212-1025.zip

Disk stats are missing from the diags, no idea why, reboot and try again, diags after the parity check/sync starts

53 minutes ago, JorgeB said:

Disk stats are missing from the diags, no idea why, reboot and try again, diags after the parity check/sync starts

Here's another try after rebooting, sync is started - vms are not. some dockers are, but none that are heavy data access that I can see.

catan-diagnostics-20221212-1156.zip

Nothing obvious that I can see, CPU is not very fast and parity sync is single threaded, but still I think it should be faster than that, did you also have dual parity with the same number of disks before?

 

Try running the diskspeed docker to rule out any disk speed issues, also run the controller bottleneck test.

13 minutes ago, JorgeB said:

Nothing obvious that I can see, CPU is not very fast and parity sync is single threaded, but still I think it should be faster than that, did you also have dual parity with the same number of disks before?

 

Try running the diskspeed docker to rule out any disk speed issues, also run the controller bottleneck test.

My parity operations have always been on the lower end, but from what I remember were at least double what I'm getting now, if not more.  Yes I had dual parity before, and MORE total disks than I have now.

 

The disk speed docker spits out code errors when I try to benchmark my drives:

Message	variable [ISSSD] doesn't exist
Stacktrace	The Error Occurred in
/var/www/Benchmark.cfm: line 482 
480: </CFIF>
481: </CFIF>
482: <CFIF IsSSD>
483: <CFIF HW[Key].Ports[PortNo].Attrib.Size.Bytes GT SSDMaxBytes>
484: <CFSET SSDMaxBytes=HW[Key].Ports[PortNo].Attrib.Size.Bytes>

Java Stacktrace	lucee.runtime.exp.ExpressionException: variable [ISSSD] doesn't exist
  at lucee.runtime.type.scope.UndefinedImpl.get(UndefinedImpl.java:242)
  at lucee.runtime.type.scope.UndefinedImpl.get(UndefinedImpl.java:184)
  at benchmark_cfm$cf.call(/Benchmark.cfm:482)
  at lucee.runtime.PageContextImpl._doInclude(PageContextImpl.java:1056)
  at lucee.runtime.PageContextImpl._doInclude(PageContextImpl.java:948)
  at lucee.runtime.listener.ClassicAppListener._onRequest(ClassicAppListener.java:65)
  at lucee.runtime.listener.MixedAppListener.onRequest(MixedAppListener.java:45)
  at lucee.runtime.PageContextImpl.execute(PageContextImpl.java:2493)
  at lucee.runtime.PageContextImpl._execute(PageContextImpl.java:2478)
  at lucee.runtime.PageContextImpl.executeCFML(PageContextImpl.java:2449)
  at lucee.runtime.engine.Request.exe(Request.java:45)
  at lucee.runtime.engine.CFMLEngineImpl._service(CFMLEngineImpl.java:1216)
  at lucee.runtime.engine.CFMLEngineImpl.serviceCFML(CFMLEngineImpl.java:1162)
  at lucee.loader.engine.CFMLEngineWrapper.serviceCFML(CFMLEngineWrapper.java:97)
  at lucee.loader.servlet.CFMLServlet.service(CFMLServlet.java:51)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
  at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
  at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197)
  at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
  at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
  at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)
  at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
  at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:687)
  at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:769)
  at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
  at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:360)
  at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399)
  at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
  at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:890)
  at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1789)
  at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
  at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
  at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
  at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
  at java.base/java.lang.Thread.run(Thread.java:829)
 
Timestamp	12/12/22 12:40:32 PM CST

 

12 minutes ago, shwa87 said:

The disk speed docker spits out code errors when I try to benchmark my drives:

Suggest posting that in the existing docker support thread.

35 minutes ago, JorgeB said:

Suggest posting that in the existing docker support thread.

Done. Thanks.  Guess I'll wait and see at this point.