sota

Um... I know it's only been running for about 30 minutes, but if the stats I just pulled for F1 are any indications of what to go by, holy hades!

So the Big Box plotter is now on the ghetto farm. 16 300GB SAS DP 15k drives, set up as individual disks. 2TB SATA WD Scorpio Blue as boot and harvest storage. 1 plot started every 10 minutes. post-started all plots delay of 50,400. (to be edited as time goes on) mover checks for new plots in harvest, and moves them to the farm. got the main script for all the other plotters dialed in I think. tuned bucket, block, thread counts, and temp storage for each. And I lost one of my smaller plotters. looks like a memory chip in it went tits up, or so it seems. still testing to see if I can bring it back online.

I'm sure the powers that be will chime in, but given the intended use of unRAID, the drive count limitation made sense. Chia is a New Thing(tm) that I'd be surprised if the unRAID dev team even saw coming, so the need for so many drives in a non-corporate environment is unexpected. I'd be willing to bet their technical background issues with having so many drives, and potentially wanting to have parity as well. Now, if unRAID wanted to offer the ability to go (virtually) unlimited drives with NO parity options, that could be interesting. Even if they put it out as a total Alpha build (it breaks, don't come crying to us), that could be viable for Chia and other future space-time coin projects. Just the thoughts of a filthy casual.

watching this one machine, waiting for it to end so I can make some changes to it, I can see during phase 4, processor and hdd spikes are in opposition to each other.

if you cut the buckets in half to 64, I saw various process lines saying the u-sort needed 6.5GB ram to work, and since it didn't have it was dropping to q-sort. I'm setting my bucket size to 6656 for the next round.

That's part of what i'm planning on doing next. reformat all plotter drives to 32k sectors. (worth potentially a couple % off total time) change to 64 buckets (could be worth another 8%?) 6k ram per plot (no idea what that improved, % wise.) stagger start plots in groups of 4 (undecided on time delay at this point though.) Think I found a disk I can use for an intermediate: 2TB WD Scorpio Blue 2.5", since i've figured out how to use the CD-ROM header in the server as standard HDD location. And as for how buckets and blocks (memory) are related, this is just from some recent observations and deductions; take them with a grain of salt: buckets are into how many pieces each phase dataset is broken into. you ideally want your blocks (memory) to be big enough to complete the sort function entirely in memory (uniform sort). if it's not, you wind up dropping to the slower (?) quick sort method. so, if you have more memory, you can have fewer buckets, and it should be able to more quickly process the entire dataset. i'm attempting to tune the buckets and blocks for each machine and type that I have here.

That's part of what i'm planning on doing next. reformat all plotter drives to 32k sectors. (worth potentially a couple % off total time) change to 64 buckets (could be worth another 8%?) 6k ram per plot (no idea what that improved, % wise.) stagger start plots in groups of 4 (undecided on time delay at this point though.) Think I found a disk I can use for an intermediate: 2TB WD Scorpio Blue 2.5", since i've figured out how to use the CD-ROM header in the server as standard HDD location.

getting some interesting preliminary results from this latest run (16p1t). phase 2 DEFINITELY suffers on only 1 thread. total estimated time to complete is up about 45%, however the average plot time is DOWN 16%. I also need figure out how to have a final plot staging drive on this machine, as having to ship the plot over the network is going to have a statistically significant impact in overall plot speed, since I can't start a new plot until completed one has left the drive, since they're only 300GB disks. or... can I? I'd run that test now, but i'm making changes to the plot parameters, as well as I want to reformat the disks to a different cluster size. oh, and P410 controller is slower than a P420.

well, up to 9 machines now, plotting. 8 of 'em are single-plot-a-time machines.

i'm looking at either 300GB SAS 15k DP w/ sleds for $23ea, or 600GB SAS 15k DP w/o sleds for $45ea. debating which to get, as the 300GB are good for now, but these boxes are also my backup hardware for my hosting server, as well as unofficial backup hardware for a particular client (unofficial, as in they haven't bought it from me, and I haven't actually asked if they want to, but I've used it to disaster recovery testing of their systems.) 600GB would be a better choice for future/other use cases. I just need to sort out what I can get sleds for. going to see if I can get the drives for $40ea, and the sleds for $5ea. hoping since I'll by buying 32 of them, they'll go for it.

I'm figuring if pools becomes a viable thing, I'll at least have all the hardware ready to go.

Yea this chia app is definitely processor/thread bound. I'm attempting to spin up chia6 and chia7 now (machines) to do single plot processes, continuously. basically I'm pulling every old/spare/to-be-disposed-of machine out of the shed, seeing if it's viable, dropping a win10 install on it, and running plots.

I would still love to have enough memory in a box to do a ram disk. do they even have software for ram disks any more?

r/chia is annoying me. can't post anything new there. automatically gets removed.

Nice. My wife learned long ago, do NOT mess with my stash. I've got some... esoteric... computer bits in the attic and shed; things that I'll probably NEVER need again in my life (5.25" FDD, every tape drive imaginable, Iomega ZIP drives, MFM/RLL controllers, every flavor of SCSI controller, etc.), but i've had happen in the past, a new client call and ask if I can fix something ancient, and having those parts means I can, AND charge a princely sum for it. looks like i'm going to have to order at least one thing... 16 port gigabit switch, as I seem to not have a spare one around here. I'll cobble together a couple 5 port units I have here for now to get things working.

I get yelled at by the wife, for hanging on to certain things, like left over lumber and bits. Now who's laughing.

I have a pile of 2x4s, plywood, and various pieces/remnants of old projects, outside under cover. I just stole from that as I usually do.

Testing the ghetto rack

I'm only using the USB connections as a stop-gap, while I figure out exactly what I'm doing.

I actually have a problem: I've been doing all this testing of plot performance, and had those 4 old desktops plodding away at plotting, that suddenly I've nearly filled a 6TB drive! i have a big stash of 4TB disks here, but they're all seagates, and they won't actually work with the WD SATA-USB cards I have here (shucked from WD mybook/element cases.) #FirstWorldProblems

got the "new" farm computer running and synced to mainnet. haven't moved the disks to it yet, as I want to let it the wallet get fully synced as well. plus i'm going to build a crazy ghetto mobile rack to put everything on and in the garage. **** is noisy in my basement.

i've also seen some noise about, for machines that don't have a TON of cores or stupid fast SSDs, instead of throwing MORE threads at a plot, throw LESS, down to a single. it's only the single phase that seems to benefit from having multiple cores, and since throwing cores at it doesn't scale with a whole multiplier in terms of time reduction on that phase, it might make sense.

had another thought. what if the slow down on 8 plots, isn't the disks, but the controller getting saturated. 8p4t were all done on the same controller. it's an HP P420 w/ 1GB cache, but what if the traffic was enough to saturate it somehow. I don't think anyone knows what style I/O Chia is making; small or big chunks basically. If it's a metric ton of little chunks, that might flood the controller with enough IOPS to cause it to slow down. Now that I have the MD1000 working (60TB raw storage, just waiting to get filled), and the 8000 elite SFF is a viable choice as a Farm Only box, once I move the Farm to it I'll experiment more with splitting plot groups across controllers. 8 300GB disks on each controller, do a pair of 4p4t-2 runs, with the -2 targets being on the opposite controllers.

(earlier) well I lied. decided to kick off a 4p4t (no -2). want a baseline of what that does solo, to be able to compare to staggered pair of 4p4t. and I just finished testing and populating the MD1000. works perfectly with an LSI SAS9200-8e controller. now can see 60TB of raw storage through that. Now i need to finalize what machine is going to manage it, and become the Farm. Thinking this old HP 8000 SFF if I can make it work again.