Chia farming, plotting; array and unassigned devices


Shunz

Recommended Posts

Gonna be exploring Chia farming, of which I believe interested unraid-dabbling folks are extremely primed to be exploring!

 

Several thoughts:

1) Storage (Farm) Locations: Should the Chia farm plots be stored on the unraid array, or should they be on unassigned devices?

Benefits of being on unassigned devices

- Reduce spin-up and wear on the array drives

- These farm plots aren't exactly critical data - if the drives are lost, just build those plots again

 

Edit - Specific Chia-only shares can be set to include specific disks, and exclude non-desired drives. This makes the spin-up point above moot, though I'm still undecided between chia storage on array, or on unassigned disks.

 

2) Plotting Locations: Chia plotting should be done on fast SSDs with high endurance. What about plotting on unraid BTRFS pools? E.g. a 2nd speedier non-redundant cache pool.

 

3) I'll probably plot on my Desktop PC, and store Farm Plots on unassigned devices. I have 2 high endurance SM863/SM963A SSDs as my cache pool, so, I hope to start farming on the unraid system as well. Waiting for a proper docker for unraid...!

Edited by Shunz
  • Like 1
Link to comment

1) I use my array as if I lose my plots I'd cry its taken so long to make them.

But you can use anything really...

I had spare drives in my array, so I just set them to spin forever.

 

2) I dont do this, I got 2 spare Intel SSD's that I use for plotting via unassigned devices

I did do 1 or 2 plots via my Cache pool as a test. Just make sure you link it via the container otherwise you'll plot to your docker.img file and it'll blow.

 

3) I use both unRAID and my gaming PC and just have my gaming PC transfer the plots to my array via a share. 

 

Hope this helps.

  • Like 1
Link to comment
1 hour ago, SavellM said:

1) I use my array as if I lose my plots I'd cry its taken so long to make them.

But you can use anything really...

I had spare drives in my array, so I just set them to spin forever.

 

2) I dont do this, I got 2 spare Intel SSD's that I use for plotting via unassigned devices

I did do 1 or 2 plots via my Cache pool as a test. Just make sure you link it via the container otherwise you'll plot to your docker.img file and it'll blow.

 

3) I use both unRAID and my gaming PC and just have my gaming PC transfer the plots to my array via a share. 

 

Hope this helps.

 

Regarding #2, Have you tried creating a 2nd cache pool for Chia purposes? I wonder if a non-redundant pool makes for faster plotting speeds (or allow for more parallel plotting)

Link to comment

Hey guys. I just found out about Chia. Is there any info you can provide about how to setup the docker/tips tricks/ and anything else I should know. I have a spare 100TB on my main server. What are your thoughts on Chia so far? I was pretty heavy into bitcoin mining a while back. I know that Chia transactions are supposed to be enabled right now. Thanks guys.

Link to comment

Seems like small players might be better to wait for pools to be available as it sounds like you will need to use new plots for pools.  Currently the network is at 2.5 exabytes and growing by a significant amount daily so small players are going to struggle to get paid at the moment....   

 

If you have 100TB of spare storage and don't mind killing a bunch of SSD's generating the plots I think you can probably turn a profit eventually... assuming the coin price doesn't fall and the network doesn't grow too much.

 

 https://chiacalculator.com/

 

Link to comment
On 5/6/2021 at 12:04 PM, Chris Pollard said:

Seems like small players might be better to wait for pools to be available as it sounds like you will need to use new plots for pools.  Currently the network is at 2.5 exabytes and growing by a significant amount daily so small players are going to struggle to get paid at the moment....   

 

If you have 100TB of spare storage and don't mind killing a bunch of SSD's generating the plots I think you can probably turn a profit eventually... assuming the coin price doesn't fall and the network doesn't grow too much.

 

 https://chiacalculator.com/

 

I agree chia pools will be really good for unraid chia mining.  

Link to comment

I currently only have a few plots on the unraid array. It keeps that disk continously spun up - which I'm not exactly excited about; I'll definitely have some drives dedicated to Chia plots, or keep chia plots off the array.

 

The other concern is whether having Chia plots on the unraid Array causes timeouts. Chia requires the plots/proofs (sorry - i haven't gotten my terms correctly yet) to be verified within SECONDS, and there has been news that NAS storage was causing verifications to timeout.

Link to comment
Posted (edited)
10 hours ago, Shunz said:

The other concern is whether having Chia plots on the unraid Array causes timeouts. Chia requires the plots/proofs (sorry - i haven't gotten my terms correctly yet) to be verified within SECONDS, and there has been news that NAS storage was causing verifications to timeout.

 

I think they recommend responding within 2 seconds so you definitely don't want drives spinning down. see :-

 

https://github.com/Chia-Network/chia-blockchain/wiki/FAQ#is-it-possible-to-have-a-proof-but-not-get-a-reward

 

Edited by Chris Pollard
Link to comment
Posted (edited)

Hi when using an external pc and not a vm don't plot to the array it will fail eventually...

but u can use a windows vm with passed through ssd and plot directly to the array for external plotting machines just copy the finished files but rename them and once u get them on the array name them back to .plot that way your harvester will recognize them.

Edited by defekt
Link to comment
On 4/30/2021 at 2:10 PM, SavellM said:

1) I use my array as if I lose my plots I'd cry its taken so long to make them.

But you can use anything really...

I had spare drives in my array, so I just set them to spin forever.

 

2) I dont do this, I got 2 spare Intel SSD's that I use for plotting via unassigned devices

I did do 1 or 2 plots via my Cache pool as a test. Just make sure you link it via the container otherwise you'll plot to your docker.img file and it'll blow.

 

3) I use both unRAID and my gaming PC and just have my gaming PC transfer the plots to my array via a share. 

 

Hope this helps.

 

I am doing the same thing. How do I get my Chia GUI to see and farm the plots created by Unraid Chia? I have added the same mnemonic key to the Unraid plotting app but the windows GUI is not counting the Unraid plots as part of my total plots.


Thanks!

Link to comment

Just a little FYI from some testing I am doing.

 

You can get some good plotting speeds with normal hard drives without the need to wear out SSD's.

 

With a regular 7200rpm 4tb single drive I was able to get an effective speed of 6.5 hours by plotting 3 in parallel.

 

By raiding 4x 4tb drives together I have gotten it down to 2.5 effective hours/plot doing 8 in parallel and trying 12 now.

 

I have a bunch of old small drives I plan on raiding together and using for plotting.

 

Just a heads up so people don't think SSD's are required to plot. That said I would still use hard drives you don't care about for plotting as it will put a lot of wear and tear on them.

 

I am doing the tests on an ubuntu machine using BTRFS filesystems. You can do the same thing with pools in unriad easier. I plan to do that once I settle on a setup.

  • Like 1
Link to comment
12 hours ago, TexasUnraid said:

Just a little FYI from some testing I am doing.

 

You can get some good plotting speeds with normal hard drives without the need to wear out SSD's.

 

With a regular 7200rpm 4tb single drive I was able to get an effective speed of 6.5 hours by plotting 3 in parallel.

 

By raiding 4x 4tb drives together I have gotten it down to 2.5 effective hours/plot doing 8 in parallel and trying 12 now.

 

I have a bunch of old small drives I plan on raiding together and using for plotting.

 

Just a heads up so people don't think SSD's are required to plot. That said I would still use hard drives you don't care about for plotting as it will put a lot of wear and tear on them.

 

I am doing the tests on an ubuntu machine using BTRFS filesystems. You can do the same thing with pools in unriad easier. I plan to do that once I settle on a setup.

Ah, so it is possible to have an amazing plotting speed (high terabytes per day) simply by using a whole bunch of cheap HDDs in parallel! (assuming a CPU with enough grunt)

Link to comment
Posted (edited)
15 hours ago, Shunz said:

Ah, so it is possible to have an amazing plotting speed (high terabytes per day) simply by using a whole bunch of cheap HDDs in parallel! (assuming a CPU with enough grunt)

 

Yep, pretty impressed myself. Just finished the 12x plots and it was a tad slower then 8x on the 4x4tb raid0. Guessing if I staggered them I could get ~10-12 plots to be a bit faster with some fine tuning.

 

As it is, just going to toss all my old drives that are basically useless to me now into a system and plot on those. Just got to make sure that you only raid together drives of similar speeds.

 

I managed to get 20x 10krpm sas 150gb drives for $15 shipped on ebay the other day, so a bunch of drives I don't care about to play with and should net some decent performance.


 

Edited by TexasUnraid
  • Like 1
Link to comment

Nice GUI docker, that is exactly what I have been looking for to start plotting on my main unraid server, might as well put it to use.

 

Saw some cheap DDR3 ECC on ebay the other day, picked up some but thought about getting enough to make a ramdrive for plotting lol. Research showed it was not as good as it sounds though since the bottleneck actually lies in the CPU for the most part.

 

Looking through my cart at some parts I have been putting off getting, if there is something you need/want for your server, get it quick. Prices are SKYROCKETING!

 

I picked up an adaptec 24 port HBA 72416 a few months ago for $80 WITH 6x breakout cables (worth ~$90 by themselves). The same card is not going for $190 without the cables! I wanted to get another just for the cables but waited too long.

Link to comment
Posted (edited)
11 hours ago, TexasUnraid said:

 

Yep, pretty impressed myself. Just finished the 12x plots and it was a tad slower then 8x on the 4x4tb raid0. Guessing if I staggered them I could get ~10-12 plots to be a bit faster with some fine tuning.

 

As it is, just going to toss all my old drives that are basically useless to me now into a system and plot on those. Just got to make sure that you only raid together drives of similar speeds.

 

I managed to get 20x 10krpm sas 150gb drives for $15 shipped on ebay the other day, so a bunch of drives I don't care about to play with and should net some decent performance.


 

Interesting.  I threw a spare/backup server I have here (HP DL380G6 dual X5570 processors, 144GB ram, 2x72GB (boot), 6x146GB SAS 15k DP (RAID0), 8x300GB SAS 15k DP (RAID0) online a couple days ago, and my plot speeds aren't nearly that good.  I got 7 plots done in about 24 hours. 3 plots on the 6x array, and 4 plots on the 8x array, all in parallel.  It's also syncing with mainnet as well, so I don't know if that's causing problems.  My performance is frankly, pathetic, given the iron thrown at it.  It has me somewhat depressed.

 

I've been seeing noises on reddit that, breaking plots up to individual drives nets better performance, so i'm considering moving all my completed plots off those drives (was going to anyways, have a stack of 4TB disks here waiting) and breaking the arrays apart, into individual disks or in the case of the 146s, RAID0 pairs to get the space needed to plot.  I should be able to do 11 plots in parallel then.

 

Oh, and the box is running Windows 10.

Edited by sota
Link to comment
Posted (edited)
30 minutes ago, sota said:

Interesting.  I threw a spare/backup server I have here (HP DL380G6 dual X5570 processors, 144GB ram, 2x72GB (boot), 6x146GB SAS 15k DP (RAID0), 8x300GB SAS 15k DP (RAID0) online a couple days ago, and my plot speeds aren't nearly that good.  I got 7 plots done in about 24 hours. 3 plots on the 6x array, and 4 plots on the 8x array, all in parallel.  It's also syncing with mainnet as well, so I don't know if that's causing problems.  My performance is frankly, pathetic, given the iron thrown at it.  It has me somewhat depressed.

 

I've been seeing noises on reddit that, breaking plots up to individual drives nets better performance, so i'm considering moving all my completed plots off those drives (was going to anyways, have a stack of 4TB disks here waiting) and breaking the arrays apart, into individual disks or in the case of the 146s, RAID0 pairs to get the space needed to plot.  I should be able to do 11 plots in parallel then.

 

Oh, and the box is running Windows 10.

 

I have seen a lot of reports that plotting is faster on linux and BTRFS is faster still then most other file systems. You might try running Ubuntu and see what the plot times do.

 

The other thing is that the first stage is the only one that can handle mutiple threads, the rest are all single threaded, so single thread performance will play a big role there. I am running 2x 2670 CPU's which have a lot higher single thread performance then you, although surprised you can't get more as I am not even stressing it at this point, just doing basic tests until pooling comes online.

 

I also increased my dirty writes in linux to 90%, so it can use up to 90% of my available memory as a cache for writes to the drives, didn't seem like this helped a ton but it might of had a larger effect then I thought. It reduced the thrashing of the drives though.

 

based on my testing so far 3x parallel for a single drive netted me the best times per plot at 6.5 hours. If I was to do that on the 4x drives I am now testing in the raid, I would be around 2.2 hours per plot, a bit faster then the 2.5 hour raid0 performance of the drives thus far. Testing now with some staggering of the plots to see if that improves things.

 

So yes, using the drives individually does seem to net slightly faster speeds all else being equal but it is a lot harder to keep track of, so there is a trade off.

 

Watch the performance metrics of the system during plotting and you can find your bottlenecks. I use netdata on linux, allows me to log the whole process for later review.

 

I don't max out the single drives IO until running 3x in parallel per drive (on these fairly fast ~170mb/s drives). With 2x plots there are times when the drive activity drops to zero.

 

CPU limits the ability to plot a single plot any faster, thats why I need multiple in parallel to max the drives IO. I have 32 threads but they are ~7 year old threads, so they leave some performance on the table lol.

 

It is looking like my limiting factor will end up being memory actually as I can only run ~17 in parallel with 64gb of ram but can do a lot more then that with my CPU's and disks before those max out. I won an auction for some cheap ECC DDR3 the other day, gonna see what 128gb of ram does ;-) with ~25-30 plots running in parallel.

 

Yes I will be over-provisining the CPU but that is not an issue if I stagger the plots as only the first stage is multi-threaded. Plus worst case it slows down a bit.

Edited by TexasUnraid
  • Like 1
Link to comment

My bigger problem right now is, I think I know why sync is horridly slow.  it has a ton of peers connected, but only one has a height value specified, and is the only one i'm getting data from.

Link to comment
22 minutes ago, sota said:

My bigger problem right now is, I think I know why sync is horridly slow.  it has a ton of peers connected, but only one has a height value specified, and is the only one i'm getting data from.

There was a new client released that moves to a dns based introducer, that should fix many sync issues.

Official docker build has hit a few roadblocks, so its not released yet, but the new windows client seems to be running really well for sync.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.