[Support] Machinaris - Chia cryptocurrency farming + Plotman plotting + Unraid WebUI


Recommended Posts

47 minutes ago, Helmonder said:

The files are still in the temp folder and the screen still shows the old plot:

 

It also took me some time to get MadMax plotting to work. I also think there's an issue that it's not clearing temp folders up.

I've got two ssds that I'm using and have one of them as the primary temp, and one as tmp2. They are slowly filling up until plotting stops. 

 

After it's all ground to a halt - see plotting and plotting2 :

root@Tower:/# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/nvme0n1p1  932G  673G  259G  73% /
tmpfs            64M     0   64M   0% /dev
tmpfs            32G     0   32G   0% /sys/fs/cgroup
shm              64M   60K   64M   1% /dev/shm
shfs            3.7T  893G  2.8T  24% /plots
/dev/sde1       447G  313G  135G  70% /plotting
/dev/nvme0n1p1  932G  673G  259G  73% /id_rsa
/dev/sdf1       447G  447G   24K 100% /plotting2
/dev/sdb1       7.3T  7.2T  153G  98% /plots2
/dev/sdc1       7.3T  7.2T  153G  98% /plots3
tmpfs            32G     0   32G   0% /proc/acpi
tmpfs            32G     0   32G   0% /sys/firmware

 

I'm having to stop plotting, clear the files, and start it off again. Don't know if this is a MadMax or Machinaris issue. @guy.davis ??

This is still faster than the chia plotter, I don't have much diskspace left...

 

Link to comment
2 hours ago, Helmonder said:

Think I found why madmax was not working... Will try again -after- I have read the instructions 🙂

(EDIT: In the meantime MADMAX is now plotting)

 

Great, please see the Wiki page for details on plotting with Madmax.

 

2 hours ago, Helmonder said:

Still think something is going on with the plotting screen though...

After starting chia plotting again I now:

 

When you see an incorrect display in the Plotting page, what does "plotman status" indicate when run at the Docker Console for Machinaris?  Does the Plotting page correct itself within 1 minute or so?

Link to comment
 
When you see an incorrect display in the Plotting page, what does "plotman status" indicate when run at the Docker Console for Machinaris?  Does the Plotting page correct itself within 1 minute or so?

Hi,

The status doea not update itself and even services stopping and starting the docker..

I will try the docker console command as soon as the issue arises again !


Verzonden vanaf mijn iPhone met Tapatalk
  • Like 1
Link to comment
1 hour ago, jsebright said:

I've got two ssds that I'm using and have one of them as the primary temp, and one as tmp2. They are slowly filling up until plotting stops. I'm having to stop plotting, clear the files, and start it off again. Don't know if this is a MadMax or Machinaris issue.

 

Glad to hear you got Madmax plotting.  Please verify your Plotting config against the sample Madmax config on the wiki.   The SSDs should only fill up as you show if:

  1. Madmax runs are crashing, failing to create a final  plot, and then Plotman is starting a new Madmax run.  In this case, I would recommend dialing down your settings to match the sample, and slowly increase only after successful runs.
  2. Plotman is configured to run multiple Madmax jobs (global_max_jobs > 1 and  tmpdir_max_jobs > 1) so Plotman is running multiple at the same time, which consumes your temp disk space.

Hope this helps!

  • Like 1
Link to comment

Thanks for your reply. Seems like the runs are crashing sometimes. The tempdirs have files from different runs leftover in them.

I've just checked my config against the wiki  sample (was doing that as I saw your previous post).

I've got max jobs set to 1, but had the staggers set differently - might have been causing the issue.

I've got the threads set to 4 - I've only got 5 pairs of cores available  (Ryzen 2600 with a pair pinned in case a VM wants a look-in) and it does tend to run them at about 90%. Perhaps this needs taking down from the default? - I'll see how it goes with the default stagger options before reducing it.

 

Thanks again for everything your doing with this.

 

Link to comment
8 hours ago, jsebright said:

 

It also took me some time to get MadMax plotting to work. I also think there's an issue that it's not clearing temp folders up.

I've got two ssds that I'm using and have one of them as the primary temp, and one as tmp2. They are slowly filling up until plotting stops. 

 

After it's all ground to a halt - see plotting and plotting2 :



root@Tower:/# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/nvme0n1p1  932G  673G  259G  73% /
tmpfs            64M     0   64M   0% /dev
tmpfs            32G     0   32G   0% /sys/fs/cgroup
shm              64M   60K   64M   1% /dev/shm
shfs            3.7T  893G  2.8T  24% /plots
/dev/sde1       447G  313G  135G  70% /plotting
/dev/nvme0n1p1  932G  673G  259G  73% /id_rsa
/dev/sdf1       447G  447G   24K 100% /plotting2
/dev/sdb1       7.3T  7.2T  153G  98% /plots2
/dev/sdc1       7.3T  7.2T  153G  98% /plots3
tmpfs            32G     0   32G   0% /proc/acpi
tmpfs            32G     0   32G   0% /sys/firmware

 

I'm having to stop plotting, clear the files, and start it off again. Don't know if this is a MadMax or Machinaris issue. @guy.davis ??

This is still faster than the chia plotter, I don't have much diskspace left...

 

 

what i do to monitor temp drive space is open several terminal windows and run the following command in each one

 

1st terminal - watch -n10 du -sh /mnt/plotting  <--- put your directory here for tmp 1

2nd terminal - watch -n10 du -sh /mnt/ram  <--- put your directory here for tmp 2

 

they should both empty out after each job

hope this helps you trouble shoot

 

EDIT : also you can use an old ipad/iphone or any tablet/phone, that you may have lying and use that to monitor your plotting screen. like i use an iphone 5 and iphone 5c to monitor drive space, and iphone 6 to monitor Machinaris GUI.

use safari to login to unraid, open terminal window and type in your commands

ALSO make sure to turn OFF sleep mode on phones/tablet

 

 

 

Edited by Spazhead
  • Like 1
Link to comment
8 hours ago, Spazhead said:

 

what i do to monitor temp drive space is open several terminal windows and run the following command in each one

 

1st terminal - watch -n10 du -sh /mnt/plotting  <--- put your directory here for tmp 1

2nd terminal - watch -n10 du -sh /mnt/ram  <--- put your directory here for tmp 2

 

Nice. Haven't seen that before. I couldn't seem to open two windows of the unraid docker console, but I did find I could just use the command   watch -n10 du -sh /plotting /plotting2       and it watches both folders in one window.

Link to comment
14 minutes ago, jsebright said:

 

Nice. Haven't seen that before. I couldn't seem to open two windows of the unraid docker console, but I did find I could just use the command   watch -n10 du -sh /plotting /plotting2       and it watches both folders in one window.

just use regular unraid terminal and use actual directories in unraid, not the forwarded ones inside docker

 

 

Edited by Spazhead
  • Like 1
Link to comment
21 minutes ago, Spazhead said:

ust use regular unraid terminal and use actual directories in unraid, not the forwarded ones inside docker

Ah yes. Why didn't I think of that? Has the advantage that it doesn't get killed if the docker restarts. Watching a few together in just one window is pretty good though, and a bit less to keep track of.

Link to comment

Hi @guy.davis, I just updated to the latest test-build, unfortunately the webserver didn't respond.

 

Internal Server Error
The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

 

Docker log shows this (don't know why the first characters are cut in the web)

 

ing Chia...
ey at path: /root/.chia/mnemonic.txt
ot directory "/plots".
ot directory "/plots02".
ot directory "/plots03".
ot directory "/plots04".
ot directory "/plots05".
ot directory "/plots06".
ot started yet
daemon
vester: started
mer: started
l_node: started
let: started
ing Plotman...
k (most recent call last):
achinaris/scripts/plotman_migrate.py", line 67, in <module>
yaml.load(pathlib.Path(PLOTMAN_CONFIG))
hia-blockchain/venv/lib/python3.9/site-packages/ruamel/yaml/main.py", line 431, in load
elf.load(fp)
hia-blockchain/venv/lib/python3.9/site-packages/ruamel/yaml/main.py", line 434, in load
onstructor.get_single_data()
hia-blockchain/venv/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 122, in get_single_data
elf.construct_document(node)
hia-blockchain/venv/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 132, in construct_document
my in generator:
hia-blockchain/venv/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1617, in construct_yaml_map
struct_mapping(node, data, deep=True)
hia-blockchain/venv/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1501, in construct_mapping
check_mapping_key(node, key_node, maptyp, key, value):
hia-blockchain/venv/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 295, in check_mapping_key
plicateKeyError(*args)
aml.constructor.DuplicateKeyError: while constructing a mapping
t/.chia/plotman/plotman.yaml", line 3, column 1
plicate key "user_interface" with value "ordereddict([('use_stty_size', True)])" (original value: "ordereddict([('use_stty_size', False)])")
t/.chia/plotman/plotman.yaml", line 19, column 1

ess this check see:
aml.readthedocs.io/en/latest/api.html#duplicate-keys

e keys will become an error in future releases, and are errors

lt when using the new API.


ing Chiadog...
Chiadog...
Machinaris API server...
Machinaris Web server...
d startup. Browse to port 8926.

 

I rolled back to :latest: 

 

Internal Server Error
Traceback (most recent call last):
File "/chia-blockchain/venv/lib/python3.8/site-packages/plotman/configuration.py", line 40, in get_validated_configs
loaded = schema.load(config_objects)
File "/chia-blockchain/venv/lib/python3.8/site-packages/marshmallow/schema.py", line 714, in load
return self._do_load(
File "/chia-blockchain/venv/lib/python3.8/site-packages/marshmallow/schema.py", line 896, in _do_load
raise exc
marshmallow.exceptions.ValidationError: {'directories': {'log': ['Missing data for required field.']}, 'logging': ['Unknown field.'], 'commands': ['Unknown field.'], 'version': ['Unknown field.']}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/chia-blockchain/venv/bin/plotman", line 8, in <module>
sys.exit(main())
File "/chia-blockchain/venv/lib/python3.8/site-packages/plotman/plotman.py", line 137, in main
cfg = configuration.get_validated_configs(config_text, config_path)
File "/chia-blockchain/venv/lib/python3.8/site-packages/plotman/configuration.py", line 42, in get_validated_configs
raise ConfigurationException(
plotman.configuration.ConfigurationException: Config file at: '/root/.chia/plotman/plotman.yaml' is malformed

 

 

Rolled back again to :test - still no difference, first internal server error is showing.

 

Your help is highly appreciated ;) Thanks!

 

Edited by localh0rst
Added more logs
Link to comment
11 minutes ago, localh0rst said:

thanks for the suggestion, unfortunately this didn't help. seems something is wrong with the plotman.yaml?

Sorry, that's something that took me a while to get right, and I couldn't see what was wrong with it. I'd suggest you try to revert to defaults and as simple as possible. Check the drive paths are mapping to where they should be.

Can't offer any more help as I don't know enough...

Link to comment
4 hours ago, localh0rst said:

Hi @guy.davis, I just updated to the latest test-build, unfortunately the webserver didn't respond.



Internal Server Error
The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

 

 

Good day, thanks for testing the test version.   Please stop the Machinaris container, delete the following files, then restart:

  • /mnt/user/appdata/machinaris/machinaris/dbs/machinaris.db
  • /mnt/user/appdata/machinaris/machinaris/dbs/stats.db
  • /mnt/user/appdata/machinaris/chiadog/dbs/chiadog.db

On startup wait a few minutes for things to populate. Afterwards, take a look at /mnt/user/appdata/machinaris/machinaris/logs/migration.log.  Thanks!

Edited by guy.davis
  • Like 1
Link to comment
1 hour ago, jsebright said:

@localh0rst

I had the internal server error. Logged into the docker and ran  "flax init"  that seemed to start it for me.  May or may not work for you...

Thanks for testing the test version.  You should not need to run init flax.  Would you mind sharing the Logs output view from Unraid for Machinaris?

Link to comment
ing Chia...
ey at path: /root/.chia/mnemonic.txt
ey at path: /root/.chia/mnemonic2.txt
ot directory "/plots".
ot directory "/plots1".
ot directory "/plots2".
ot directory "/plots3".
ot started yet
daemon
vester: started
mer: started
l_node: started
let: started
annot create directory '/root/.chia/flax': File exists
ing Flax...
ey at path: /root/.chia/mnemonic.txt
ey at path: /root/.chia/mnemonic2.txt
ot directory "/plots".
ot directory "/plots1".
ot directory "/plots2".
ot directory "/plots3".
ot started yet
daemon
vester: started
mer: started
l_node: started
let: started
ing Plotman...
ing Chiadog...
Chiadog...
ing Flaxdog...
Flaxdog...
Machinaris API server...
Machinaris Web server...
d startup. Browse to port 8926.

I might have restarted the docker since - not sure sorry.

With the usual first few characters cut off - If you know where I can get all the output I'll update it.

 - still getting plotting failures. This page https://github.com/madMAx43v3r/chia-plotter/issues/574 seems to suggest it may be RAM related. I have had some other crashes with the intense processing (otherwise the server is stable, but only lightly used), so I might do a full reboot and see what I can tweak.

Link to comment
9 hours ago, guy.davis said:

Hi all! Test version (ghcr.io/guydavis/machinaris:test) now has initial support for concurrent farming of the same plots on the Flax network.  This is early stages, with some known issues, but feedback on the implementation so far is appreciated. Details on the Flax wiki page. Thanks! 

flax_summary.png


I just tried installing on another fresh unraid server.
But flax does not come up.

I have set  blockchains; to chia, flax.

Unraid log shows this: 

not even a mention of flax.

 

Spoiler

ing Chia...
ey at path: /root/.chia/mnemonic.txt
ing Chia...
ey at path: /root/.chia/mnemonic.txt
ot directory "/plots".
ot directory "/plots".
ot started yet
daemon
ot started yet
daemon
vester: started
mer: started
l_node: started
let: started
ing Plotman...
ing Chiadog...
Chiadog...
Machinaris API server...
Machinaris Web server...
d startup. Browse to port 8926.

 

Edited by Struck
spelling
Link to comment

 

1 hour ago, Struck said:

I just tried installing on another fresh unraid server.
But flax does not come up.

I have set  blockchains; to chia, flax.

Unraid log shows this: 

not even a mention of flax.

 

Hi, thanks for testing the test version.  I have enabled Flax by default now to simplify things. Please update to the latest ghcr.io/guydavis/machinaris:test image. Thanks again for testing Flax integration in Machinaris!

 

Edited by guy.davis
Link to comment
45 minutes ago, guy.davis said:

 

 

Hi, thanks for testing the test version.  Did you enable Advanced View and edit the new Environment Variable named 'blockchains' (using Unraid Docker edit page)  and then restart the Machinaris container?  Also, did you change the repository to "ghcr.io/guydavis/machinaris:test"?  Thanks again for testing Flax integration in Machinaris!

Yes and Yes

 

Saw your edit.
I saw that the key of blockchains was set to chia, instead of blockchains

this fixed it for me atm

Edited by Struck
updated
  • Like 1
Link to comment

any chance you can integrate the other 2 forks (chaingreen, and spare) into the container as well? that would be sweet!

 

my plan at this point is, to have the container be the farm, and have it mostly talk to just a couple of harvesters i'm going to set up.  any docs/how-to's on how to tell the container about havesters?

Link to comment
49 minutes ago, sota said:

my plan at this point is, to have the container be the farm, and have it mostly talk to just a couple of harvesters i'm going to set up.  any docs/how-to's on how to tell the container about havesters?

 

Hi! The Machinaris workers (harvester, plotter, or harvester+plotter) connect to the Machinaris controller/fullnode, by hostname or by IP address. To launch a harvester, try these directions.  Let me know if you have any trouble.

Link to comment

Flax netspace is showing up as ? and I'm currently having issues connecting to the chia network. This was working prior to updating to the :test branch. Also, Flax appears to be not seeing half my plots that are on a shared network drive, but chia does?

 

image.png.9be87f3226c20668e1f204bdc6f0e866.png

 

image.thumb.png.67dbc13d4312d0302bb5a00df2f81672.png

 

 

edit: I think this may have something to do with timelord. 

 

root@InnisShallows:/chia-blockchain# chia start all
chia_harvester: Already running, use `-r` to restart
chia_timelord_launcher: started
chia_timelord: Already running, use `-r` to restart
chia_farmer: Already running, use `-r` to restart
chia_full_node: Already running, use `-r` to restart
chia_wallet: Already running, use `-r` to restart
root@InnisShallows:/chia-blockchain# chia_timelord_launcher -r
Traceback (most recent call last):
  File "/chia-blockchain/venv/bin/chia_timelord_launcher", line 11, in <module>
    load_entry_point('chia-blockchain==1.1.7.dev79+gaba34c1c.d20210629', 'console_scripts', 'chia_timelord_launcher')()
  File "/chia-blockchain/venv/lib/python3.9/site-packages/chia_blockchain-1.1.7.dev79+gaba34c1c.d20210629-py3.9.egg/chia/timelord/timelord_launcher.py", line 107, in main
    loop.run_until_complete(spawn_all_processes(config, net_config))
  File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/chia-blockchain/venv/lib/python3.9/site-packages/chia_blockchain-1.1.7.dev79+gaba34c1c.d20210629-py3.9.egg/chia/timelord/timelord_launcher.py", line 85, in spawn_all_processes
    await asyncio.gather(*awaitables)
  File "/chia-blockchain/venv/lib/python3.9/site-packages/chia_blockchain-1.1.7.dev79+gaba34c1c.d20210629-py3.9.egg/chia/timelord/timelord_launcher.py", line 45, in spawn_process
    path_to_vdf_client = find_vdf_client()
  File "/chia-blockchain/venv/lib/python3.9/site-packages/chia_blockchain-1.1.7.dev79+gaba34c1c.d20210629-py3.9.egg/chia/timelord/timelord_launcher.py", line 39, in find_vdf_client
    raise FileNotFoundError("can't find vdf_client binary")
FileNotFoundError: can't find vdf_client binary

 

Edited by sworcery
Link to comment

Loving the remote workers to consolidate everything to a single interface. Wish Machinaris could be a stand alone install for all my windows remote harvesters I have running.

 

With that, I want to report my estimated time to win has the wrong info.  If I compare to Chia calculator it seems like the expected time to win is using a remote worker instance to calculate my time to win, and not combining the estimate with the primary instance. This setup has the primary +1 remote full node.

639 total plots, chia calc 4 months

166 on the remote worker, chia calc about 1 year

473 on the primary instance, chia calc 5 months

Machinaris expected time to win: 1 Year

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.