MrChunky

It seems that Unraid does not play well with some aspects of the storj v3 container. In particular the volume mapping seems to be broken, I am not sure why or how. My node failed twice in the last month. Currently I have no reliable solution on running v3 node on unraid docker. The config below is just the latest iteration, that does not seem to work reliably. I am not, the second paragraph is copy pasta, that's why it sounds like I am talking in the first person. Since UnRaid template editor does not support proper config options, I would imagine they have no way of supporting their version of plug and play template. Although, it is trivial to configure it yourself. For any one who wants to do it themselves: 1. Got to community Apps, type in storj, then click "get more results from DockerHub", click on the official storagenode template by storjlabs. The repository should be storjlabs/storagenode:alpha 2. Add the following values manually by clicking "Add another Path, Port, Variable, Label or Device": Port: Host Port: 28967, Connection Type: TCP Port: Host Port: 28967, Connection Type: UDP Variable: Key: WALLET, Value: YOUR_WALLET_ADDRESS Variable: Key: EMAIL, Value: YOUR_EMAIL_ADDRESS Variable: Key: ADDRESS, Value: YOUR_EXTERNAL_IP_ADDRESS:28967 Variable: Key: BANDWIDTH, Value: YOUR_BANDWIDTH (per month) Variable: Key: STORAGE, Value: YOUR_STORAGE_VOLUME 3. Map the storage locations by enabling advanced view In the extra parameters field enter: --mount type=bind,source="/mnt/user/appdata/storj_cert/storj/identity/storagenode/",destination=/app/identity --mount type=bind,source="/mnt/user/storjv3/",destination=/app/config Replace the locations in quotation marks with the locations for your configuration, first one for your identity token location and second for your storage location. 4. Launch docker by clicking Apply

Hey @Jcloud , I wanted to mention that the storj node v3 template you have put up is misconfigured. The storj and identity locations should be mounted as --mount rather than -v. This can be achieved in UnRaid through extra parameters only afaik. The difference is very dangerous: if docker will not map the disk to the container for any reason, with the -v option it will create an empty disk inside the container. The customers' data will be stored inside this container. When you then remove the container, the data will be removed with it. In case of --mount it will just fail. Please, replace the -v options to the --mount options ASAP. The --mount option has a different syntax. Please, read about it in the section Running the Storage Node of our Documentation.

That would be lovely!

I didnt find a good guide for integrating Collabora into Nextcloud on unraid... So I thought I would share my config. I am using Nextcloud v13. Prerequisites: Letsencrypt nginx docker Nextcloud docker on a subdomain configure in nginx (e.g. could.yourdomain.com). Instalation: Configure Letsencrypt/nginx Add new subdomain to your letsencrypt configuration (e.g. office.) by editing the lets encrypt docker parameters. Edit the nginx default file (usually: user/mnt/appdata/letsencrypt/nginx/site-confs/default) by adding a new subdomain block for collabora. server { listen 443 ssl; server_name office.yourdomain.com; ssl_certificate /config/keys/letsencrypt/fullchain.pem; ssl_certificate_key /config/keys/letsencrypt/privkey.pem; ssl_dhparam /config/nginx/dhparams.pem; ssl_ciphers 'XXX'; ssl_prefer_server_ciphers on; # static files location ^~ /loleaflet { include /config/nginx/proxy.conf; proxy_pass https://xxx.xxx.xx.xx:9980; } # WOPI discovery URL location ^~ /hosting/discovery { include /config/nginx/proxy.conf; proxy_pass https://xxx.xxx.xx.xx:9980; } # main websocket location ~ ^/lool/(.*)/ws$ { include /config/nginx/proxy.conf; proxy_pass https://xxx.xxx.xx.xx:9980; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "Upgrade"; proxy_set_header Host $http_host; } # download, presentation and image upload location ~ ^/lool { include /config/nginx/proxy.conf; proxy_pass https://xxx.xxx.xx.xx:9980; } # Admin Console websocket location ^~ /lool/adminws { include /config/nginx/proxy.conf; proxy_pass https://xxx.xxx.xx.xx:9980; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "Upgrade"; proxy_set_header Host $http_host; } } Install the Collabora docker (I'm using chvb's Repository docker here, collabora/code) and fill out the docker variables appropriately. In the Domain field the subdomain of your nextcloud has to be entered in the following format: cloud\\.yourdomain\\.com Now you can download the collabora app through the nextcloud UI and put in the collabora address in the setting menu for the collabora app. (e.g. https://office.yourdomain.com) Restart letsencrypt, collabora and nextcloud. Now if you click on word document in nextcloud it should open in browser with collabora. My nextcloud nginx config for reference: server { listen 443 ssl; root /config/www ; index index.html index.htm index.php; server_name cloud.XXX.com; ssl_certificate /config/keys/letsencrypt/fullchain.pem; ssl_certificate_key /config/keys/letsencrypt/privkey.pem; ssl_dhparam /config/nginx/dhparams.pem; ssl_ciphers 'XXX'; ssl_prefer_server_ciphers on; ssl_session_cache shared:SSL:10m; add_header Front-End-Https on; location / { include /config/nginx/proxy.conf; proxy_pass https://xxx.xxx.xx.xx:444/; } }

Possible solution

Hey nuhll, I think the issue was the docker config from Jcloud. If you are using the non monitor version of storj, these dockers were always behaving erratically for me. The monitor version which was the one he created later works as expected for me.

SIA is not playing nice with UnRaid system for me. I have my sia share set to use the cache drives. Firstly the mover is not able to move anything because all the share files are constantly open by sia. Secondly when you shut sia down and attempt to move, sia writes the sparse preallocated file as a full file. e.g. you have 8TB share defined, instead of moving the size that is stored, unraid writes a 8TB file, which is obviously impossible if you do not have 8TB free on a single disk (correct me if I am wrong here). I would recommend either using a share that does not use cache and splitting up you share folders into many small chunks by creating multiple share folders. Or using unassigned devices and external drives.

Stroj is working as expected with multiple nodes and reporting to StorjStat for me now:) So good job all around @Jcloud. I am looking into SIA now and I got the docker installed, ports forwarded. When I open terminal within unraid gui for that docker (new feature in unraid), I tried a few commands and I just cannot find how to interact with it. Tried host commands and siad comands, nothing found. how do you do it? The GUI shows syncing activity but the daemon says unreachable from terminal... I am still syncing so that might be the issue? Figured it out: Have to use ./siac to access terminal commands from within UnRaid docker terminal. e.g.: ./siac -h ./siac folder add /sia-data/xxx 8TB Or from SIA UI terminal: ./siac --addr xxx.xxx.xxx.xxx:9980 ./siac --addr xxx.xxx.xxx.xxx:9980 host folder add /sia-data/xxx 8TB Also don't forget to port forward both TCP and UDP... Took me a while to figure that out. There is a small mistake in your instructions: False and True should be the other way around

What do you mean by restore the GUI? Is that a specific action? Thanks for the quick reply

Got myself a new weird problem. Apps tab is missing. When I try to directly access IP:Port/Apps I just get a blank page. In Plugins section the community applications plugin is looking strange: I am running Unraid v 6.5.1-rc6. Nothing of note in Syslog... some PHP warnings... Any help would be much appreciated.

I am running both repos, and its the same situation. I presume the network was being tested, and now its not. This is why the alocs are basically 0, not because of the repo. If you read the subreddit, everyone is in the same boat.

Sorry for the spam. I was thinking that it would make sense (to me) to not start the main node that resides in the root folder and only start additional nodes. It would make the folder structure make more sense, as all nodes will be in respective sub folders. Also potentially avoid complications with debugging several conditions (main/ additional nodes).

Yep, sorted it out. Unassigned devices runs under nobody on the /storj folder. Changed perms to root and its all good now.

In my case, when I set it to False, I get UPnP connection. When it is set to true I get TCP. All my nodes are set to do not traverse and working fine. I have a floating rule for the ports on PfSence since I wanna circumvent pfblockerng geo blocking.

It is container with a single node that existed previously on ht old template. But I am getting the same error on the completely new nodes I also just restarted an old mode in host mode, to check that it is not the problem and I have the same error. The only other thing I changed in the config is: Do Not Traverse NAT to True. * starting node with config at /storj/config.json {"level":"info","message":"got status query","timestamp":"2018-04-21T07:21:34.665Z"} [07:21:34] 94fc2d886f6c664a20f55f8c357c189775f79a46 | Submit to Storjstat [07:21:34] ERROR {"error":"Invalid farmer data (spaceUsedBytes)"}

Played around with a new docker. Thanks a bunch for getting this done. 1. Firstly, I think that bridge still works better. If I try to run multiples of the new docker (e.g. I want to share unassigned devices and main storage) the second docker tries to launch with the same IDs as the first in Host mode. Not a problem in Bridge mode. 2. Secondly the storagemonitor is not working for me. I get: * starting daemon in foreground {"level":"info","message":"attempting to start node with config at path /storj/config.json","timestamp":"2018-04-20T07:05:12.296Z"} * starting node with config at /storj/config.json {"level":"info","message":"got status query","timestamp":"2018-04-20T07:05:12.498Z"} [07:05:12] 94fc2d886f6c664a20f55f8c357c189775f79a46 | Submit to Storjstat [07:05:12] ERROR {"error":"Invalid farmer data (spaceUsedBytes)"} {"level":"info","message":"got status query","timestamp":"2018-04-20T07:16:41.768Z"} {"level":"info","message":"got status query","timestamp":"2018-04-20T07:16:56.028Z"} {"level":"info","message":"got status query","timestamp":"2018-04-20T07:20:12.528Z"} [07:20:12] 94fc2d886f6c664a20f55f8c357c189775f79a46 | Submit to Storjstat [07:20:12] 94fc2d886f6c664a20f55f8c357c189775f79a46 | Error {"level":"info","message":"got status query","timestamp":"2018-04-20T07:20:41.421Z"} {"level":"info","message":"got status query","timestamp":"2018-04-20T07:35:12.628Z"} [07:35:12] 94fc2d886f6c664a20f55f8c357c189775f79a46 | Submit to Storjstat [07:35:12] 94fc2d886f6c664a20f55f8c357c189775f79a46 | Error ... This node does however have stored data: ┌─────────────────────────────────────────────┬─────────┬──────────┬──────────┬─────────┬───────────────┬─────────┬──────────┬───────────┬──────────────┐ │ Node │ Status │ Uptime │ Restarts │ Peers │ Allocs │ Delta │ Port │ Shared │ Bridges │ ├─────────────────────────────────────────────┼─────────┼──────────┼──────────┼─────────┼───────────────┼─────────┼──────────┼───────────┼──────────────┤ │ 94fc2d886f6c664a20f55f8c357c189775f79a46 │ running │ 3h 40m … │ 0 │ 162 │ 42 │ 2ms │ 4000 │ 13.5GB │ connected │ │ → /storj/share │ │ │ │ │ 4 received │ │ (TCP) │ (0%) │ │ └─────────────────────────────────────────────┴─────────┴──────────┴──────────┴─────────┴───────────────┴─────────┴──────────┴───────────┴──────────────┘ All the numbers are blue in my storjstat website. Not sure what other details I can provide to diagnose this... 3. Another thing is that I tried to migrate multiple nodes to the new docker. The first node (that is in the root folder) gets the stored data identified properly in the terminal with "storjshare status" command. The extra nodes do not get existing data shown properly, although they connect fine. This could be peculiarity of my setup as I ran the nodes of unassigned devices in R/W Salve mode...

Looking good, I got the nodes up and running from the latest docker! How do you check that the storjmonitor is reporting properly? Everything is up, but I don't see any changes on StorjStat.com... Probably need to wait for someone to actually put things on the share

I would really appreciate that too. If you are in too deep, you could possibly ask someone on the forums like aptalca, who has a lot of experience with dockers to look over it.

Wouldn't this docker implementation completely circumvent the limits storj imposed on the number of nodes being limited by the number of cores on one machine?

I have been looking at my firewall and it looks like the tunnel ports are indeed unused. Keeping the whole range open for now.

What you define in the json file only affects the docker internally. What you are showing in the image above are the ports you have forwarded to the docker when you created it. To me it looks like you need to change the ports in the template configuration to what you want e.g. 4000-4003, 4004-4007 etc. (step 4 in what I posted above). If this is not fixing your problem, I would guess that there is some issue on your side unrelated to Storj...

@L0rdRaiden That's because your containers are in host/br0 modes. They need to be in bridge modes. the container port field is only relevant in bridge mode.

@L0rdRaiden You need to first enable the advanced view and switch the container networking from host to bridge. You should then be able to see all the options you need. The screen shot of my view is in the post above.

Figured out fool proof way to add additional nodes using multiple containers. I was not able to find the Storj container using community apps plugin, so here is how I did it: 1. Add repository (address field with red arrow) and click "Add Container" 2. You will then see the container/docker configuration screen. Here you need to delete the line with address (red arrow) and switch to advanced view in the top right corner. 3. Then switch from Host to Bridge 4. Add storage location again (you have to use separate folders for each storj container/ docker) and the network ports you plan to use (I used a field of 4 ports for each container: 4000-4003 for container 1, 4004-4007 for container 2 etc.) 5. Launch the docker, then shut it down immediately and edit the config file located in the folder you pointed to earlier. Edit the following fields: "rpcAddress": "yourdomain.ddns.com", "rpcPort": 4000, "doNotTraverseNat": true, "tunnelGatewayRange": { "min": 4001, "max": 4003 }, 6. Now you can relaunch the container. Everything should run fine... I have 3 running now... I think some steps are possibly unnecessary like deleting the storage field before reentering it as well as using 4 ports for each docker and specifying the tunneling ports. This config worked for me though, so here you go. Don't forget to forward your ports though the NAT and the firewall.