Cant figure out how to get mover working

[RaidUnnewb]

Hello.

I started my trial of Unraid and I just cannot get this thing working. Set up a few shares, using SSD and a HDD cache depending on what I want to happen.
Havent tested the SSD's yet, but the mover just refuses to move from the HDD cache. Share settings are Yes, not prefer. Tried Mover Tuning plugin, still didnt work. Deleted that, still doesnt work. Cache is completely full right now and getting errors and this in the log file when I manually click move.

 

Ive had to delete everyhting a few times trying to set it up again in case I made a wrong error. Tired of running day and a half long parity checks every time I wipe the server...

 

I tried creating a diagnostic log file under tools, but it was taking forever and looked like it was logging every file I had, not something I want to share. Unless it just looks like its logging every single music folder, looking for .cfg's? Can I turn that part off?

 

Thank you!

 

 

 

 

[trurl]

55 minutes ago, RaidUnnewb said:

tried creating a diagnostic log file under tools, but it was taking forever and looked like it was logging every file I had

Are you sure your files are in user shares and not at the top level of pool or array disk?

[RaidUnnewb]

Thank you for responding

 

Here are the files it posts, every single one in my shares lol. Dont know why it needs to look at my PrusaSlicer folders

Could be top level. I guess I dont know what that means. I have a user share called "backups_computer" using a HDD cache set to Yes. I drag my files into the network mapped drive onto the share. Its my understanding that the files goto the cache, then mover supposed to move them to array.

 

 

Buttt, that doesnt really matter anyway since:

This is what happenes every time I try to take a diagnostic. 32GB ram.... On Unraid and main comp both.. ;/

 

Tried using firefox instead of chrome, came back and took this partway through.... (The only tab open is the diagnostics running)

 

And its still looking at every file and folder I have saved... Now its scanning my android phone backup saves. Needs to know I use Fandango and Fobo Bike tire pressure monitoring apps

Edited October 17, 2022 by RaidUnnewb
more info

[RaidUnnewb]

Woot, finally got the diagnostics!!!
Now, hmm. How to share without giving personal info away? Anonymize didnt seem to work, probably because I enabled mover logging a while ago to try and troubleshoot. Turned it off, but still have my computer name, username, file names, ect.

 

Heres a list of errors I was getting in the syslog.txt file:

btrfs_print_data_csum_error: 697 callbacks suppressed

BTRFS warning (device dm-4): csum failed root 5 ino 84018 off 447078400 csum 0x6b8931dd expected csum 0xae5354a1 mirror 1

/usr/local/sbin/mover &> /dev/null &

 shfs: copy_file: //..p/... /mnt/disk3/backups_computer/windows/fullbackups/DT-43420/Backup Set 2022-10-09 191436/Backup Files 2022-10-11 161052/Backup files 320.zip.partial (5) Input/output error

BTRFS error (device dm-4): bdev /dev/mapper/sde1 errs: wr 0, rd 0, flush 0, corrupt 149052, gen 0

create_parent: /mnt/cache_slowhdd/backups_computer/2021 old c drive backup/Prusa3D/PrusaSlicer/resources/profiles/Creality error: Inappropriate ioctl for device
Oct 13 16:01:26 <name of my unraid server>  move: file: //..l/...

 

Rest of errors are multiples of those with different files. Chronological order.

Running Unraid 6.11.1.
 

Willing to DM the zip, or if anyone can show me how to delete all the personal info before posting on internet, id appreciate it

Thank you!

 

[trurl]

Post a screenshot of User Shares

[trurl]

1 hour ago, RaidUnnewb said:

drag my files into the network mapped drive

I recommend not sharing disks on the network and only working with user shares.

 

1 hour ago, RaidUnnewb said:

Could be top level. I guess I dont know what that means

Any folder at the top level of pool or array disk is automatically a user share named for the folder.

 

I suspect you have accidentally created a lot of user shares by writing directly to a disk instead of to user shares.

[trurl]

1 minute ago, trurl said:

accidentally created a lot of user shares by writing directly to a disk instead of to user shares

Any shares created in this way will have default settings until you change them. One of those defaults is cache:no, and cache:no shares are ignored by mover.

[RaidUnnewb]

51 minutes ago, trurl said:

Post a screenshot of User Shares

 

Thinking about just scrapping the entire HDD cache and moving it to my main computer for temp storage. Dont really like having to split up my shares based on which cache to use.

Heres all my network drives. I was following youtube tutorials that said I have to map em ;/
Needed the drives themselves so I could use Windows backups. Everything else I use the folders for, one for each share.

 

Do you think using Windows writing to network drives is the reason its all messed up?

[RaidUnnewb]

/sigh

 

Right after posting the last reply, the server window refused to communicate again. Hard to do a hard reboot. Now Its checking Parity for another 2 days....

 

 

Heres the folder structure inside the cache drive thats not working:

 

Heres the index for the backups share:

 

And the index for the downloads share using the HDD:

 

 

Is the fact that the top one doesnt have /user/ the issue?

 

 

Thank you!

[trurl]

18 hours ago, RaidUnnewb said:

Needed the drives themselves so I could use Windows backups

Don't know why Windows would require actual drive on a remote computer. Any network share can be mapped to a drive on the Windows PC. I personally never bother with mapping drives since most software can browse the network without needing to specify a mapped drive.

 

19 hours ago, trurl said:

I recommend not sharing disks on the network and only working with user shares

It is even possible to lose data by working with disks and user shares at the same time when moving or copying.

 

Mover won't move files at the top level of a drive, it only works on user shares, which are top level folders.

 

What do you get from the command line with this?

ls /mnt/cache_slowhdd

 

 

 

 

[RaidUnnewb]

 

Also, after the hard reboot, it decided to work last night. And now my cache is empty like it should be. Will test tonight by throwing stuff on there.

 

Its been a week now so I don't remember which video. But I am using File History, which I guess is from windows 7 but is still in current windows. I've set every share to only be accessible via a user, so isn't it already using user shares? Tried adding a regular network location after deleting my shared drives. Still appears to work, and also still appears to show drive icon within File History.

 

Also, am I being dumb by deleting the name of my server in msPaint each screenshot?

 

Thank you for your patience. Am well aware of the potential frustration saying the same thing 40 times a day, every day

 

 

[trurl]

19 hours ago, RaidUnnewb said:

am I being dumb by deleting the name of my server in msPaint each screenshot

You will get better advice if you post diagnostics, which will include your server name in each line of syslog, and also lot of other useful information you don't need to worry about.

 

19 hours ago, RaidUnnewb said:

set every share to only be accessible via a user, so isn't it already using user shares

You can see all of your user shares on the User Shares page in the Unraid webUI.

On 10/16/2022 at 10:09 PM, trurl said:

Any folder at the top level of pool or array disk is automatically a user share named for the folder.

Each user share has its own security settings, but Windows doesn't allow multiple logins to a remote resource, even though it will ask you to login again. This might be causing you problems if you are trying to have different users from one computer.

 

 

[RaidUnnewb]

6 hours ago, trurl said:

You will get better advice if you post diagnostics, which will include your server name in each line of syslog, and also lot of other useful information you don't need to worry about.

 

You can see all of your user shares on the User Shares page in the Unraid webUI.

Each user share has its own security settings, but Windows doesn't allow multiple logins to a remote resource, even though it will ask you to login again. This might be causing you problems if you are trying to have different users from one computer.

 

 

I would be happy to post diagnostics, if it didnt contain a bunch of personal data, file structures, filenames, ect. Is there a way to scrub that, or delete the old one and make a new one now that I have mover logging disabled?

Thanks!

[RaidUnnewb]

Fck..... New problem....
Gotta love Move defaults instead of Copy defaults....

I restarted my server today, and cant decrypt the drives due to the encryption key being moved to my backup folder inside the vault..... I didnt realize until this second that when I dragged the folder to the drive, it CUT and Pasted instead of simply copying it. Its happened before but I thought I must have accidentally held shift or something.

Is there any way to decrypt, or did I lose a few gigs of data over the years permanently? (I have my diagnostics file I took originally if its in there)

 

Thank you!

Edited October 19, 2022 by RaidUnnewb

[trurl]

41 minutes ago, RaidUnnewb said:

now that I have mover logging disabled

You would have to reboot to get syslog cleared of those logged moves.

 

16 minutes ago, RaidUnnewb said:

any way to decrypt

Not without the key. You wouldn't want it to work any other way, right?

[RaidUnnewb]

7 minutes ago, trurl said:

You would have to reboot to get syslog cleared of those logged moves.

 

Not without the key. You wouldn't want it to work any other way, right?

Yeah true. Was just hoping that since I was using such little space (1TB total) on a 16TB parity drive, there was a chance that the parity drive could be read since if its a 1, itll be a 1 (if that makes sense).

 

Ill wait an hour and just completely nuke the server. Start a new trial ect, since the past 10 days have been over 8 just writing parityies every time I screw up or the server refuses to speak. This will give me a chance to take out the HDD cache and only use 1 directory to keep it simpler.

 

 

For future knowledge: is there a way to keep unraid (maybe this is just windows 11, first weeks of using this too) from MOVING when I meant to COPY? Default settings, ect?

 

Thank you for all your help, even if it was ultimately .....

Edited October 19, 2022 by RaidUnnewb

[trurl]

2 minutes ago, RaidUnnewb said:

hoping that since I was using such little space (1TB total) on a 16TB parity drive, there was a chance that the parity drive could be read since if its a 1, itll be a 1 (if that makes sense).

No it doesn't make any sense at all. All of parity is always used, it is always completely full of parity bits. Doesn't matter at all how much space is used on any of your data disks. Parity contains none of your data, it is just parity bits that allows a missing disk to be calculated from parity and all the other array disks.

 

This is basically the way parity always works wherever it is used in computers and communications. Other conventional RAID systems stripe parity and data across the array but parity is still just an extra bit that allows a missing bit to be calculated from all the other bits. With Unraid, parity is all on one disk, and each data disk is an independent filesystem that can be read all by itself. This allows mixed size disks to be used in the array and allows disks that haven't failed to still be read even if too many other disks have failed, unlike conventional RAID.

 

In the special case where there is only a single data disk, parity is effectively a mirror since that is just how the calculation comes out. But in that case if the single data disk was encrypted, so would the mirror be.

 

https://wiki.unraid.net/Manual/Overview#Parity-Protected_Array

 

 

[RaidUnnewb]

Thanks. I was thinking that in the case of a single disk info plus parity being a mirror, since parity can only be done when the array is mounted and decrypted, then the parity would just be the data itself, not encrypted.

 

Thank you for your time, nuking the server /sigh.....

Can delete this entire thread since no issues were actually solved, just a circle of frustrations

[trurl]

7 minutes ago, RaidUnnewb said:

since parity can only be done when the array is mounted and decrypted, then the parity would just be the data itself, not encrypted.

That wouldn't be secure, and besides, parity is just bits. It doesn't know anything at all about encryption or filesystems. In the case of a single data disk, it is just a mirror of the exact bits on that single data disk. And those exact bits are an encrypted filesystem.

[trurl]

40 minutes ago, RaidUnnewb said:

For future knowledge: is there a way to keep unraid (maybe this is just windows 11, first weeks of using this too) from MOVING when I meant to COPY?

If you dragged them, then I have to assume you were doing this from some other OS on the network, and that other OS is doing what it does.

 

There are several ways to work with files directly on the server. I recommend Dynamix File Manager plugin (requires Unraid 6.10 or greater)

[trurl]

1 hour ago, RaidUnnewb said:

cant decrypt the drives due to the encryption key being moved to my backup folder inside the vault..... I didnt realize until this second that when I dragged the folder to the drive, it CUT and Pasted instead of simply copying it

What good would a copy of the key be if it was on encrypted storage anyway. You need a different backup plan.

[RaidUnnewb]

12 minutes ago, trurl said:

If you dragged them, then I have to assume you were doing this from some other OS on the network, and that other OS is doing what it does.

 

There are several ways to work with files directly on the server. I recommend Dynamix File Manager plugin (requires Unraid 6.10 or greater)

I am using windows. Unraid is running on another computer. I simply dragged the folders I want copied onto the network shares. It didnt copy, it Moved. I am assuming this is a windows thing, since I am new to this version of windows too.

[RaidUnnewb]

7 minutes ago, trurl said:

What good would a copy of the key be if it was on encrypted storage anyway. You need a different backup plan.

I dont really care if copies of the keys are everywhere. Unraid being encrpyted is simply so if I have to RMA a drive or something, or someone comes in and swipes a drive (no clue why anyone would) then it doesnt have all my banking info, pron, whatever just sitting there. Im not fooling myself into attempting to keep the NSA out of my files.