Copying to /mnt/user/[shares] from NTFS drives

[JayJ]

I'm finally doing it! I'm a WHS2011 refugee that has been sitting on a DrivePool install for far too long. Have finally blown away Windows 10 and have started off with Unraid. It all seems to make sense in terms of setting things up, but my google-fu is failing me and I can't seem to join the dots on how the /mnt/user/share concept works.

 

I have setup an array, and parity hasn't synced yet. I've turned it off in the hope that I once I've copied my data over I can sync it. I've mounted my old NTFS drives that have data and am trying to copy them over one by one into shares that I've created through the UI in /mnt/user/share {edited typo had /mnt/share} (something along the lines of cp -rv /mnt/disks/ntfs_disk1 /mnt/user/share/downloads.

 

I get that /mnt/user is supposed to be a logical volume that spans the disks in the array but it doesn't seem to be working. What is happening is I'm filling up the rootfs and the shares don't seem to be writing to the array. I can't seem to figure out what to google for...what am i doing wrong?

Edited September 17, 2021 by JayJ
typo

[trurl]

9 minutes ago, JayJ said:

mnt/share

This is wrong and is a path in rootfs

 

 

9 minutes ago, JayJ said:

mnt/user/share

This is right .

 

Which are you doing? 

[trurl]

Try Midnight Commander (mc at the command line), that's what I usually do instead of fumbling with Linux commands. 

 

Google Midnight Commander 

[JayJ]

sorry, i am trying to copy to mnt/user/[share].  i'd imagine i'd have the same issue with MC right as that's just doing a cp. 

[trurl]

mc just makes it easier to not make a mistake at the command line. 

 

Post Diagnostics 

[trurl]

Is the array started? If not /mnt/user is also a path in rootfs. 

 

Midnight Commander would make it obvious the path doesn't exist 

[trurl]

46 minutes ago, JayJ said:

parity hasn't synced yet. I've turned it off

Just unassign parity if you want to sync it later. You have to start the array or its disks and the user shares aren't mounted. 

[trurl]

If you created the /mnt/user path by using that command without the array started you will probably have to reboot. 

[itimpi]

1 hour ago, trurl said:

This is wrong and is a path in rootfs

 

 

This is right .

 

Which are you doing? 

You need to make sure any path starts with a / to avoid it being a relative path.

[JayJ]

3 hours ago, trurl said:

If you created the /mnt/user path by using that command without the array started you will probably have to reboot. 

i created the share via the UI to make sure that any logic that needs to get applied occurs.  

 

to be clear, what is happening is i set the cp to run and then i get out of space errors and the entire computer cracks it because / is full...the top line when i run a df is at 100%.  going into /mnt/user/[shares] and deleting things reduces the rootfs back down.  I've had this issue twice now, first time i blew everything away and started again from scratch--i can't figure out what i'm doing wrong!

blacktron-diagnostics-20210918-0046.zip

[itimpi]

If you misspelt a share name (or got its case wrong) then you would end up writing to RAM rather than physical storage.

 

you might want to get a

  ls -d /mnt/user/* 

when you have this problem to ensure that every folder showing there corresponds to a share you have set up and thus a folder on at least one of your drives.

[trurl]

9 hours ago, JayJ said:

to be clear

Still not entirely clear. Really only need the answer to one question. Is the array actually started when you issue that cp command?

 

Actually, this is a better question. Did you ever issue that cp command without the array started?

[JayJ]

yes i think you're both right.  at some stage I must have had a typo and attempted to copy either with the array not started or mistyped a directory.  after a good nights sleep and some valheim distraction i wrote a script to do the copy with some validation and monitoring on free space and it appears to be running and copying correctly. thank you all for your help! This has been super useful at helping me understand how unraid works under the hood.

 

For anyone migrating from DrivePool to Unraid, here is the script that I'm using to move the files over and the steps I've taken.

 

What I did was:

1. Create an array with sufficient space for my data.
2. Create the shares and make sure that I've set appropriate minimum free space.
3. Turned off parity syncing while copying.
4. Mount the drives using the Unassigned Drive plugin.
5. Run the below script.  The -n switch on the cp command won't overwrite copy so is useful if it needs to stop and be restarted.

I'm not a dev, so I don't expect that the below is the best way of doing things though. 

 

#!/bin/bash

# set array share paths
downloads=/mnt/user/downloads
person=/mnt/user/person
media=/mnt/user/media

# check shares exist
if [ ! -d $downloads ]; then
	echo "Download share doesn't exist"
	exit 9999
fi

if [ ! -d $person ]; then
	echo "person share doesn't exist"
	exit 9999
fi

if [ ! -d $media ]; then
	echo "Media share doesn't exist"
	exit 9999
fi

# enumerate through mounted ntfs disk using unassigned drive plugin
for diskname in /mnt/disks/*
do
	# set the DrivePool directory name on the mounted disk
    sourcefolder=$diskname/Pool*/ServerFolders

	# check / isn't filling up
	freespace=`df / | awk '/[0-9]%/{print $(NF-2)}'`
	if [[ $freespace -lt 6500000 ]]; then
	   echo "/ is filling up and is running out of space"
	   exit 9999
    fi

	# copy files from ntfs to array
	cp -rvn $sourcefolder/person/* $person/
	cp -rvn $sourcefolder/Downloads/* $downloads/
	cp -rvn $sourcefolder/Music/* $media/music/
	cp -rvn $sourcefolder/Videos/TV\ Shows/* /$media/videos/tv/
	cp -rvn $sourcefolder/Videos/Movies/* $media/videos/movies/
done

 

Edited September 18, 2021 by JayJ