cyberspectre Posted July 4, 2019 Share Posted July 4, 2019 I'm a graphic designer and I just took on a new client. They use Dropbox to manage their assets. To give me the things I'll need to do work for them, they shared some folders with me via Dropbox. Now, because Dropbox is basically a file-sharing extortion racket, I was forced to register a paid $12/mo account myself just to access the files my client is already paying them to store. Straight up robbery, but whatever, right? Cost of doing business. I can live with that. The real problem is, how do I sync my client's files? My PC is a Windows VM running in Unraid. I keep all my files on the array, with scheduled backup via SpiderOak. Aside from a passed-through SSD used solely for games, my VM doesn't have a physical disk to speak of. Just a disk image with enough space to hold the OS. And that's a problem because Dropbox's desktop client doesn't let you use Samba shares, even if you mount as a network drive. My first instinct was to use a Docker container instead, and give it paths to my array. There are Dropbox docker images available on Dockerhub. I tried this one. But it doesn't work because apparently, Dropbox's latest requirements cause their headless client to reject anything but an EXT4 filesystem. Somehow, it's able to detect that the container path is actually a directory in a BTRFS filesystem. So it throws an error and refuses to sync. Someone did a workaround for the filesystem restriction, but it specifically says "This is an experimental fix not supported by Dropbox. It might cause data loss.” My client will not be pleased with me if my machine corrupts or erases their files. So this really is not an option. I would just download the files from the web browser, but the folders are so large that the Dropbox web interface gives an error when I try it. The hell should I do? Quote Link to comment
JonathanM Posted July 4, 2019 Share Posted July 4, 2019 I've got an idea, not an ideal solution, but it will get you by for now. Add a second disk to your VM, so it's seen as local storage for your VM's dropbox. You can either define a vdisk.img on an array disk, or if you have the extra ports and physical space, you could pass through a whole physical disk. Either way you can add the extra space as a path or drive letter inside the VM. Quote Link to comment
cyberspectre Posted July 4, 2019 Author Share Posted July 4, 2019 21 minutes ago, jonathanm said: I've got an idea, not an ideal solution, but it will get you by for now. Add a second disk to your VM, so it's seen as local storage for your VM's dropbox. You can either define a vdisk.img on an array disk, or if you have the extra ports and physical space, you could pass through a whole physical disk. Either way you can add the extra space as a path or drive letter inside the VM. Good call. That will at least let me sync the files, so that's a start. But I'd really prefer to keep their files where I keep all my other projects... on the array. I wonder if there's some way to do this. Quote Link to comment
ken-ji Posted July 4, 2019 Share Posted July 4, 2019 Well, if the additional disk to your VM will only keep Dropbox files, you can happily store the additional disk image on the array. there won't be too big an impact on your VM only the saving of a file would be impacted by somewhat slow array write speeds. Otherwise, you might want to look to using rclone as a tool for syncing local files to Dropbox. rclone is a CLI tool though and does have some realtime support for syncing directories. Quote Link to comment
itimpi Posted July 4, 2019 Share Posted July 4, 2019 Do you actually need the Dropbox client on the Windows PC or merely access to the files stored in yous Dropbox? The reason I ask is that I got around the requirement for a Linux client to have an ext4 file system by creating an image file that I mounted using the loop device. I then formatted the contents of the image file to ext4 format; mounted this image at /mnt/cache/Dropbox; and set a Dropbox docker container running with this mounted image as the datastore. The /mnt/cache/Dropbox folder shows up as the share 'Dropbox' so can be accessed by any client capable of accessing shares. I have the more detailed instructions on the exact steps available if you want them although I would need to dig them out. Quote Link to comment
cyberspectre Posted July 4, 2019 Author Share Posted July 4, 2019 3 hours ago, itimpi said: Do you actually need the Dropbox client on the Windows PC or merely access to the files stored in yous Dropbox? The reason I ask is that I got around the requirement for a Linux client to have an ext4 file system by creating an image file that I mounted using the loop device. I then formatted the contents of the image file to ext4 format; mounted this image at /mnt/cache/Dropbox; and set a Dropbox docker container running with this mounted image as the datastore. The /mnt/cache/Dropbox folder shows up as the share 'Dropbox' so can be accessed by any client capable of accessing shares. I have the more detailed instructions on the exact steps available if you want them although I would need to dig them out. That sounds like exactly what I need! Can you post instructions? Quote Link to comment
itimpi Posted July 5, 2019 Share Posted July 5, 2019 (edited) On 7/4/2019 at 7:35 PM, cyberspectre said: That sounds like exactly what I need! Can you post instructions? I cannot locate my exact notes on setting this up at the moment. However attached is a script I have set under User scripts to run on array start that should give you enough to get this working. #!/bin/bash # Script (partially commented out) for moving dropbox to a loopback image file # This gets around the restriction of Dropbox now only supporting ext4 on Linux # (see https://popey.com/blog/posts/fixing-broken-dropbox-sync-support.html) # # we need to leave active the bit about mounting the image at array start # Location of the image which will contain the new ext4 partition # Typically this would be on the cache but this is not mandatory. DROPBOXDISK=/mnt/cache DROPBOXFILE=$DROPBOXDISK/Dropbox.img # Current location of my existing Dropbox folder # (using User Share means we do not need t/ know device currently used) DROPBOXHOME=/mnt/cache/Dropbox # Where we will copy the existing folder to. If you have little space, you could make this # a folder on a USB drive. If you have no existing folder this is not required DROPBOXBACKUP=/mnt/cache/Dropbox.old # What size is the Dropbox image file going to be. It makes sense to set this # to whatever the capacity of your Dropbox account is, or a little more. #DROPBOXSIZE="20G" # Start by making sure the disk to be used is present if [ ! -e $DROPBOXDISK ]; then logger -s "ERROR: $DROPBOXDISK disk not present" exit -1 else # ...and that we have not already followed this process if [ -e $DROPBOXFILE ]; then logger -s "Found existing $DROPBOXFILE" else # Create a 'sparse' file which will start out small and grow to the maximum # size defined above. So we don't eat all that space immediately. sudo dd if=/dev/zero of="$DROPBOXFILE" bs=1 count=0 seek="$DROPBOXSIZE" # Format it ext4, because Dropbox Inc. says so sudo mkfs.ext4 "$DROPBOXFILE" # Move the current Dropbox folder to the backup location if [ -f $DROPBOXHOME]; then sudo mv "$DROPBOXHOME" "$DROPBOXBACKUP" fi # Make a new Dropbox folder to replace the old one. This will be the mount point # under which the sparse image file will be mounted. We want to make sure it is # on the same device as the image file. sudo mkdir $DROPBOXDISK/`basename $DROPBOXHOME` # Set ownership and permissions on the new folder so Dropbox has access # (generic Linux case) # sudo chown $(id -un) "$DROPBOXHOME" # sudo chgrp $(id -gn) "$DROPBOXHOME" # (Unraid specific case) sudo chown nobody "$DROPBOXHOME" sudo chgrp users "$DROPBOXHOME" # Make sure the mount point can't be written to if for some reason the partition # doesn't get mounted. We don't want dropbox to see an empty folder and think 'yay, let's delete # all his files because this folder is empty, that must be what they want' sudo chattr +i "$DROPBOXHOME" # Mount the sparse file at the dropbox mount point sudo mount -o loop "$DROPBOXFILE" "$DROPBOXHOME" # Copy the files from the existing dropbox folder to the new one, which will put them # inside the sparse file. You should see the file grow as this runs. if [ -d $DROPBOXBACKUP ]; then sudo rsync -a "$DROPBOXBACKUP"/ "$DROPBOXHOME"/ fi # Let's unmount it as we normally want to use fstab entry sudo umount "$DROPBOXHOME" fi fi # Create a line in our /etc/fstab so this gets mounted on every boot up status=`grep -o "$DROPBOXHOME" /etc/fstab` if [ $? -eq 0 ]; then logger -s "Entry for $DROPBOXHOME already present in /etc/fstab" else echo "$DROPBOXFILE" "$DROPBOXHOME" ext4 loop,defaults,rw,relatime,exec,user_xattr 0 0 | sudo tee -a /etc/fstab logger -s "Entry for $DROPBOXHOME added to /etcfstab" fi # This will mount as per the fstab sudo mount -a logger -s "$DROPBOXFILE mounted at $DROPBOXHOME" exit 0 and then on array stop I run #!/bin/bash umount /mnt/cache/Dropbox logger "Dropbox image unmounted" This was based on a post I found online, but I cannot find the URL of that original post. I then installed 'roninkenji's docker container via Community Applications and configured it to use the Dropbox folder I had just set up. Edited July 9, 2019 by itimpi Quote Link to comment
cyberspectre Posted July 14, 2019 Author Share Posted July 14, 2019 On 7/5/2019 at 3:19 AM, itimpi said: I cannot locate my exact notes on setting this up at the moment. However attached is a script I have set under User scripts to run on array start that should give you enough to get this working. #!/bin/bash # Script (partially commented out) for moving dropbox to a loopback image file # This gets around the restriction of Dropbox now only supporting ext4 on Linux # (see https://popey.com/blog/posts/fixing-broken-dropbox-sync-support.html) # # we need to leave active the bit about mounting the image at array start # Location of the image which will contain the new ext4 partition # Typically this would be on the cache but this is not mandatory. DROPBOXDISK=/mnt/cache DROPBOXFILE=$DROPBOXDISK/Dropbox.img # Current location of my existing Dropbox folder # (using User Share means we do not need t/ know device currently used) DROPBOXHOME=/mnt/cache/Dropbox # Where we will copy the existing folder to. If you have little space, you could make this # a folder on a USB drive. If you have no existing folder this is not required DROPBOXBACKUP=/mnt/cache/Dropbox.old # What size is the Dropbox image file going to be. It makes sense to set this # to whatever the capacity of your Dropbox account is, or a little more. #DROPBOXSIZE="20G" # Start by making sure the disk to be used is present if [ ! -e $DROPBOXDISK ]; then logger -s "ERROR: $DROPBOXDISK disk not present" exit -1 else # ...and that we have not already followed this process if [ -e $DROPBOXFILE ]; then logger -s "Found existing $DROPBOXFILE" else # Create a 'sparse' file which will start out small and grow to the maximum # size defined above. So we don't eat all that space immediately. sudo dd if=/dev/zero of="$DROPBOXFILE" bs=1 count=0 seek="$DROPBOXSIZE" # Format it ext4, because Dropbox Inc. says so sudo mkfs.ext4 "$DROPBOXFILE" # Move the current Dropbox folder to the backup location if [ -f $DROPBOXHOME]; then sudo mv "$DROPBOXHOME" "$DROPBOXBACKUP" fi # Make a new Dropbox folder to replace the old one. This will be the mount point # under which the sparse image file will be mounted. We want to make sure it is # on the same device as the image file. sudo mkdir $DROPBOXDISK/`basename $DROPBOXHOME` # Set ownership and permissions on the new folder so Dropbox has access # (generic Linux case) # sudo chown $(id -un) "$DROPBOXHOME" # sudo chgrp $(id -gn) "$DROPBOXHOME" # (Unraid specific case) sudo chown nobody "$DROPBOXHOME" sudo chgrp users "$DROPBOXHOME" # Make sure the mount point can't be written to if for some reason the partition # doesn't get mounted. We don't want dropbox to see an empty folder and think 'yay, let's delete # all his files because this folder is empty, that must be what they want' sudo chattr +i "$DROPBOXHOME" # Mount the sparse file at the dropbox mount point sudo mount -o loop "$DROPBOXFILE" "$DROPBOXHOME" # Copy the files from the existing dropbox folder to the new one, which will put them # inside the sparse file. You should see the file grow as this runs. if [ -d $DROPBOXBACKUP ]; then sudo rsync -a "$DROPBOXBACKUP"/ "$DROPBOXHOME"/ fi # Let's unmount it as we normally want to use fstab entry sudo umount "$DROPBOXHOME" fi fi # Create a line in our /etc/fstab so this gets mounted on every boot up status=`grep -o "$DROPBOXHOME" /etc/fstab` if [ $? -eq 0 ]; then logger -s "Entry for $DROPBOXHOME already present in /etc/fstab" else echo "$DROPBOXFILE" "$DROPBOXHOME" ext4 loop,defaults,rw,relatime,exec,user_xattr 0 0 | sudo tee -a /etc/fstab logger -s "Entry for $DROPBOXHOME added to /etcfstab" fi # This will mount as per the fstab sudo mount -a logger -s "$DROPBOXFILE mounted at $DROPBOXHOME" exit 0 and then on array stop I run #!/bin/bash umount /mnt/cache/Dropbox logger "Dropbox image unmounted" This was based on a post I found online, but I cannot find the URL of that original post. I then installed 'roninkenji's docker container via Community Applications and configured it to use the Dropbox folder I had just set up. Thank you! I'm going to try this immediately after I install some upgrades in my server. Doing a 1TB M.2 cache drive and that's the perfect place to put this raw image file. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.