Suggestions for Dropbox > Unraid server predicament


Recommended Posts

I'm a graphic designer and I just took on a new client. They use Dropbox to manage their assets. To give me the things I'll need to do work for them, they shared some folders with me via Dropbox. Now, because Dropbox is basically a file-sharing extortion racket, I was forced to register a paid $12/mo account myself just to access the files my client is already paying them to store. Straight up robbery, but whatever, right? Cost of doing business. I can live with that.

 

The real problem is, how do I sync my client's files? My PC is a Windows VM running in Unraid. I keep all my files on the array, with scheduled backup via SpiderOak. Aside from a passed-through SSD used solely for games, my VM doesn't have a physical disk to speak of. Just a disk image with enough space to hold the OS. And that's a problem because Dropbox's desktop client doesn't let you use Samba shares, even if you mount as a network drive.

 

My first instinct was to use a Docker container instead, and give it paths to my array. There are Dropbox docker images available on Dockerhub. I tried this one. But it doesn't work because apparently, Dropbox's latest requirements cause their headless client to reject anything but an EXT4 filesystem. Somehow, it's able to detect that the container path is actually a directory in a BTRFS filesystem. So it throws an error and refuses to sync.

 

Someone did a workaround for the filesystem restriction, but it specifically says "This is an experimental fix not supported by Dropbox. It might cause data loss.” My client will not be pleased with me if my machine corrupts or erases their files. So this really is not an option.

 

I would just download the files from the web browser, but the folders are so large that the Dropbox web interface gives an error when I try it. 

 

The hell should I do? 

Link to comment

I've got an idea, not an ideal solution, but it will get you by for now.

Add a second disk to your VM, so it's seen as local storage for your VM's dropbox. You can either define a vdisk.img on an array disk, or if you have the extra ports and physical space, you could pass through a whole physical disk. Either way you can add the extra space as a path or drive letter inside the VM.

Link to comment
21 minutes ago, jonathanm said:

I've got an idea, not an ideal solution, but it will get you by for now.

Add a second disk to your VM, so it's seen as local storage for your VM's dropbox. You can either define a vdisk.img on an array disk, or if you have the extra ports and physical space, you could pass through a whole physical disk. Either way you can add the extra space as a path or drive letter inside the VM.

Good call. That will at least let me sync the files, so that's a start. But I'd really prefer to keep their files where I keep all my other projects... on the array. I wonder if there's some way to do this.

Link to comment

Well, if the additional disk to your VM will only keep Dropbox files, you can happily store the additional disk image on the array. there won't be too big an impact on your VM only the saving of a file would be impacted by somewhat slow array write speeds.

 

Otherwise, you might want to look to using rclone as a tool for syncing local files to Dropbox. rclone is a CLI tool though and does have some realtime support for syncing directories.

 

Link to comment

Do you actually need the Dropbox client on the Windows PC or merely access to the files stored in yous Dropbox?

 

The reason I ask is that I got around the requirement for a Linux client to have an ext4 file system by creating an image file that I mounted using the loop device.   I then formatted the contents of the image file to ext4 format; mounted this image at /mnt/cache/Dropbox; and set a Dropbox docker container running with this mounted image as the datastore.  The /mnt/cache/Dropbox folder shows up as the share 'Dropbox' so can be accessed by any client capable of accessing shares.

 

I have the more detailed instructions on the exact steps available if you want them although I would need to dig them out.

Link to comment
3 hours ago, itimpi said:

Do you actually need the Dropbox client on the Windows PC or merely access to the files stored in yous Dropbox?

 

The reason I ask is that I got around the requirement for a Linux client to have an ext4 file system by creating an image file that I mounted using the loop device.   I then formatted the contents of the image file to ext4 format; mounted this image at /mnt/cache/Dropbox; and set a Dropbox docker container running with this mounted image as the datastore.  The /mnt/cache/Dropbox folder shows up as the share 'Dropbox' so can be accessed by any client capable of accessing shares.

 

I have the more detailed instructions on the exact steps available if you want them although I would need to dig them out.

That sounds like exactly what I need! Can you post instructions?

Link to comment
On 7/4/2019 at 7:35 PM, cyberspectre said:

That sounds like exactly what I need! Can you post instructions?

I cannot locate my exact notes on setting this up at the moment.   However attached is a script I have set under User 

scripts to run on array start that should give you enough to get this working.

 

#!/bin/bash
#  Script (partially commented out) for moving dropbox to a loopback image file
#  This gets around the restriction of Dropbox now only supporting ext4 on Linux
#  (see https://popey.com/blog/posts/fixing-broken-dropbox-sync-support.html)
#
#  we need to leave active the bit about mounting the image at array start

# Location of the image which will contain the new ext4 partition
# Typically this would be on the cache but this is not mandatory.
DROPBOXDISK=/mnt/cache
DROPBOXFILE=$DROPBOXDISK/Dropbox.img

# Current location of my existing Dropbox folder
# (using User Share means we do not need t/ know device currently used)
DROPBOXHOME=/mnt/cache/Dropbox

# Where we will copy the existing folder to. If you have little space, you could make this
# a folder on a USB drive.  If you have no existing folder this is not required
DROPBOXBACKUP=/mnt/cache/Dropbox.old

# What size is the Dropbox image file going to be. It makes sense to set this
# to whatever the capacity of your Dropbox account is, or a little more.
#DROPBOXSIZE="20G"

# Start by making sure the disk to be used is present
if [ ! -e $DROPBOXDISK ]; then
  logger -s "ERROR:  $DROPBOXDISK disk not present"
  exit -1
else
  # ...and that we have not already followed this process
  if [ -e $DROPBOXFILE ]; then
    logger -s "Found existing $DROPBOXFILE"
  else
    # Create a 'sparse' file which will start out small and grow to the maximum
    # size defined above. So we don't eat all that space immediately.
    sudo dd if=/dev/zero of="$DROPBOXFILE" bs=1 count=0 seek="$DROPBOXSIZE"

    # Format it ext4, because Dropbox Inc. says so
    sudo mkfs.ext4 "$DROPBOXFILE"

    # Move the current Dropbox folder to the backup location
    if [ -f $DROPBOXHOME]; then
      sudo mv "$DROPBOXHOME" "$DROPBOXBACKUP"
    fi

    # Make a new Dropbox folder to replace the old one. This will be the mount point
    # under which the sparse image file will be mounted.  We want to make sure it is
    # on the same device as the image file.
    sudo mkdir $DROPBOXDISK/`basename $DROPBOXHOME`

    # Set ownership and permissions on the new folder so Dropbox has access
    # (generic Linux case)
    # sudo chown $(id -un) "$DROPBOXHOME"
    # sudo chgrp $(id -gn) "$DROPBOXHOME"
    # (Unraid specific case)
    sudo chown nobody "$DROPBOXHOME"
    sudo chgrp users "$DROPBOXHOME"

    # Make sure the mount point can't be written to if for some reason the partition 
    # doesn't get mounted. We don't want dropbox to see an empty folder and think 'yay, let's delete
    # all his files because this folder is empty, that must be what they want'
    sudo chattr +i "$DROPBOXHOME"
 
    # Mount the sparse file at the dropbox mount point
    sudo mount -o loop "$DROPBOXFILE" "$DROPBOXHOME"

    # Copy the files from the existing dropbox folder to the new one, which will put them
    # inside the sparse file. You should see the file grow as this runs.
    if [ -d $DROPBOXBACKUP ]; then
      sudo rsync -a "$DROPBOXBACKUP"/ "$DROPBOXHOME"/
    fi

    # Let's unmount it as we normally want to use fstab entry
    sudo umount "$DROPBOXHOME"
  fi
fi

# Create a line in our /etc/fstab so this gets mounted on every boot up
status=`grep -o "$DROPBOXHOME" /etc/fstab`
if [ $? -eq 0 ]; then
  logger -s "Entry for $DROPBOXHOME already present in /etc/fstab"
else
  echo "$DROPBOXFILE" "$DROPBOXHOME" ext4 loop,defaults,rw,relatime,exec,user_xattr 0 0 | sudo tee -a /etc/fstab
  logger -s "Entry for $DROPBOXHOME added to /etcfstab"
fi

# This will mount as per the fstab 
sudo mount -a

logger -s "$DROPBOXFILE mounted at $DROPBOXHOME"
exit 0




and then on array stop I run

 

#!/bin/bash
umount /mnt/cache/Dropbox
logger "Dropbox image unmounted"

 

This was based on a post I found online, but I cannot find the URL of that original post.

 

I then installed 'roninkenji's docker container via Community Applications and configured it to use the Dropbox folder I had just set up.

Edited by itimpi
Link to comment
  • 2 weeks later...
On 7/5/2019 at 3:19 AM, itimpi said:

I cannot locate my exact notes on setting this up at the moment.   However attached is a script I have set under User 

scripts to run on array start that should give you enough to get this working.

 


#!/bin/bash
#  Script (partially commented out) for moving dropbox to a loopback image file
#  This gets around the restriction of Dropbox now only supporting ext4 on Linux
#  (see https://popey.com/blog/posts/fixing-broken-dropbox-sync-support.html)
#
#  we need to leave active the bit about mounting the image at array start

# Location of the image which will contain the new ext4 partition
# Typically this would be on the cache but this is not mandatory.
DROPBOXDISK=/mnt/cache
DROPBOXFILE=$DROPBOXDISK/Dropbox.img

# Current location of my existing Dropbox folder
# (using User Share means we do not need t/ know device currently used)
DROPBOXHOME=/mnt/cache/Dropbox

# Where we will copy the existing folder to. If you have little space, you could make this
# a folder on a USB drive.  If you have no existing folder this is not required
DROPBOXBACKUP=/mnt/cache/Dropbox.old

# What size is the Dropbox image file going to be. It makes sense to set this
# to whatever the capacity of your Dropbox account is, or a little more.
#DROPBOXSIZE="20G"

# Start by making sure the disk to be used is present
if [ ! -e $DROPBOXDISK ]; then
  logger -s "ERROR:  $DROPBOXDISK disk not present"
  exit -1
else
  # ...and that we have not already followed this process
  if [ -e $DROPBOXFILE ]; then
    logger -s "Found existing $DROPBOXFILE"
  else
    # Create a 'sparse' file which will start out small and grow to the maximum
    # size defined above. So we don't eat all that space immediately.
    sudo dd if=/dev/zero of="$DROPBOXFILE" bs=1 count=0 seek="$DROPBOXSIZE"

    # Format it ext4, because Dropbox Inc. says so
    sudo mkfs.ext4 "$DROPBOXFILE"

    # Move the current Dropbox folder to the backup location
    if [ -f $DROPBOXHOME]; then
      sudo mv "$DROPBOXHOME" "$DROPBOXBACKUP"
    fi

    # Make a new Dropbox folder to replace the old one. This will be the mount point
    # under which the sparse image file will be mounted.  We want to make sure it is
    # on the same device as the image file.
    sudo mkdir $DROPBOXDISK/`basename $DROPBOXHOME`

    # Set ownership and permissions on the new folder so Dropbox has access
    # (generic Linux case)
    # sudo chown $(id -un) "$DROPBOXHOME"
    # sudo chgrp $(id -gn) "$DROPBOXHOME"
    # (Unraid specific case)
    sudo chown nobody "$DROPBOXHOME"
    sudo chgrp users "$DROPBOXHOME"

    # Make sure the mount point can't be written to if for some reason the partition 
    # doesn't get mounted. We don't want dropbox to see an empty folder and think 'yay, let's delete
    # all his files because this folder is empty, that must be what they want'
    sudo chattr +i "$DROPBOXHOME"
 
    # Mount the sparse file at the dropbox mount point
    sudo mount -o loop "$DROPBOXFILE" "$DROPBOXHOME"

    # Copy the files from the existing dropbox folder to the new one, which will put them
    # inside the sparse file. You should see the file grow as this runs.
    if [ -d $DROPBOXBACKUP ]; then
      sudo rsync -a "$DROPBOXBACKUP"/ "$DROPBOXHOME"/
    fi

    # Let's unmount it as we normally want to use fstab entry
    sudo umount "$DROPBOXHOME"
  fi
fi

# Create a line in our /etc/fstab so this gets mounted on every boot up
status=`grep -o "$DROPBOXHOME" /etc/fstab`
if [ $? -eq 0 ]; then
  logger -s "Entry for $DROPBOXHOME already present in /etc/fstab"
else
  echo "$DROPBOXFILE" "$DROPBOXHOME" ext4 loop,defaults,rw,relatime,exec,user_xattr 0 0 | sudo tee -a /etc/fstab
  logger -s "Entry for $DROPBOXHOME added to /etcfstab"
fi

# This will mount as per the fstab 
sudo mount -a

logger -s "$DROPBOXFILE mounted at $DROPBOXHOME"
exit 0




and then on array stop I run

 


#!/bin/bash
umount /mnt/cache/Dropbox
logger "Dropbox image unmounted"

 

This was based on a post I found online, but I cannot find the URL of that original post.

 

I then installed 'roninkenji's docker container via Community Applications and configured it to use the Dropbox folder I had just set up.

Thank you! I'm going to try this immediately after I install some upgrades in my server. Doing a 1TB M.2 cache drive and that's the perfect place to put this raw image file.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.