[DEPRECATED] How to partition, format ext4, and mount an SSD outside the array

[dmacias]

I think you have to do fdisk -c -u

[eroz]

I think you have to do fdisk -c -u

I had to use fdisk without any options and then I was able to continue.

[jumperalex]

Hmm I wonder if there has been a change in fdisk in recent versions.  I'll look into it. 

[jumperalex]

Not sure what the issue was originally but it doesn't appear to be the drive mount script.  However is there any way to view the drive from windows explorer?  I'm not 100% sure if I need to create the directories for NZBGet prior to assigning them and wanted to see the folder structure in the explorer for the appdisk.

 

For example I'm trying to create a NZBGet folder for NZBDrone to drop files into: /mnt/appdisk/nzbget/tv

 

Getting a path does not exist error, not sure if I could view the appdisk in the windows explorer to create these folders?

 

Sean did you figure it out?  Sorry I haven't chimed it sooner, but it seems that I'm not getting reliable notifications when people post to this thread.  In any case let us know if you worked out your issue.

[Nezil]

First off, great tutorial jumperalex!

 

I've actually been using an external 2.5 inch laptop drive connected via USB for my applications for several years now. My unRAID system has 15 3.5" drives, and that's pretty much all I can fit in my case, but there was just enough room behind the motherboard at the bottom of the case for a drive like this. I also figure that it runs cooler and draws less power than a 3.5" drive.

 

In case anyone is interested, my method is subtly different from yours, and some of the changes might be useful to others on the forum. I'll try to talk through each one in turn:

• I ended up formatting my drive ReiserFS, simply because it was easy and I didn't need TRIM capabilities (this alters the mount command slightly of course).
  
• I mounted my drive in /apps in the root of the file system rather than in /mnt. I was worried that some shutdown scripts might try to unmount everything in /mnt, and I wouldn't want that to happen.
  
• I named the partition on my external drive 'Applications' and mounted using the /dev/disk/by-label/Applications path rather than the disk ID path, which is slightly more human readable.
  
• I chose to create an extremely simple plugin for mounting my drive, and put the plugin in the /boot/plugins folder rather than the conventional /boot/config/plugins location. Plugins in this location are installed before any plugins in the normal location, and some of my plugins reference my /apps drive at installation. The go script is run as the last thing once unRAID is booted after all plugins are installed.
  
• My smb-extra.conf file is slightly different. I have set up a single user account that I use for connecting to the unRAID system and browsing shares in normal use. I therefore defined the additional share as browseable and publicly readable, but only writeable for this one user.
  

My smb-extra.conf looks like this:

[Apps]
        path = /apps
        comment = Installed Applications Disk
        browseable = yes
        # Secure
        public = yes
        writeable = no
        write list = neil

 

The full contents of my Extradrive-noarch-1nr.plg file are as follows:

<PLUGIN>
<!--
2012-04-20 - Neil Robinson: First release

The plugin mounts an additional drive outside of the array, before additional plugins are loaded.
-->

<!-- Install Script -->

<FILE Name="/tmp/extradrive-install" Run="/bin/bash">
<INLINE>
<![CDATA[

if mount | grep -q /apps; then
  echo "Apps drive mounted"
else
  echo "Mounting apps drive..."
  if [ ! -d /apps ]; then
    mkdir /apps
  fi
  mount -o noatime,nodiratime -t reiserfs /dev/disk/by-label/Applications /apps 2>&1
fi

echo "Done."        
  
]]>
</INLINE>
</FILE>

<!-- Plugin-Installed File -->

<FILE Name="/var/log/plugins/extradrive">
<INLINE>
<![CDATA[
The extradrive plugin mounts an additional drive outside of the array, before additional 
plugins are loaded. This is as an alternative to the cache drive.
]]>
</INLINE>
</FILE>
</PLUGIN>

[jumperalex]

Nezil, that is great stuff! Thanks!  With your permission I'll see about incorporating those optional differences into the OP.

 

I do have to ask though, are you sure GO is the last thing executed?  I had thought it was first and that is why, compared to SNAP, it didn't cause problems with plugins that needed the drive mounted first in order to work.

 

If that is true though, then I really like the idea of using your plug-in because I know we're supposed to keep GO as clean as possible and I'd rather use a plugin as well.  Though ... for things like Safe-Mode we still might want to allow out-of-array mounts to still occur [shrug] I'm just sort of thinking out loud at this point.  Always a dangerous thing ;-)

 

Anyway, thanks again.  More input is always better.  Hopefully my server upgrade goes smoothly this weekend and I'll have time to dig into the OP with some changes.

[unTER]

Thank you Nezil for the new info and jumperalex for maintaining your original posts, as it's still linked to all the time. I look forward to the update.

[Nezil]

Having started reading this thread, I went back and had a look at my plugin and smb-extra.conf to see if there was anything that I could do to enhance them at all.

 

It turns out there is... And it's all to do with permissions.

 

As I'd previously posted, I was exporting my apps drive as a Samba share, but to be honest, I wasn't really using it. I used the apps drive primarily for plugins that required more storage space, and didn't really have a need to connect to the drive from another machine until recently. In the last month or so however, I've started using the drive as the download location for torrents, and I obviously want access to the Completed Downloads directory.

 

For security reasons, I don't really need to see the modify that my plugins are using, so I set the permissions on these folders to only be readable by their respective owners.

 

To illustrate the point, here is the output of an ls -la of my /apps drive:

 

drwxrwxr-x 17 root         users  4096 2014-05-11 14:18 ./
drwxr-xr-x 18 root         root      0 2014-05-11 14:40 ../
drwx------  2 root         root   4096 2014-04-16 12:04 .ssh/
drwxrwx---  7 transmission users  4096 2014-05-11 15:09 Downloads/
drwx------  3 unraid-plex  users  4096 2012-12-08 00:23 Plex\ Media\ Server/
drwx------  5 root         root   4096 2014-05-09 15:32 PlexConnect/
drwx------  4 root         root   4096 2012-12-08 22:05 crashplan/
drwx------  2 root         root   4096 2014-03-17 13:38 hosts_update/
drwx------  2 root         root  16384 2014-05-10 13:03 lost+found/
drwx------  2 root         root   4096 2014-05-08 10:42 openvpn/
drwx------ 10 sickbeard    users  4096 2014-05-11 15:25 sickbeard/
drwx------ 18 slimserver   users  4096 2013-12-01 14:20 slimserver/
drwx------  2 root         root   4096 2014-04-16 12:01 ssh/
drwx------  3 unraid-plex  users 12288 2014-05-11 20:18 tmp/
drwx------  5 transmission users  4096 2014-05-11 20:37 transmission/

 

Transmission, Plex, Sickbeard and Slimserver all run as their own users. The other applications run as root and therefore the folders are simply owned by root.

 

The Downloads folder, which is used for torrent downloads is owned by the transmission user, but is also Read, Write and Executable by the users group, which the user 'neil' is a member of.

 

The smb-extra.conf permissions are in addition to the file permissions on the host OS, so if I connect to the Samba share as 'neil', even though I would see all of the folders in the root of the share, I wouldn't be able to access them, which is what I want! If I don't have access to them though, I don't really want to see them in the Samba share either, and luckily Samba has an option for this. My smb-extra.conf now looks like this:

 

[Apps]
        path = /apps
        comment = Installed Applications Disk
        browseable = yes
        # Secure
        public = yes
        writeable = no
        write list = neil
hide unreadable = yes

 

Now all that I see when I connect to the Samba Apps share, is the Downloads folder, to which I have read and write access.

 

One final change was necessary to allow me to actually create new folders at the root of the Apps share. By default, the new folder into which the extra drive is mounted, is owned by the user and group root. I wasn't able to write to the root of the folder as 'neil', so I changed the group ownership to 'users' and made it Read, Write and Executable to the group.

 

The relevant lines in my extradrive plugin now look like this:

 

if mount | grep -q /apps; then
  echo "Apps drive mounted"
else
  echo "Mounting apps drive..."
  if [ ! -d /apps ]; then
    mkdir /apps
  fi
  mount -o noatime,nodiratime -t ext4 /dev/disk/by-label/Applications /apps 2>&1
  chgrp users /apps
  chmod 775 /apps  
fi

echo "Done." 

 

As you can see... I also changed my drive to ext4 format in the hope that it suffers less from fragmentation, particularly now that I'm using it for torrent downloads.

[Nezil]

One last thing to do, might be to add the smb-extra.conf information to the plugin. I don't need it myself because I'd already created the file, but for it to be truly usable as a plugin by other users, this would be necessary.

 

If anyone would like that doing, please let me know, and I'll look at creating a fully fledged plugin for the purpose.

[jumperalex]

That is some great work there Nezil. 100% applicable here, but probably its also worthy of its own thread so other's can find it easier.  Seriously ... completely and totally on-topic, but will probably get better visibility in its own thread as well.

[archedraft]

SO first off great guide! The only problem is connecting from the share as my normal user. If I remove "force user = root" from smb-extra.conf I can map a drive from Windows 7, however, I cannot make a new file in that folder. What am I doing wrong?

[jumperalex]

Yeah I noticed that too but it hasn't been something I needed badly enough.  I'll see if I can look into it but I'm pretty engrossed in VM stuff right now.  Though I think Nezil posted with some new thoughts that might solve your problem.  Check his posts.  I said I would put them in OP when I had the time, but like I said, I've been engrossed with VMs  But Ill get there.

[archedraft]

Yeah I hear you with attempting to figure out all the VM stuff! I re-read Nezil posts and it sounds like I may be able to create folders as long as I am not in the root of my mounted drive. I'll have to mess around with this tomorrow though.

[jumperalex]

Yeah I hear you with attempting to figure out all the VM stuff! I re-read Nezil posts and it sounds like I may be able to create folders as long as I am not in the root of my mounted drive. I'll have to mess around with this tomorrow though.

 

Yes indeed I can copy to folders just not to the root of the drive.  Since I normally have folders is why I didn't much care.

 

Sent from my Nexus 7 using Tapatalk

 

 

[trurl]

...I do have to ask though, are you sure GO is the last thing executed?...

Just thought I would comment on this question. If you look in syslog you can actually see this.

 

First you get a line that says "Installing system plugins". This is the plugins in /boot/plugins.

Then later you get a line that says "Installing user plugins". This is the plugins in /boot/config/plugins.

Then later you get a line that says "Starting go script". This is, of course, the go script.

 

Before any of this, unRAID installs any packages in /boot/extra.

 

[dalben]

I use my cache drive purely for apps.  128gb SSD which is hardly utilized.  I want to start preparing for a dip into the VM pool.

 

If I follow these instructions I assume I will have a drive mounted in my system, outside of th unRAID array, that can still db accessed by my cron jobs for copying, run unRAID plugins and be able to be used as the VM repository and disk allocation ?

 

The only real change would be the /mnt/cache to /mnt/whatevericallit mount name ?

[jumperalex]

...I do have to ask though, are you sure GO is the last thing executed?...

Just thought I would comment on this question. If you look in syslog you can actually see this.

 

First you get a line that says "Installing system plugins". This is the plugins in /boot/plugins.

Then later you get a line that says "Installing user plugins". This is the plugins in /boot/config/plugins.

Then later you get a line that says "Starting go script". This is, of course, the go script.

 

Before any of this, unRAID installs any packages in /boot/extra.

 

Well now THAT is damn weird.  I can't argue with the log, but I just know that people that were having trouble with SNAP not making their disk available in time for another plugin solved their issue by mounting via GO.  With SNAP it was a question of alphabetical name order being a toss-up, but with what you're saying it should always fail if using GO to mount.  If my appdisk was not available before Plex then Plex would fail, and it has never failed. [shrug]

 

I'm just holding out for Tom to make my guide obsolete by providing a webgui function to mount non-array disks for use with VMs and do it very early in the boot process for obvious reasons..

[jumperalex]

I use my cache drive purely for apps.  128gb SSD which is hardly utilized.  I want to start preparing for a dip into the VM pool.

 

If I follow these instructions I assume I will have a drive mounted in my system, outside of th unRAID array, that can still db accessed by my cron jobs for copying, run unRAID plugins and be able to be used as the VM repository and disk allocation ?

 

The only real change would be the /mnt/cache to /mnt/whatevericallit mount name ?

 

That is exactly what I'm doing.  My SSD is mounted in unRaid as /mnt/appdisk. Inside that I have /mnt/appdisk/vm/arch which holds my arch.img, data.img and other associated vm files.  It also had (until I migrated Plex to my archVM) my unraid Plex library folder /mnt/appdisk/plex.  The whole disk is shared by unRaid via samba as "appshare" so remote machines can see it.  That was useful for getting into my plex logs from my PC without having to go into an ssh session to access /mnt/appdisk/plex...

 

I have an unRaid cronjob setup to rsync from /mnt/appdisk/vm/arch to /mnt/user/backup/vm/arch.  You have to do it from unRaid because the VM gets paused to make the copy.

[trurl]

...I do have to ask though, are you sure GO is the last thing executed?...

Just thought I would comment on this question. If you look in syslog you can actually see this.

 

First you get a line that says "Installing system plugins". This is the plugins in /boot/plugins.

Then later you get a line that says "Installing user plugins". This is the plugins in /boot/config/plugins.

Then later you get a line that says "Starting go script". This is, of course, the go script.

 

Before any of this, unRAID installs any packages in /boot/extra.

 

Well now THAT is damn weird.  I can't argue with the log, but I just know that people that were having trouble with SNAP not making their disk available in time for another plugin solved their issue by mounting via GO.  With SNAP it was a question of alphabetical name order being a toss-up, but with what you're saying it should always fail if using GO to mount.  If my appdisk was not available before Plex then Plex would fail, and it has never failed. [shrug]

 

I'm just holding out for Tom to make my guide obsolete by providing a webgui function to mount non-array disks for use with VMs and do it very early in the boot process for obvious reasons..

There are some fuzzy areas about this for me as well. I seem to recall the cache disk being preferred over any array disk for plugin installations. I had the impression that cache might be available at times when the array was not, but I don't know where I get these ideas.

[jumperalex]

Well I think the idea of cache over array was more about not keeping two drives spinning.  Especially with plex which is constantly writing to its log file.  I know that is what started me down the path of SSD and non-array mounting

 

And also a little about performance since even reads aren't bare-drive fast and of course writes really aren't.

[Nezil]

Yes indeed I can copy to folders just not to the root of the drive.  Since I normally have folders is why I didn't much care.

 

Sent from my Nexus 7 using Tapatalk

Yeah I hear you with attempting to figure out all the VM stuff! I re-read Nezil posts and it sounds like I may be able to create folders as long as I am not in the root of my mounted drive. I'll have to mess around with this tomorrow though.

 

The reason that you're unable to create folders in the root of the share, is because the ownership of the folder on the unRAID system is probably root:root, with default permissions of drwxr-xr-x:

 

I added the following command to my plugin to solve this problem:

[pre]chgrp users /apps

chmod g+w /apps  [/pre]This changes the ownership to root:users, and makes the directory writable for members of the group users. The user that connects to the SMB share is a member of the users group, and is therefore able to write to the root of the share.

 

By the way, the commands should be issued, I think, after the drive is mounted, rather than before. This might not be necessary, but that's how I did it.

[dalben]

That is exactly what I'm doing.  My SSD is mounted in unRaid as /mnt/appdisk. Inside that I have /mnt/appdisk/vm/arch which holds my arch.img, data.img and other associated vm files.  It also had (until I migrated Plex to my archVM) my unraid Plex library folder /mnt/appdisk/plex.  The whole disk is shared by unRaid via samba as "appshare" so remote machines can see it.  That was useful for getting into my plex logs from my PC without having to go into an ssh session to access /mnt/appdisk/plex...

 

Cool, thanks.  I'll give this a shot when I get some time.  I like the idea.

[SSD]

... For my drive I choose 1024 because I know it works and that value should work for just about any drive.  If you are unsure, you will have to do your own research to find the correct location for the start of the partition.

 

root@Tower:~# fdisk -cu /dev/sdg

 

Command (m for help): n

Command action

  e  extended

  p  primary partition (1-4) p

 

Partition number (1-4): 1

 

First sector (1-117231407, default 117230592): 1024

 

Thanks jumperalex for posting these instructions!!!

 

I am following and just curious about the choice of starting sector. I did a bit of research and found that it is recommended to align SSDs (and other disks too) at 1M boundary, which would mean that 2048 would be a good starting sector number. Also saw it recommended to leave 10-25% of the space unpartitioned to improve longevity. I have a Samsung 840 EVO 250G and thinking to partition it so 225G instead of 250G.

 

HERE IS a link that explains why 2048 is a good number.

 

Thoughts anyone?

[eroz]

If you have any existing partitions first delete them using 'd' and then 'w' to write the partition and exit.  Reissue fdisk -cu /dev/sdX and use 'n' to create a new partition.    You'll be presented with a series of questions which I've shown below with recommended answers.  For my drive I choose 1024 because I know it works and that value should work for just about any drive.  If you are unsure, you will have to do your own research to find the correct location for the start of the partition.

 

root@Tower:~# fdisk -cu /dev/sdg

 

Command (m for help): n

Command action

  e  extended

  p  primary partition (1-4) p

 

Partition number (1-4): 1

 

First sector (1-117231407, default 117230592): 1024

 

Thanks jumperalex for posting these instructions!!!

 

I am following and just curious about the choice of starting sector. I did a bit of research and found that it is recommended to align SSDs (and other disks too) at 1M boundary, which would mean that 2048 would be a good starting sector number. Also saw it recommended to leave 10-25% of the space unpartitioned to improve longevity. I have a Samsung 840 EVO and thinking to partion it so 225G instead of 250G.

 

HERE IS a link that explains why 2048 is a good number.

 

Thoughts anyone?

When I formatted my Samsung 840EVO 500GB,  2048 was the lowest option I had so that is what I chose.

[jumperalex]

... For my drive I choose 1024 because I know it works and that value should work for just about any drive.  If you are unsure, you will have to do your own research to find the correct location for the start of the partition.

 

root@Tower:~# fdisk -cu /dev/sdg

 

Command (m for help): n

Command action

  e  extended

  p  primary partition (1-4) p

 

Partition number (1-4): 1

 

First sector (1-117231407, default 117230592): 1024

 

Thanks jumperalex for posting these instructions!!!

 

I am following and just curious about the choice of starting sector. I did a bit of research and found that it is recommended to align SSDs (and other disks too) at 1M boundary, which would mean that 2048 would be a good starting sector number. Also saw it recommended to leave 10-25% of the space unpartitioned to improve longevity. I have a Samsung 840 EVO 250G and thinking to partition it so 225G instead of 250G.

 

HERE IS a link that explains why 2048 is a good number.

 

Thoughts anyone?

This was my guide http://williamscott.me/blog/2010/12/ssd-tweaks-linux-align-partitions/  2048 isn't wrong and might be better for newer drives also using large block size.  But in either case, that is why i said do some research for your own needs   I do know alignment advice gets more complicated with RAID array's and iirc 2048 as well as some other settings are used to optimize RAID stripes etc.