New Application: S. N. A. P.

queeg · April 27, 2010

root@TowerDemo:/boot/config# /boot/config/snap/snap.sh -m MyTemp

Mounting filesystem for /dev/disk/by-id/ata-Maxtor_6Y160P0_Y42K7MWE-part1 not attempted: Filesystem (silicon_medley_raid_member) is not supported.

"Filesystem (silicon_medley_raid_member) is not supported."

I was using this as my cache drive before then I unassigned to try the SNAP. Do I need to format the drive before mounting it?

thanks,

~joy

Yes, the disk has to have at least one partition with a supported filesystem on it. I use unMenu to create a filesystem on my drives because it's easy. If you don't have unMenu installed then it's possible to format the drive from a telnet session. SNAP can't format or create filesystems. Let me know if you need help with that.

I'm not familiar with whatever silicon_medley_raid_member is. Doesn't the drive have to be formatted to use it as a cache drive?

jouyang · April 27, 2010

I got no idea but I am doing a preclear on that disk right now.

thanks,

~joy

queeg · April 27, 2010

... Also, I think you should include step 5 to your steps because it took me, a linux noob a while to figure out how to run it.

Thanks for the suggestion. I've made some changes to the first post including the install instructions. Are the directions more useful now?

jouyang · April 28, 2010

Strange, I did a preclear but still can't mount it. Do I need to pull the drive out and format it with Windows to NTFS?

root@TowerDemo:/boot/config/snap# snap.sh -m MyTemp

File system is unknown, cannot be mounted for device: /dev/disk/by-id/ata-Maxtor_6Y160P0_Y42K7MWE-part1

root@TowerDemo:/boot/config/snap# cd /mnt

root@TowerDemo:/mnt# ls

disk1/ hdd1/

Is there something wrong with the script because it's complaining line 710,712 and 722 for "No such file or directory"

root@TowerDemo:/boot/config/snap# snap.sh -a Y42K7MWE MyTemp

root@TowerDemo:/boot/config/snap# snap.sh -b

User nobody password set to none.

./snap.sh: line 710: ./snap.sh: No such file or directory

./snap.sh: line 712: ./snap.sh: No such file or directory

./snap.sh: All shares mounted.

./snap.sh: line 722: ./snap.sh: No such file or directory

Yes, the Note section after step4 is alot better. :-)

thanks,

~joy

queeg · April 28, 2010

I don't think the preclear formats the partition. I think it just tests the drive by writing various bit patterns.

There are several supported filesystems. I'm using reiserfs and ntfs. If the drive in question is not going to be used from Windows then I'd format it with reiserfs. You can do that manually if you want.

From your post the other day I saw this:

unRAID=No SNAP=Yes 3710 /dev/hdd /dev/disk/by-id/ata-Maxtor_6Y160P0_Y42K7MWE

Linux seems to make all sorts of device names for the same hard drive. You can use any of them to format the drive. The line above says that /dev/hdd is the same as /dev/disk/by-id/ata-Maxtor_6Y160P0_Y42K7MWE. I use the shorter name only because it's easier to type in. The following says to make a reiserfs on the first partition of the drive.

mkreiserfs -q /dev/hdd1

or

mkreiserfs -q /dev/disk/by-id/ata-Maxtor_6Y160P0_Y42K7MWE-part1

Please note: If you install, remove or even hotplug a drive then it's possible the /dev/hd[driveletter] can change. Those are assigned like hda, hdb... or sda,sdb... in order of how slackware discovers them at boot time. However, the /dev/disk/by-id/ata-Maxtor_6Y160P0_Y42K7MWE will not vary which is why SNAP uses that device name.

Also: Running mkreiserfs should only takes a ferw minutes.

queeg · April 28, 2010

eSATA! eSata! eSata!

I've finally gotten around to testing esata hotplugging. It's working exactly the same as usb hotplugging except that the device id looks like a standard internal sata drive name.

This is the dock I'm using: http://www.newegg.com/Product/Product.aspx?Item=N82E16817707170&cm_re=dual_sata_dock-_-17-707-170-_-Product

Currently I'm connecting the dock to my internal pci express card: http://www.monoprice.com/products/product.asp?c_id=104&cp_id=10407&cs_id=1040702&p_id=2530&seq=1&format=2

I'm using one of those esata to sata adapters (no electronics).

What I've tested so far:

1) connecting the dock to the unRAID server while the unRAID server was powered down. Then, while the unRAID server is booted up, I pushed the ON button for disk1 in the dock. It seems to take longer for slackware to detect the docked esata drive than it takes for a USB hotplugged drive. It took about 30 seconds in all for the device to be detected and SNAP to share it out. Not too bad.

2) While the drive was mounted and shared, I pulled the drive out of the dock. In about 20 seconds to see the drive unmount.

3.) While the dock turned on and button for drive1 pushed ON, I pushed the drive into the dock. In this case it took less than 10 seconds before the drive was mounted and shared by SNAP.

Other notes: This particular drive I was testing with has ntfs filesystem.

Here's an example of what my /dev/disk/by-id/* directory looks like. 6RY6PEEY is the serial number of my dock'd sata drive.

ata-ST31500341AS_9VS3057K@

ata-ST31500341AS_9VS3057K-part1@

ata-ST31500341AS_9VS33P6J@

ata-ST31500341AS_9VS33P6J-part1@

ata-ST3250410AS_6RY6PEEY@

ata-ST3250410AS_6RY6PEEY-part1@

ata-ST3500320AS_5QM01S58@

ata-ST3500320AS_5QM01S58-part1@

ata-ST3500320AS_9QM26B35@

ata-ST3500320AS_9QM26B35-part1@

scsi-SATA_ST31500341AS_9VS3057K@

scsi-SATA_ST31500341AS_9VS3057K-part1@

scsi-SATA_ST31500341AS_9VS33P6J@

scsi-SATA_ST31500341AS_9VS33P6J-part1@

scsi-SATA_ST3250410AS_6RY6PEEY@

scsi-SATA_ST3250410AS_6RY6PEEY-part1@

scsi-SATA_ST3500320AS_5QM01S58@

scsi-SATA_ST3500320AS_5QM01S58-part1@

scsi-SATA_ST3500320AS_9QM26B35@

scsi-SATA_ST3500320AS_9QM26B35-part1@

usb-Lexar_JD_FireFly_JMSH6RPBKDR7NYYFGIMM-0:0@

usb-Lexar_JD_FireFly_JMSH6RPBKDR7NYYFGIMM-0:0-part1@

This thread shows what esata cards I'm using and what slots they work in.

http://lime-technology.com/forum/index.php?topic=6157.msg60239#msg60239

jouyang · April 30, 2010

Queeg,

I just want to say thank you. I finally got it working. :-)

Keep up the good work.

thanks,

~joy

queeg · May 6, 2010

A new version of SNAP is available. Version 0.4 See details and downloads in the first post of this thread.

New feature: Automatically transfer pictures from your Digital Camera to unRAID.

queeg · May 7, 2010

Updated version 0.46 with bug fix. Download on the first page in this topic.

RobJ · June 1, 2010

I've added SNAP to the UnRAID Add Ons page, direct link is SNAP. Please review and revise as necessary. I'm sure it can be improved. I would welcome your ongoing maintenance of the SNAP section as well.

By the way, I welcome any users who would like to contribute to the unRAID Wiki, to help maintain the UnRAID Add Ons page, needs a LOT of work. Please review the Start Contributing wiki page first.

gfjardim · June 5, 2010

How to know if share event is auto running the right script?

queeg · June 5, 2010

How to know if share event is auto running the right script?

Here is an excerpt from my hotPlugAfterShareEvent.sh event script. In this case, the script I'm running for this event with this particular device is the copy command to copy pictures from my camera to the destination directory. Since I'm not running a script, I can infer that the copy command is being run because the entries in snapEvents.sh and I have set it to beep so I hear it executing the code. However, if I wanted to have it run silently I could simply echo some information into the snapEvents.log file - see below for example. The blue text shows how to log the information and what it would look like in the log file.

----Excerpt from snapEvents.log file-----

/boot/config/snap/snapEvents.sh -e HOTPLUG_IN -d usb-OLYMPUS_C770UZ_000369020971-0:0 -s 000369020971-0:0 -h OlympusC770

/boot/config/snap/snapEvents.sh -e AFTER_MOUNT -p /dev/disk/by-id/usb-OLYMPUS_C770UZ_000369020971-0:0-part1 -m /mnt/disk/OlympusC770 -s 000369020971-0:0 -h OlympusC770

/boot/config/snap/snapEvents.sh -e AFTER_SHARE -s 000369020971-0:0 -h OlympusC770 -m /mnt/disk/OlympusC770 -d usb-OLYMPUS_C770UZ_000369020971-0:0

Running the copy (my script) for event: AFTER_SHARE, on sharename: OlympusC770

----

----excerpt from hotPlugAfterShareEvent.sh----

echo "Running hotPlugAfterShareEvent.sh"

case $SHARENAME in

'OlympusC770')

beep -l 100 -f 2000 -n -l 150 -f 3000

destination="/mnt/user/Public/OlympusC770"

if [ ! -d $destination ]

then

mkdir $destination

fi

# Add this to log the script actions to the snapEvent.log file

echo "Running the copy (my script) for event: $EVENT, on sharename: $SHARENAME" >> $SNAPHOME/snapEvent.log

cp -u -r $MOUNTPOINT/DCIM/100OLYMP/* $destination 2>&1 >> $SNAPHOME/hotPlugAfterShareEvent.log

chmod 666 $destination/*

beep -l 100 -f 2000 -n -l 150 -f 3000

beep -r 5 -l 100 -f 2000

;;

----

gfjardim · June 13, 2010

I've modded my hotPlugAfterShareEvent.sh script to automatic backup some of my files on a specific disk hot-plug event. Note that the disk is mounted to Backup_Disk share.

case $SHARENAME in
'Backup_Disk')
SOURCE=/mnt/user/
DEST=$MOUNTPOINT
beep  -l 100 -f 2000 -n -l 150 -f 3000
beep  -l 100 -f 2000 -n -l 150 -f 3000	
logger The backup job is starting.
rsync  -aP --delete --exclude ".*/"  \
		--exclude "Videos"  \
		$SOURCE/ $DEST/Current/
cd $DEST/Current
mkdir ../`date +%m-%d-%Y`
cp -al * $DEST/`date +%m-%d-%Y`/
logger The backup job has succeeded. 
sync
fuser -k $DEST
cd $SNAPHOME 
./snap.sh -M Backup_Disk
sleep 5
DEVICE=$(readlink /dev/disk/by-id/$DEVICEID | cut -d / -f 3)
echo 1 > /sys/block/$DEVICE/device/delete
sleep 2
echo "$0 - Device /dev/$DEVICE has been successfully removed." >> $SNAPHOME/snapEvents.log
logger Backup disk can be safely removed.
beep  -l 100 -f 2000 -n -l 150 -f 3000
beep  -l 100 -f 2000 -n -l 150 -f 3000
beep  -r 5 -l 100 -f 2000
;;
esac

Rsync is configured to skip all hidden folders and Videos folder under /mnt/user directory.

The disk is connected, there are beeps on script start, rsync is used to create the backup, "cp" is used to create hard links, SNAP is used to umount and unshare the disk, and then the disk node is deleted, making unplugging the drive safe. Finally, there are more beeps. The syslog show the following messages:

Jun 13 18:46:37 Server kernel: scsi 5:0:0:0: Direct-Access ATA WDC WD5000AAKS-0 01.0 PQ: 0 ANSI: 5
Jun 13 18:46:37 Server kernel: [232]: scst_suspend_activity:599:suspend_count 0

Jun 13 18:46:37 Server kernel: [232]: scst_susp_wait:578:wait_event() returned 0

Jun 13 18:46:37 Server kernel: [232]: scst_suspend_activity:644:Waiting for 0 active commands finally to complete

Jun 13 18:46:37 Server kernel: [232]: scst_susp_wait:578:wait_event() returned 0

Jun 13 18:46:37 Server kernel: [232]: __scst_resume_activity:675:suspend_count 0 left

Jun 13 18:46:37 Server kernel: [232]: scst: scst_register_device:792:Attached to scsi5, channel 0, id 0, lun 0, type 0

Jun 13 18:46:37 Server kernel: sd 5:0:0:0: [sdk] 976771055 512-byte logical blocks: (500 GB/465 GiB)

Jun 13 18:46:37 Server kernel: sd 5:0:0:0: [sdk] Write Protect is off

Jun 13 18:46:37 Server kernel: sd 5:0:0:0: [sdk] Mode Sense: 00 3a 00 00

Jun 13 18:46:37 Server kernel: sd 5:0:0:0: [sdk] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

Jun 13 18:46:37 Server kernel: sdk: sdk1

Jun 13 18:46:37 Server kernel: sd 5:0:0:0: [sdk] Attached SCSI disk

Jun 13 18:46:42 Server ntfs-3g[3541]: Version 2010.3.6 integrated FUSE 27

Jun 13 18:46:42 Server ntfs-3g[3541]: Mounted /dev/sdk1 (Read-Write, label "", NTFS 3.1)

Jun 13 18:46:42 Server ntfs-3g[3541]: Cmdline options: rw,umask=111,dmask=000

Jun 13 18:46:42 Server ntfs-3g[3541]: Mount options: rw,silent,allow_other,nonempty,relatime,fsname=/dev/sdk1,blkdev,blksize=4096,default_permissions

Jun 13 18:46:42 Server ntfs-3g[3541]: Global ownership and permissions enforced, configuration type 1

Jun 13 18:46:45 Server logger: The backup job is starting.

Jun 13 18:51:09 Server logger: The backup job has succeeded.

Jun 13 18:51:14 Server ntfs-3g[3541]: Unmounting /dev/sdk1 ()

Jun 13 18:51:19 Server kernel: [17419]: scst_suspend_activity:599:suspend_count 0

Jun 13 18:51:19 Server kernel: [17419]: scst_susp_wait:578:wait_event() returned 0

Jun 13 18:51:19 Server kernel: [17419]: scst_suspend_activity:644:Waiting for 0 active commands finally to complete

Jun 13 18:51:19 Server kernel: [17419]: scst_susp_wait:578:wait_event() returned 0

Jun 13 18:51:19 Server kernel: [17419]: __scst_resume_activity:675:suspend_count 0 left

Jun 13 18:51:19 Server kernel: [17419]: scst: scst_unregister_device:857:Detached from scsi5, channel 0, id 0, lun 0, type 0

Jun 13 18:51:19 Server kernel: sd 5:0:0:0: [sdk] Synchronizing SCSI cache

Jun 13 18:51:19 Server kernel: sd 5:0:0:0: [sdk] Stopping disk

Jun 13 18:51:19 Server kernel: ata12.00: disabled

Jun 13 18:51:19 Server logger: Backup disk can be safely removed.

Here this code is working wonderfully, but if anyone feels capable of improving it, be my guest.

queeg · June 14, 2010

I've modded my hotPlugAfterShareEvent.sh script to automatic backup some of my files on a specific disk hot-plug event. Note that the disk is mounted to Backup_Disk share.

Here this code is working wonderfully.

Great work! I'm sure this will be helpful to others. Thanks for posting the code.

adspence · August 8, 2010

Struggling with the device setup with SNAP.

Here are my devices :-

./snap.sh -listDeviceConfigurationInfo

unRAID=No SNAP=No 3815 /dev/sdf /dev/disk/by-id/ata-Hitachi_HDS722020ALA330_JK1171YAG7UTSS

unRAID=Yes SNAP=No 3607 /dev/sdc /dev/disk/by-id/ata-SAMSUNG_HD103UJ_S13PJ1DQ402223

unRAID=Yes SNAP=No 3665 /dev/sdd /dev/disk/by-id/ata-SAMSUNG_HD103UJ_S13PJ1DQ402224

unRAID=No SNAP=No 3847 /dev/sdg /dev/disk/by-id/ata-WDC_WD10EACS-00ZJB0_WD-WCASJ0338244

unRAID=No SNAP=No 3466 /dev/sda /dev/disk/by-id/ata-WDC_WD10EAVS-00D7B0_WD-WCAU41934321

unRAID=No SNAP=No 3732 /dev/sde /dev/disk/by-id/ata-WDC_WD2500JB-00GVA0_WD-WCAL73606663

unRAID=No SNAP=No 3815 /dev/sdf /dev/disk/by-id/scsi-SATA_Hitachi_HDS7220_JK1171YAG7UTSS

unRAID=Yes SNAP=No 3607 /dev/sdc /dev/disk/by-id/scsi-SATA_SAMSUNG_HD103UJS13PJ1DQ402223

unRAID=Yes SNAP=No 3665 /dev/sdd /dev/disk/by-id/scsi-SATA_SAMSUNG_HD103UJS13PJ1DQ402224

unRAID=No SNAP=No 3847 /dev/sdg /dev/disk/by-id/scsi-SATA_WDC_WD10EACS-00_WD-WCASJ0338244

unRAID=No SNAP=No 3466 /dev/sda /dev/disk/by-id/scsi-SATA_WDC_WD10EAVS-00_WD-WCAU41934321

unRAID=No SNAP=No 3732 /dev/sde /dev/disk/by-id/scsi-SATA_WDC_WD2500JB-00_WD-WCAL73606663

unRAID=No SNAP=No 3551 /dev/sdb /dev/disk/by-id/usb-Generic-_SD_MMC_20090815198100000-0:0

If I run the following :-

./snap.sh -a JK1171YAG7UTSS DeviceBay1

I get what I expect

/boot/config/snap# ./snap.sh -listDeviceConfigurationInfo

unRAID=No SNAP=Yes 3815 /dev/sdf /dev/disk/by-id/ata-Hitachi_HDS722020ALA330_JK1171YAG7UTSS

unRAID=Yes SNAP=No 3607 /dev/sdc /dev/disk/by-id/ata-SAMSUNG_HD103UJ_S13PJ1DQ402223

unRAID=Yes SNAP=No 3665 /dev/sdd /dev/disk/by-id/ata-SAMSUNG_HD103UJ_S13PJ1DQ402224

unRAID=No SNAP=No 3847 /dev/sdg /dev/disk/by-id/ata-WDC_WD10EACS-00ZJB0_WD-WCASJ0338244

unRAID=No SNAP=No 3466 /dev/sda /dev/disk/by-id/ata-WDC_WD10EAVS-00D7B0_WD-WCAU41934321

unRAID=No SNAP=No 3732 /dev/sde /dev/disk/by-id/ata-WDC_WD2500JB-00GVA0_WD-WCAL73606663

unRAID=No SNAP=Yes 3815 /dev/sdf /dev/disk/by-id/scsi-SATA_Hitachi_HDS7220_JK1171YAG7UTSS

unRAID=Yes SNAP=No 3607 /dev/sdc /dev/disk/by-id/scsi-SATA_SAMSUNG_HD103UJS13PJ1DQ402223

unRAID=Yes SNAP=No 3665 /dev/sdd /dev/disk/by-id/scsi-SATA_SAMSUNG_HD103UJS13PJ1DQ402224

unRAID=No SNAP=No 3847 /dev/sdg /dev/disk/by-id/scsi-SATA_WDC_WD10EACS-00_WD-WCASJ0338244

unRAID=No SNAP=No 3466 /dev/sda /dev/disk/by-id/scsi-SATA_WDC_WD10EAVS-00_WD-WCAU41934321

unRAID=No SNAP=No 3732 /dev/sde /dev/disk/by-id/scsi-SATA_WDC_WD2500JB-00_WD-WCAL73606663

unRAID=No SNAP=No 3551 /dev/sdb /dev/disk/by-id/usb-Generic-_SD_MMC_20090815198100000-0:0

But if I then also try to add another :-

/boot/config/snap# ./snap.sh -a WCASJ0338244 DriveBay2

Then it is added to snap.list

/boot/config/snap# more snap.list

JK1171YAG7UTSS=DriveBay1

WCASJ0338244=DriveBay2

But isn't visible :-

/boot/config/snap# ./snap.sh -listDeviceConfigurationInfo

unRAID=No SNAP=Yes 3815 /dev/sdf /dev/disk/by-id/ata-Hitachi_HDS722020ALA330_JK1171YAG7UTSS