January 21, 200917 yr OK here is a shorter version of the question. If I had a drive assigned as disk7 and then pulled that drive out, reformatted it NTFS, and then reinserted it back in the unraid array and then started the array, is unraid supposed to rewrite zeros to the disk and then format it? Or does this happen during mounting without showing any notification? Because this is what I did, and the drive just shows up as disk7 and appears to be writing and reading data normally. Assuming parity is rebuilt and I am able to copy data and serve it from disk7, is this safe? Was unraid supposed to format the drive when it mounted it to disk7 or does it just do it automatically? By the way, thanks very much for the help. Unraid is very cool if I can keep it running in stable fashion.
January 22, 200917 yr bump, since I simplified the question To answer your question we need a bit more detail. Log in via telnet and type: mount Let us see the output. As far as I know, unRAID will not accept an NTFS formatted drive into the protected array without clearing it and re-formatting it as a reiser file-system. I'm not sure if anybody has ever did what you have done, though, as adding it, then reformatting it, then putting it back. I can tell you this, the NTFS driver supplied with unraid is a read-only driver. It has no ability to create files or append to files. (or rather, those abilities are *very* brain-damaged) The ntfs-3g driver is fully read/write capable, but it is not part of unRAID by default.. We'll know more when we see the output of the "mount" command. You might also attach a copy of your syslog. Instructions in the wiki under troubleshooting.
January 22, 200917 yr At a base level, unRAID probably doesn't much care what is on the disk. It is monitoring low level I/O to the disk. But the emhttp interface wouldn't understand and would think the disk was unformatted. I know that unRAID only supports ReiserFS, so I expect that is a gotcha in there somewhere with NTFS. Just not sure where.
January 22, 200917 yr At a base level, unRAID probably doesn't much care what is on the disk. It is monitoring low level I/O to the disk. Correct... It just knows bits. The "md" driver itself could care less about the content, or file-system, or even if there is a file-system at all. But the emhttp interface wouldn't understand and would think the disk was unformatted. Perhaps... but there is an outside chance it might sneak through the first of the tests. If the mbr matches our format, return 0. * If the mbr matches our format except that partition 1 type is 0, return 1. This indicates * that the disk has been factory-cleared. * If the mbr differs by more than the partition 1 type, return 2. This just indicates that * the disk is "foreign". Tom's basic logic looks at the partition type byte, and the partition start and end relative to the drive's geometry. If jji666's reformatting did not change any of the bytes from when it was formatted as reiserfs, and it left the partition type as 83, then it might sneak past the test for a foreign file-system type. He compares the entire first 512 bytes on the disk with the expected structure he would put there based on the drive geometry and size. I had to deal with all that when I wrote the pre-clear script. Even 1 byte wrong and the drive was not considered pre-cleared. In the same way, 1 byte wrong and it would not be "properly partitioned" either. However, when it went to mount the file-system, I would expect emhttp to fail to mount it as a reiserfs, and as you said, show it as "unformatted" I know that unRAID only supports ReiserFS, so I expect that is a gotcha in there somewhere with NTFS. Just not sure where. Me neither... We'll have to see the output of his mount command. If it says ntfs, and he is writing to the drive without using the ntfs-3g driver, odds are it is really going to be a mess. Joe L.
January 23, 200917 yr Author All- Thanks for your responses with this. Since the problem I was having with my unraid box (independent of the drive format issue) continued, I actually reformatted my usb key and rebuilt the array. So I cannot use a mount command now and this question will have to (and hopefully will) remain theoretical for a while. Thanks again! Unraid is great, I just hope I can keep my boxes stable!
January 26, 200917 yr I hope I'm not thread-jacking as I believe I can add some possible information from a very similar situation for future reference. Originally I attached Disk2 and it came up as unformatted (was originally the parity of another array). In my case I hit the restore button and brought up the array then it game be the little format button which I hit. As I had not added any information to this drive I brought it back to a Windows box to shuffle some files, while reformatting it as NTFS. After bringing it back to the unRAID box, it mounted as Disk2 just fine. Since unRAID had prior knowledge of the device ID, I believe that has some relevance. Unfortunately I don't have a free drive to setup parity quite yet to see if it will go that far. I couldn't see anything remotely related to NTFS in ps. Write performance is a little low compared to reiser w/o parity ~ 30MB/sec although read is fine ~70MB/sec over the SMB shares (hdparm on /dev/sdb reports 81.27MB/sec). Processor utilization is really low compared to NTFS-3G which floors my atom and can only write ~ 15MB/sec without parity once again. I took the drive out and tested it back in the windows box after witting in unraid and windows doesn't see it as an NTFS drive, shows up as being formatted in a foreign OS. The output from mount is as follows: fusectl on /sys/fs/fuse/connections type fusectl (rw) usbfs on /proc/bus/usb type usbfs (rw) /dev/sda1 on /boot type vfat (rw,umask=077,shortname=mixed) /dev/md1 on /mnt/disk1 type reiserfs (rw,noatime,nodiratime) /dev/md2 on /mnt/disk2 type reiserfs (rw,noatime,nodiratime) As far as I can tell, mount thinks md2 is reiserfs. I'm guessing the unRAID driver reports that regardless of what is actually on the drive. DF reveals: Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 1962224 41196 1921028 3% /boot /dev/md1 732552188 680175008 52377180 93% /mnt/disk1 /dev/md2 732552188 1118412 731433776 1% /mnt/disk2 Sorry I cannot add more diagnostics as I'm not really that savvy in this department. If there is additional tests I can perform I can probably repeat the process on another drive. For the meantime I need to move data to this drive and wiill be formatting this one as Reiser and proceeding to mount it normally. Editted for early-morning inaccuracies.
January 26, 200917 yr I hope I'm not thread-jacking as I believe I can add some possible information from a very similar situation for future reference. I don't think you are thread-jacking. I think you have both uncovered a subtle bug in detecting the state of an existing disk's file-system. Originally I attached Disk2 and it came up as unformatted (was originally the parity of another array). In my case I hit the restore button and brought up the array then it game be the little format button which I hit. As I had not added any information to this drive I brought it back to a Windows box to shuffle some files, while reformatting it as NTFS. After bringing it back to the unRAID box, it mounted as Disk2 just fine. Since unRAID had prior knowledge of the device ID, I believe that has some relevance. Unfortunately I don't have a free drive to setup parity quite yet to see if it will go that far. I couldn't see anything remotely related to NTFS in ps. Write performance is a little low compared to reiser w/o parity ~ 30MB/sec although read is fine ~70MB/sec over the SMB shares (hdparm on /dev/sdb reports 81.27MB/sec). Processor utilization is really low compared to NTFS-3G which floors my atom and can only write ~ 15MB/sec without parity once again. I took the drive out and tested it back in the windows box after witting in unraid and windows doesn't see it as an NTFS drive, shows up as being formatted in a foreign OS. The output from mount is as follows: fusectl on /sys/fs/fuse/connections type fusectl (rw) usbfs on /proc/bus/usb type usbfs (rw) /dev/sda1 on /boot type vfat (rw,umask=077,shortname=mixed) /dev/md1 on /mnt/disk1 type reiserfs (rw,noatime,nodiratime) /dev/md2 on /mnt/disk2 type reiserfs (rw,noatime,nodiratime) As far as I can tell, mount thinks md2 is reiserfs. I'm guessing the unRAID driver reports that regardless of what is actually on the drive. DF reveals: Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 1962224 41196 1921028 3% /boot /dev/md1 732552188 680175008 52377180 93% /mnt/disk1 /dev/md2 732552188 1118412 731433776 1% /mnt/disk2 Sorry I cannot add more diagnostics as I'm not really that savvy in this department. If there is additional tests I can perform I can probably repeat the process on another drive. For the meantime I need to move data to this drive and wiill be formatting this one as Reiser and proceeding to mount it normally. Editted for early-morning inaccuracies. Mount sure thinks it is a reiserfs, and it did mount. I wonder about what you would see if you did a reiserfsck on the disk. unRAId probably should have cleared the disk, but it was "fooled" when the reiserfs "mount" succeeded. I am glad it is not attempting to use the built-in ntfs driver, as that is unable to create files. if you find a moment before you re-format that drive as reiserfs, can you run the following and post the results: dd if=/dev/hdb count=1 | od -x -A d fdisk -l /dev/hdb vol_id /dev/hdb1 I am assuming that disk2 is /dev/hdb, use the correct disk if I am mistaken. The vol_id command needs the partition, so if the drive is /dev/hdc, the correct command would be vol_id /dev/hdc1 (with a 1 on the end of the device name to indicate the first partition) Joe L.
January 26, 200917 yr dd if=/dev/sdb count=1 | od -x -A d yields: dd: opening `dev/sdb': No such file or directory fdisk -l /dev/sdb yields: Disk /dev/sdb: 750.1 GB, 750156374016 bytes 1 heads, 63 sectors/track, 23256336 cylinders Units = cylinders of 63 * 512 = 32256 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sdb1 2 23256336 732574552+ 83 Linux Partition 1 does not end on cylinder boundary. vol_id /dev/sdb1 yields: ID_FS_USAGE=filesystem ID_FS_TYPE=reiserfs ID_FS_VERSION=3.6 ID_FS_UUID=9c846678-7e43-44e2-bc69-0bd896898111 ID_FS_UUID_ENC=9c846678-7e43-44e2-bc69-0bd896898111 ID_FS_LABEL= ID_FS_LABEL_ENC= Whatever driver it uses to mount the drive when it reports NTFS, it can create files although not readable within windows. I believe this thread may be relevant as well: http://lime-technology.com/forum/index.php?topic=3091.0 I was under the impression the disk information was stored within the super.dat file. I renamed it and it came back up without the format button either. Is there any meta information stored on the disk? Edit: formatting. 04:30 is a horrid time to wakeup. Edit2: Somehow I lost some additional information I tried to post. I tried re-assigning it to Slot3, it came up as reiser instead of ntfs. After trying reiserfsck on the drive it showed it had 1 error and running reiserfsck -fix-fixable should fix it. I changed it back to Slot2, it still reported it as reiser. After running reiserfsck -fix-fixable on the drive it reveals: reiserfs_open: the reiserfs superblock cannot be found on /dev/sdb. Failed to open the filesystem. If the partition table has not been changed, and the partition is valid and it really contains a reiserfs partition, then the superblock is corrupted and you need to run this utility with --rebuild-sb.
January 26, 200917 yr dd if=/dev/sdb count=1 | od -x -A d yields: dd: opening `dev/sdb': No such file or directory fdisk -l /dev/sdb yields: Disk /dev/sdb: 750.1 GB, 750156374016 bytes 1 heads, 63 sectors/track, 23256336 cylinders Units = cylinders of 63 * 512 = 32256 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sdb1 2 23256336 732574552+ 83 Linux Partition 1 does not end on cylinder boundary. vol_id /dev/sdb1 yields: ID_FS_USAGE=filesystem ID_FS_TYPE=reiserfs ID_FS_VERSION=3.6 ID_FS_UUID=9c846678-7e43-44e2-bc69-0bd896898111 ID_FS_UUID_ENC=9c846678-7e43-44e2-bc69-0bd896898111 ID_FS_LABEL= ID_FS_LABEL_ENC= Whatever driver it uses to mount the drive when it reports NTFS, it can create files although not readable within windows. I believe this thread may be relevant as well: http://lime-technology.com/forum/index.php?topic=3091.0 I was under the impression the disk information was stored within the super.dat file. I renamed it and it came back up without the format button either. Is there any meta information stored on the disk? Edit: formatting. 04:30 is a horrid time to wakeup. You might have left off the leading "/" on the first "dd" command. dev/sdb would not exist unless you had "cd'd" to "/" first. /dev/sdb would exist. It is very interesting that the vol_id command also thinks it is a reiserfs file-system. Were these before of after you re-formatted the drive as reiserfs? Joe L.
January 26, 200917 yr I haven't reformatted it as reiserfs yet. Sorry, I editted the previous post with additional information. I copy/pasted the command again dd if=/dev/sdb count=1 | od -x -A d (it is very likely I messed up the first time) and it now reveals: 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.000412076 s, 1.2 MB/s 0000000 0000 0000 0000 0000 0000 0000 0000 0000 * 0000448 0000 0083 0000 003f 0000 66b1 5754 0000 0000464 0000 0000 0000 0000 0000 0000 0000 0000 * 0000496 0000 0000 0000 0000 0000 0000 0000 aa55 0000512 For the record, that means nothing to me. As of now, I suspect the contents of the drive differ from when it originally mounted as NTFS, as its no longer reporting that in the web interface. There is still nothing of importance on this drive.
January 26, 200917 yr Re-reading my previous post I realized I forgot the "1" on reiserfsck. Whoops. re-runing... taking some time. I believe this thread is relevant as well: http://lime-technology.com/forum/index.php?topic=3091.0
January 26, 200917 yr I haven't reformatted it as reiserfs yet. Sorry, I editted the previous post with additional information. I copy/pasted the command again dd if=/dev/sdb count=1 | od -x -A d (it is very likely I messed up the first time) and it now reveals: 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.000412076 s, 1.2 MB/s 0000000 0000 0000 0000 0000 0000 0000 0000 0000 * 0000448 0000 0083 0000 003f 0000 66b1 5754 0000 0000464 0000 0000 0000 0000 0000 0000 0000 0000 * 0000496 0000 0000 0000 0000 0000 0000 0000 aa55 0000512 For the record, that means nothing to me. It is OK, it means something to me. It shows the first 512 bytes on your disk. It shows that there is a single partition of type "83" going from sector 63 to the end of the disk. (exactly like the fdisk command shows) It shows a proper clearing of the master boot record area of the disk, and a proper MBR signature "aa55") I completely missed that you did not have a parity drive installed. Without one there is no need for any lengthy clearing of the drive prior to it being included in the array. The actual reiserfs formatting only takes a matter of 20-30 seconds, you may never have seen it if you were not paying close attention. I don't think you have any reason to re-format the drive. You can simply delete the files on it if they are no longer needed. As of now, I suspect the contents of the drive differ from when it originally mounted as NTFS, as its no longer reporting that in the web interface. There is still nothing of importance on this drive. I think your array is working exactly as it is designed. I suspect you missed the reiserfs formatting, but it did occur, especially since the vol_if says reiserfs. The bug, if any, is that the file-system type as shown on your first screen shot was grabbed when the interface first scanned the drives, but not updated when the drive was re-formatted as reiserfs. Until you rebooted, it still "said" ntfs. It was actually re-formatted as a reiserfs once it was added to the array. Joe L.
January 26, 200917 yr Re-reading my previous post I realized I forgot the "1" on reiserfsck. Whoops. re-runing... taking some time. I believe this thread is relevant as well: http://lime-technology.com/forum/index.php?topic=3091.0 Yup, same bug.
January 26, 200917 yr Ok Thanks. I just re-ran reiserfsck and it came through clear. I hope I haven't wasted too much of your time this morning I'm off to work.
Archived
This topic is now archived and is closed to further replies.