SNAP versus Cache drive

[joshpond]

Hi all,

 

Just trying to get my unraid box setup and have a quick question.

 

I have a mythtv box and I'd like to save all my recordings to the server box but I don't need the parity backup as I usually watch the tv show and then delete. (So a faster read/write would help).

 

Am I best using SNAP to mount an internal SATA drive for all the recordings or put them all in the cache drive and set it to not transfer?

 

Thanks Josh

[queeg]

Hi all,

 

Just trying to get my unraid box setup and have a quick question.

 

I have a mythtv box and I'd like to save all my recordings to the server box but I don't need the parity backup as I usually watch the tv show and then delete. (So a faster read/write would help).

 

Am I best using SNAP to mount an internal SATA drive for all the recordings or put them all in the cache drive and set it to not transfer?

 

Thanks Josh

 

If you have a cache drive set to not transfer then even stuff you want to send to the array will linger on the cache drive.  Probably not what you want.

Using SNAP means the drive could be hot swapped if you had some need of doing that.  Does a cache drive remain visible when the array is stopped?

[graywolf]

How about a drive that is outside of the array but mounted?

Would probably also need to SAMBA or NTFS it?

[queeg]

How about a drive that is outside of the array but mounted?

Would probably also need to SAMBA or NTFS it?

 

That's what SNAP does.

[joshpond]

If there is no big speed difference, I think SNAP may be the way to go. I'd just like it to be on the server as that is always on. (I can get the mythtv pc to turn on and record as needed.)

 

If there are any other sugestions, I'm all ears.

 

Thanks Josh

[Joe L.]

You may be over-thinking this.  As long as the unRAID server can be written to at a rate fast enough to keep up with the mythtv box, then it does not matter where you store it.  I'd personally just write to the protected array.  Even with the slower 5400 RPM drives it will still probably get 20 to 25 MB/s write speed.

 

If you want to write to a user-share backed by the "cache" drive because it will only spin up one drive rather than two, fine.  Unless you change the default schedule they will not be "moved" to the protected array from the cache drive until the middle of the night (if they are still there and you've not deleted them).  You'll still get to them via the same user-share either way, using the identical path.  Having the cache drive defined and enabled on a user-share is completely transparent to your file-explorer.

 

Joe L.

[nia]

You may be over-thinking this.   As long as the unRAID server can be written to at a rate fast enough to keep up with the mythtv box, then it does not matter where you store it.   I'd personally just write to the protected array. 

Among other uses, I also use my unRAID for the purpose joshpond wants (just using GBPVR).

 

I completely agree with Joe L. I see no real reason not to simply have the files on a disk in the array. The copy process may be  slower than it would be to an unprotected drive, but to me it seems irrelevant if the copy of a show for later viewing takes 1 or 2 minutes.

 

If you are actually writing directly to the server while recording, it is also more than quick enough even for HD recordings (assuming you are on a 1Gb-network). Not knowing the current state of MythTV, I know other PVR's have had issues with writing directly to shares. Maybe something to be aware of? YMMV.

[queeg]

One thing to remember when storing data in the array is it takes as long to delete it as it does to write it.  I'm using some non-array disks for video editing.  When I get to a point with some files that I want to keep them I move them into the array. 

[Joe L.]

One thing to remember when storing data in the array is it takes as long to delete it as it does to write it. 

Not exactly true, but it might seem that way.  It is partly a function of the reiserfs, as it re-balances the file-tree when the files are deleted.  Some other file-system types don't do that, and therefore are faster when deleting files.

[queeg]

One thing to remember when storing data in the array is it takes as long to delete it as it does to write it. 

Not exactly true, but it might seem that way.  It is partly a function of the reiserfs, as it re-balances the file-tree when the files are deleted.   Some other file-system types don't do that, and therefore are faster when deleting files.

 

If a file is written to an array disk, the parity disk is also written to.  When the file is deleted, it has to be deleted from the parity disk.  It takes as long.

[BRiT]

No. It does not take as long as writing the entire file the first time. Deleting a file only updates a few inode pointers. It marks the inodes as freee but does not overwrite the data blocks. It does not rewrite the data sectors. As Joe L mentioned, reiserfs does some additional filesystem housekeeping on the directory blocks when deleting files.

[queeg]

One thing to remember when storing data in the array is it takes as long to delete it as it does to write it. 

Not exactly true, but it might seem that way.  It is partly a function of the reiserfs, as it re-balances the file-tree when the files are deleted.   Some other file-system types don't do that, and therefore are faster when deleting files.

 

If a file is written to an array disk, the parity disk is also written to.  When the file is deleted, it has to be deleted from the parity disk.  It takes as long.

 

Ok, I just tested it and Joe is right.  I don't know how I came to believe that deletes took a long time but I stand corrected.  My bad.

[graywolf]

No. It does not take as long as writing the entire file the first time. Deleting a file only updates a few inode pointers. It marks the inodes as freee but does not overwrite the data blocks. It does not rewrite the data sectors. As Joe L mentioned, reiserfs does some additional filesystem housekeeping on the directory blocks when deleting files.

 

Although I did work one place where Deletes were Wipes that did overwrite the data blocks. Always a PITA.

 

 

[joshpond]

Thanks guys,

 

Probably right about overthinking it but I had fun thinking about it. I have a smaller 500GB hdd so I might just set it up as a SNAP drive or just add it to the array.

 

Thanks Josh

[Joe L.]

One thing to remember when storing data in the array is it takes as long to delete it as it does to write it.

Not exactly true, but it might seem that way.  It is partly a function of the reiserfs, as it re-balances the file-tree when the files are deleted.   Some other file-system types don't do that, and therefore are faster when deleting files.

 

If a file is written to an array disk, the parity disk is also written to.  When the file is deleted, it has to be deleted from the parity disk.  It takes as long.

You are making the assumption that all the blocks "written" when creating the file are again "written" when it is deleted.

 

That is not true. (unless you are on a highly secure government server where any file deleted is completely overwritten with random data first.  I've only read about them, never worked on one, but they are out there, I'm sure)

 

All that is written when the file is deleted are the directory entries and the "housekeeping" bits used to track the blocks used in the file-system.

 

You can prove it to yourself.

 

Clear the statistics on your server.   Copy a large file to another disk or to a new name on an existing disk.   Refresh the browser window and take note of how make "writes" are made to the disk.

 

Then, delete the file you just created.  Again, take note of how many "writes" occur.

 

I just did it here.   I copied 10 DVD ISO images from one disk to another.  (roughly 50 Gig worth of files)   The copy occurred at about 32 MB/s.  (5 I/O operations per block.  Read source file, target data block, target parity block. Then write target data block and parity block.)

 

The statistics on the unRAID console said there were 125463 "writes"   It took 26 minutes, 11 seconds.  

 

I then deleted those same 10 ISO images from the disk were I had copied them.  It took 13.68 seconds and the management interface showed it only took 469 writes to the disk to perform the deletion.

 

It takes nowhere near as long to delete files as it does to create them.

 

Joe L.

[queeg]

 

You are making the assumption that all the blocks "written" when creating the fine are again "written" when it is deleted.

 

That is not true. (unless you are on a highly secure government server where any file deleted is completely overwritten with random data first).  I've only read about them, never worked on one, but they are out there, I'm sure)

 

Joe L.

 

Yes, I was thinking that and now I know I was mistaken.  Thanks Joe. 

[chanders]

Joe, are you always so thorough? 

[Joe L.]

Joe, are you always so thorough? 

No.