Jump to content

dheg

Members
 View Profile  See their activity

  • Posts

    413

  • Joined

  • Last visited

 Content Type 

Posts posted by dheg

    ATLAS My Virtualized unRAID server

    in Virtualizing Unraid

    Posted

    @SidebandSamurai

     

    wow, thanks a lot, I have some more reading to do  :)

    RAM won't be a problem, I have 16 GB and just ordered 2 8GB sticks. My intention is to dedicate at least 8GB, could be more if needed, for 4 1TB HDDs. After Johnm's post I decided to go with this, so 8GB for a 4 TB array should be plenty.

    Napp-it seems easy, Nexenta nicer... maybe I should try  ;)

     

    Have anyone tried this case. The 'Add to Cart' button is calling my name ::)

    Newznab

    in User Customizations

    Posted

    This is a short summary of my findings:

    [*]Update binaries searches usenet for headers.

    [*]Update releases consolidates the headers into releases.

    [*]Post Processing looks up the releases (NFO's) on the internet and fills the information on the web interface.

    By default, newznab works sequentially (assuming you use the screen or init scripts):

    [*]Updating binaries

    [*]Updating releases

    [*]Post processing 100 releases, regardless of how many you actually have

     

    However if you follow these instructions you will make post processing to run continuously, kind of multitasking. It will post process old releases while it's grabbing headers or sorting releases.

     

    Benefits:

    • Post processing will run continuously and will, potentially, post process more releases per day
    • You will not spam Amazon's API, assuming of course you change the postprocess.php file in www/lib to process just one release at a time

     

    However, as the postprocess.php script will also be called during standard operations, there is a chance (don't know how big) that they will clash and get killed, so you'll have to re-start it again.

     

    I hope it's clear  :)!

    ATLAS My Virtualized unRAID server

    in Virtualizing Unraid

    Posted

    I hadn't thought about booting FreeNAS or NAS4Free from the same stick as my ESX. How exactly would that be setup? How would you make the stick a part of the data store?

     

    To boot from a USB you would still need a datastore to define the guest. ESXi will boot the VM that will boot FreeNAS from the USB. Kind of like unRAID. I'm trying a different approach, to boot FreeNAS from a CF card that will be a datastore. I haven't received the parts yet, so it's early to say whether this will work

    ATLAS My Virtualized unRAID server

    in Virtualizing Unraid

    Posted

    my answers in RED

    Since I’ll be using only 4 drives, can I plug the M1015 in one of the PCIE x4 ports without sacrificing performance?

    Yes. you will be fine

    Thanks

     

    I know close to nothing about FreeNAS, but I believe it’s a stripped raid. What kind of writing/reading performance improvement should I expect?

    You have a few ways to set it up. you can do what is equivalent to a raid5 (RAIDZ) or a raid10 (Striped Mirrored Vdev’s). the Striped Mirrored Vdev’s will net you more speed at the cost of 2drives for parity. look back a few pages, SEE LINK Im actually getting better writes then my SATA3 Performance Pro2 and slightly less performance for the reads.

    Thanks for the link to your post, that was the one that first put me on the idea of using FreeNAS, although I couldn't find it. This thread has become HUGE!

    If you would to start again, would you still recommend OpenIndiana? Considering I'm a newbie, FreeNAS seems to have a larger community for support.

     

    I was actually thinking of raid-z2 (which i believe is equivalent to raid6). I may seem to be too paranoid, but right at the beginning of starting using unraid I had two drives failing in just 3 days. So I lost some data. You are never too sure.

    The thing about raid 10 is that if two drives within the same vdev fail,  you are done, right?

    Any other alternatives?

     

    Are SDDs really worth it for FreeNAS, or should I go with spinners? If using SSDs, what kind of speed should I expect? What about spinners? Please feel free to recommend parts  :)!

    Spinners are fine for this. the reason i say that is for space. with 4x 2TB Spinner Green Drives,  you will get around 6TB at 450MB/s read/write in RAIDZ. I would expect more in the 500-600 MB/s range with the new Seagate 7200 RPM 1TB platter drives.

     

    yes with 4x 250MB SSD, You will get 750MB at maybe 1000MB/s read and 700MB/s Write (am totally guessing here, I could be wrong. [you might even maxout the PCIe bus first. although 4x should be ok to handle it]) At a ridiculous cost. 

     

    When you weigh the size of array vs performance VS the cost. the spinners look like more bang for the buck.

    Spinner = 6TB @ 450MB/s for $360 vs SSD = 750MB @ Say 800MB/s (ish) for $750.

    you are fully right, I just needed someone else than my wife to tell me about the cost of SSD drives  ;)

    Besides, once the server is up, I don't really need SSD speeds.

     

    Don't forget, you can put your unRAID cache drive on this array...

    I didn't mention it, but my goal is to use FreeNAS for the cache drive as well as the datastores. I should say that one of my datastores is a newznab server and it uses a lot of space...

     

    When looking around I’ve seen many, reasonably priced, 1U SATA II enclosures for 4 drives. However, when looking for SATA III, prices really go up, especially here. I’m Spanish, so apart from the spelling mistakes (sorry!), my options to shop around aren’t many. If I go with SSDs, going with SATA III is a no brainer, but if using spinners, could I take SATA II?

    Sata2 should be fine for spinners. for SSD, youll want SATA3

    I guess i need to see what the item is you're looking at.

    I’ve found these so far:

    https://ri-vier.eu/1u-server-case-w-4-hotswappable-satasas-drive-bay-rpc1204-p-3.html?cPath=1_3_4

    https://ri-vier.eu/1u-server-case-w-4-hotswappable-satasas-drive-bay-rpc1004-p-1.html?cPath=1_3_4

     

    And then suddenly today I found a SATA III, which is nice thinking ahead:

    http://www.xcase.co.uk/X-Case-RM-140-1u-4-drive-hotswap-case-with-rails-p/case-xcase-140.htm

     

    Anything else I’m missing?

    Yes, if you are making a this a 1U standalone server, you're gigabit network will maxout before your NAS array.

    You will feel the slowness of the gigabit.

    consider making the freenas a guest ON the ESXi server itself. passthough the M1015 to the guest and use that. by using the ESXi VLAN, you can then use the 10GB virtual nics and get full NAS speed.

     

    Also, unless you are going full standalone freeNAS with iSCSI target (or NFS). you will still need 1 datastore drive local on the server.

    I  already thought of that. I'll make the FreeNAS a guest on the ESXi server to use the vmxnet3 lan. I'm thinking of using a CF card as the FreeNAS datastore. I ordered this and I'm waiting for it to arrive to test it

    I want it to be a standalone case so I can use the 24 drives of the norco for unraid, but it'll be daisy chained to the server.

     

    EDIT:

    I looked for some real world benchmarks for SSD's in raid5

    http://www.storagereview.com/intel_ssd_510_raid_review

     

    Pretty sad honestly.. about a 20% performance gain. so my guesses would be about correct with M3's, Performance PRO's or any other max IOP drives like the sandisk extreme.

    Thanks a lot for that, you saved me some serious money and maybe also my wife. I'll go with the spinners.

     

    However, this brings a new issue: cooling. One of the nice features, apart from speed and consumption, is that they don't need much cooling. I'll have to give some though to this.

     

    I'll keep you posted.

    ATLAS My Virtualized unRAID server

    in Virtualizing Unraid

    Posted

    I’m thinking of implementing FreeNAS as a fileserver for the datastores. Main reason is redundancy. Right now my server has only one SDD as a datastore, so if it fails I’m sc***  :(!

     

    However, and although my main purpose is redundancy, I’m also very happy with the speed of my setup. I’m using a 256MB Plextor M3 SDD connected to one of the SATA III ports of the X9SCM-F-O. Last time I checked reading speeds were in excess of 500MB/s  ;D. So ideally I’d like to get raid protection without sacrificing performance (and even improve it if possible).

     

    My intention is to dedicate a M1015 for the FreeNAS fileserver with only 4 drives (1 dedicated 1U case), so the M1015 will be under used. And here is where I start having questions:

     

    • Since I’ll be using only 4 drives, can I plug the M1015 in one of the PCIE x4 ports without sacrificing performance?
    • I know close to nothing about FreeNAS, but I believe it’s a stripped raid. What kind of writing/reading performance improvement should I expect?
    • Are SDDs really worth it for FreeNAS, or should I go with spinners? If using SSDs, what kind of speed should I expect? What about spinners? Please feel free to recommend parts  :)!
    • When looking around I’ve seen many, reasonably priced, 1U SATA II enclosures for 4 drives. However, when looking for SATA III, prices really go up, especially here. I’m Spanish, so apart from the spelling mistakes (sorry!), my options to shop around aren’t many. If I go with SSDs, going with SATA III is a no brainer, but if using spinners, could I take SATA II?
    • Anything else I’m missing?

     

    I know Johnm has a similar setup, but does anyone else have/is planning, something like this? I wouldn’t like to overload Johm with so many questions; he has already contributed so much to this thread  ;)!

     

    Newznab

    in User Customizations

    Posted

    First, need to setup unrar and then in setup specify where unrar is, and also setup for deep dive password checking.

    Then supposedly the  ./misc/testing/update_parsing.php script should help look into nfo & the rar files to try and help make out the garbled ones.  just add that to your newznab_local.sh script to run each time.

     

    Where should it go? after the update_releases.php?

     

    Newznab

    in User Customizations

    Posted

    I have so many releases with a crapload of scrambled letters, numbers, etc that there is no way of knowing what they are.  Does everyone else see this as well?  Most end up in the Misc catagory.  Of course I do have the releases with normal titles as well... but figure allot of the really good stuff is burried in the garbled releases.  :-

     

    There are a couple other scripts that can help..

     

    First, need to setup unrar and then in setup specify where unrar is, and also setup for deep dive password checking.

    Then supposedly the  ./misc/testing/update_parsing.php script should help look into nfo & the rar files to try and help make out the garbled ones.  just add that to your newznab_local.sh script to run each time.

     

    There are also other scripts in the ./misc/testing directory that might help.

     

    need to do the setup myself, but supposedly from the irc chat, that is the only/best way to try and figure out the garbled releases

     

    There are a lot of scripts in the testing folder. It would be nice if they had a list of what they did so we can choose to run them. I'm already using the importmodified from there which is great but that's only because I read about it on a blog.

    What does it do? Do you have the link to that blog?

     

    Sent from my GT-I9100 using Tapatalk 2

     

     

    Newznab

    in User Customizations

    Posted

    As far as auto starting, I wonder if you could point a symbolic link or create a new file in /etc/init.d? Wouldn't that auto start with the server?

     

    Well, FWIW, I'm a linux noob. i know I should create a script, copy it to init.d, make it executable and update rc. the question is: what goes in the script  :D?

    Newznab

    in User Customizations

    Posted

    what exactly is the benefit of running newznab via the init script?

     

    To be honest, I don't know if there are any actual benefits. It runs in a row:

    • update_binaries.php
    • update_releases.php
    • update_predb.php true
    • optimise_db.php
    • update_tvschedule.php
    • update_theaters.php

    It then waits 10min (you can modify this value) and starts all over again. I don't know about the cron jobs some are using because I haven't use them. About the default screen script "newznab_screen.sh", it does pretty much the same, although you have to actually run it.

     

    I use the init because it runs automatically. That's the only reason. I use a dedicated server that reboots automatically on security updates. I never have to worry whether newznab is running, because it is. I can close the console and putty terminal without worrying whether I connected or disconnected screen. Basically, it's a self-running server.

     

    The only thing I'm missing is a low HDD space mail warning (almost there) and how to start sphinx automatically on reboot. I can then forget about it, which it's the actual purpose or running a server, isn't it?

     

    Newznab

    in User Customizations

    Posted

    I'm almost there, running quite stable for two days, indexing a few groups (7) and running the init newznab.sh script. So everything is pretty much automated without screen.

     

    I'm only missing how to autotart the sphinx daemon (./nnindexer.php daemon).

     

    Any hints?

     

    PS: running a VM in ESXi with 50GB HDD, 4GB RAM and 4 cores. I know this is too much, but I don't to run into issues because of lack of resources :-)

    Newznab Plug-in

    in 6.0 (Unverified)

    Posted

     

    Cheers mate, that helped I have the backfill running I think at the moment :) something is happening as my CPU usage is way up.

     

    Here is my output after starting the back fill command, any care to tell me if this looks ok ?

     

    root@Tower:/mnt/cache/Apps/newsnab/misc/update_scripts# php ./backfill_threaded.php

    Starting threaded backfill process

    [Thread-MASTER] Spawning new thread.  Still have 3 group(s) to update after this

    [Thread-MASTER] Spawning new thread.  Still have 2 group(s) to update after this

    [Thread-26231] Begining backfill processing for group alt.binaries.x264

    [Thread-MASTER] Spawning new thread.  Still have 1 group(s) to update after this

    [Thread-MASTER] Spawning new thread.  Still have 0 group(s) to update after this

    No more groups to process - Initiating shutdown

    [Thread-26234] Begining backfill processing for group alt.binaries.hdtv

    [Thread-26233] Begining backfill processing for group alt.binaries.hdtv.x264

    [Thread-26232] Begining backfill processing for group alt.binaries.multimedia

     

    It looks like you have 4 groups to backfill and all started well. Did you modify the number of threads?

     

    Let us know how it goes and how long does it take (how many days are you backfilling?) , I'm also interested in the threaded scripts.

     

    By the way,  did you stop the regular updating (screen, cron,  newznab.sh,  whatever you're using)? I read somewhere (can't find where)  it's better to stop updating while backfilling.

     

    Sent from my GT-P7500 using Tapatalk 2

     

     

    newznab + unRAID integration?

    in User Customizations

    Posted

    First, where do you have ./newznzb.sh ? is it in /etc/init.d ?

    Yes, I have newznab.sh in /etc/inid.d

     

    Second, did you edit your newznab.sh file and change the path variable to the correct path (the location of update_script) ? this is required

    The path was also good. I forgot to change the sleep time and left it at 10 sec, but this shouldn't have stoped it

     

    I finally got it working. This what I did:

     

    cp /var/www/newznab/misc/update_scripts/nix_scripts/newznab.sh /etc/init.d
sudo mcedit /etc/init.d/newznab.sh

    some people might prefer nado or vi, I like mcedit, it's quite user friendly.

     

    In the file I changed this values

     

    NN_PATH="/var/www/newznab/misc/update_scripts"

    and

    NN_SLEEP_TIME="600"

    I'm just indexing 5 groups so far

     

    I then made the file executable and install the init scripts

    chmod +x /etc/init.d/newznab.sh
update-rc.d newznab.sh defaults

     

    I rebooted the system, and there it is newznab running as a daemon, beautiful  :D

     

    root      927  0.0  0.0  4400  464 ?        S    20:21  0:00 /bin/sh /etc/rc2.d/S20newznab.sh start

     

     

     

    newznab + unRAID integration?

    in User Customizations

    Posted

    Had the same problem.

    I have no idea why but the newznab+ doesn't include cron_scripts folder in svn... lol

    i just downloaded the free version from the website and just copied over the folder

    in that folder, there should be newznab.sh. Edit the path to your updare_script folder.

    Also, make sure to make it excutable 

    chmod +x ./newznab.sh

     

    running 

    ./newznab.sh start

    should be enough to run update every 10 mins..

     

    Could you explain this further? I followed your instructions but newznab didn't automatically start on system reboot.

     

    It worked only when running as root although I got an error. Please see below my output

     

    ubuntu@hades:~$ ./newznab.sh start

    -bash: ./newznab.sh: No such file or directory

     

    ubuntu@hades:~$ sudo su

    root@hades:/home/ubuntu# cd /etc/init.d

    root@hades:/etc/init.d# ./newznab.sh start

    Starting Newznab binaries update...root@hades:/etc/init.d#                                    [OK]

    E: File read error

    E: File read error

    E: File read error

     

    root@hades:/etc/init.d#

     

    newznab + unRAID integration?

    in User Customizations

    Posted

    Until we have a fully working plugin... Don't forget to update your NewzNab from time to time.

     

    There were a couple of updates that came out today... One being backfill.

     

    To update, go to where you installed NewzNab and type:

     

    svn update

     

    I'm getting this message:

     

    $ sudo svn update

    Skipped '.'

     

    Is this ok?

     

     

    PS: running a dedicated VM within ESXi

     

    [SOLVED] Issues mounting flash drive in linux server

    in General Support (V5 and Older)

    Posted

    I'm running 5rc8a as a VM in ESXi.

     

    I can see the flash drive from my windows desktop, so it's shared, no doubt about it, however, when I try to mount on linux it doesn't work (running linux server 12.04 x64)

     

    trying to mount with

    sudo mount -t cifs //192.168.1.153/flash /mnt/flash -o username=root,password=******,iocharset=utf8,file_mode=0777,dir_mode=0777

     

    What am I doing wrong?

    Try a different user. I don't think root has the ability to mount shares in rc8.

    And how is that the flash can be accessed from windows machines?

     

×
×
  • Create New...