What do you want in a custom unRAID package (BubbaRaid)

[NLS]

Since you may have missed this, I'd like to point you to my post:

http://lime-technology.com/forum/index.php?topic=2505.msg24396#msg24396

...if you have something to say about #3 and #4.

In fact let me copy them here:

 

3) what about the transfer speed tweak? is it there? I am not sure if it is still needed in 4.4 but I have 4.4 and doesn't seem to hurt - couldn't find a reply in the thread (but did find the question asked before)

 

4) any chance of a working (and configured) properly ftpd? or is out of the scope (and abilities) of bubbaRAID? - it was in my original requests, still I don't know if it is there or not

 

Plus what I asked here:

http://lime-technology.com/forum/index.php?topic=2505.msg24503#msg24503

 

"Is funambol server completely out of the scope of your package?"

 

Thanks.

 

[Bigdady92]

Fully understandable bubba.

 

I'm a sysadmin myself. I could hack away at the slackware stuff (I'm more into redhat/debian) but I figured if it was on your list it wouldn't be a bad thing to have rolled into it.

 

[bubbaQ]

I figured if it was on your list it wouldn't be a bad thing to have rolled into it.

 

Don't ever be shy about asking for a particular package.

 

There is no real hacking involved... the step-by-step instructions in the Wiki are no more difficult than baking a cake.

 

3) what about the transfer speed tweak?

 

It ain't needed.  The later version of unRAID ALREADY set the readahead buffer.  If you doubt me, just qquery the read-ahead biffer setting on a stock 4.3 or 4.4 system.  If you still think it is needed, you can added it to your go script.

 

any chance of a working (and configured) properly ftpd?

 

Not likely.

 

funambol

 

Answered 7 posts back.

 

BubbaRaid is intended to provide applications, particular in streaming media, with wide appeal to users who are not capable of rolling their own kernel.  Not I said not "not capable" rather than "capable, but do not want to."

 

I know it would be nice to have a plug-and-play super-unRAID that had everything and a duck, preconfigured and running.  But that simply ain't going to happen.

 

If it were not possible for a user to easily install a full Slackware distro and add unRAID to it, I'd probably be more open minded... but it really is simple for a user to do.

 

[NLS]

Fair enough.

 

Could you elaborate a bit on the ftpd thing?

Is it something specific that complicates things?

 

And please if 0.0.9 (or newer) is not coming in complete form soon (like next five days), can you at least tell me if it is possible to follow the following procedure instead of the one described a few pages back?

 

1) Expand 0.0.7 on a 4.4 install (because this is my current install).

2) Before running anything, expand 0.0.9 over the above.

 

Or should I REALLY go back to 4.3.3, then 0.0.7, then 4.4, then 0.0.9?

 

 

[bubbaQ]

Could you elaborate a bit on the ftpd thing?

Is it something specific that complicates things?

 

Just a lack of time, and the priority is low since vsftp is already in the stock unRAID distro.

 

I'm working on a web-based configurator for the applications, so users just fill out info in a web form, rather than editing the config files directly.  Then it will do search and replace in the config files to implement the user's choices.  This will prevent overwriting of user's customizations in config files.  It's real slick and can easily handle new applications and config files with a minimal of effort, and even flag non-default values.  The tricky part is the regular expressions and finalizing some architecture decisions -- for example should config file entries that are managed by the web UI be collected together at the top or bottom of the config file, or left in their original position with a comment added above?

 

I'm also not happy with the size of the distro... over 100MB.  My first attack on that reduces memory requirements and the size of the initramfs, but results in more activity on the flash, which I don't want.  Plus the size issue is not solved, just "delayed" and will come back as I add more applications.  I'd like to store some parts of BubbaRaid on the array, but that is tricky.  I'm considering a bootstrap loader for BubbaRaid, that will ask the user where on the array they want BubbaRaid to live, and then populate it.  I'd like to use a web-retrieval method for populating large applications, since that will reduce the download size of the base distro, and users will only download the additional applications they want.  That is what I am doing with slimserver now.

 

Now when I think about all this work, I love programming challenges, and this is a nice project to me.  But OTOH, my unRAID servers are all installed on a full Slackware distro, with many additional applications, including even my weather station monitoring (http://www.weather-watch.com/smf/index.php?PHPSESSID=4um6pfvvm4tfqih1t3bo0as7p2&board=21.0) and a bunch of other stuff.  So why do the extra work?  I originally thought the step-by-step instructions I put in the Wiki on installing unRAID on a full Slackware distro would have been enough for most people to be able to take their unRAID server anywhere they wanted, but I was obviously wrong.

 

0.0.10 will not be out in 5 days, .... but probably before 5 weeks.  I might can do an interim "fresh install" one sooner, but it will overwrite all the BubbaRaid config files, so if you have customized any of them, the customizations will be lost.

 

I added version protection to 0.0.7, so the batch files to install it check your version of unRAID on the stick, and won't install BubbaRaid unless the version of unRAID you have is 4.3.  You can edit the enable_bubba batch file in 0.0.7 to delete the version checking if you want, and it will install over 4.4.  Then unzip 0.0.9 over the top of that without rebooting, and you will be good to go.

 

[NLS]

Well if you ask me, left in original position with comment (and if the comment includes "BubbaRAID" in the string, it is very easy for less-than-basic users to see what happened with a simple search).

 

The problem with downloading packages as part of the install is that you need to make sure your specific "tested" version is downloaded and make sure it still works in the future etc. Other than that, it is a nice system.

 

...As is to store some things in some place on the unRAID hard disks.

 

I'll look in enable_bubba for the version checking routine. If I mess the string, "I'll be back".

 

Thanks. Very interesting what you are doing. As another older (huge) thread points out though, unRAID should at some point "natively" have some plug-in infrastructure (wouldn't bother Tom with unnecessary debugging - which I know is one of his fears - but would help you guys take things forward).

 

 

 

 

[bubbaQ]

The problem with downloading packages as part of the install is that you need to make sure your specific "tested" version is downloaded and make sure it still works in the future etc. Other than that, it is a nice system.

 

As the Slimserver issues show, downloading has potential problems.  Where I can, I plan on hosting the packages on *my* server... not downloading them from the original source, to address that problem.

 

I'd venture that extra debugging is only one concern of Tom... the extra support issues of enabling third-party stuff within unRAID will likely be much larger.

[NLS]

yes but it is very easy to "point" when it's not your problem (eg. "hey what you ask is for bubba" or Joe L. or whoever)

 

after all we all know Tom - he will just ignore whatever doesn't involve his build (and even some that does ) - so shouldn't be that much of a problem

 

anyway - let's not move off topic

I am going to attempt the surgery tonight...

Shouldn't be too hard (or I have really become a glorified user after 25 years in computing).

 

[NLS]

hm I didn't find any version checking function in enable_bubba

what I did find was "unraid.version" file with the version in text inside it, that is overwritten with the "correct" on in 0.0.9 anyway

 

am I missing something?

 

(btw could/should enable or disable scripts be run directly over the flash share or when stick is mounted locally on windows machine?)

 

 

[bubbaQ]

Whoops... my bad.  The version checking is in the "upgrade_bubba.sh" script... not the enable_bubba batch file.

 

You should be good to go.

[NLS]

erm... can't find this script

 

(even used search command in the zip)

 

[NLS]

(newer news) ...I tried anyway... the batch executed fine, the stick rebooted fine (although just before the login prompt in my console I see:

 

mkdir: cannot create directory /mnt/disk1: file exists

and cannot find readable session directory /mnt/disk1/torrents/downloading/rtorrent session from config/root/.rtorrent.rc check permissions

 

and most importantly all windows in dashboard show empty config files (plus clicking on for example lighttpd control panel button, does nothing)

 

so something is wrong eh?

 

help?

 

on the bright side unraid works normal (and the old config page) but is performing a parity rebuild for some reason (normal?)

 

 

[bubbaQ]

mkdir: cannot create directory /mnt/disk1: file exists

and cannot find readable session directory /mnt/disk1/torrents/downloading/rtorrent session from config/root/.rtorrent.rc check permissions

 

Those are OK. 

 

(plus clicking on for example lighttpd control panel button, does nothing)

 

(sigh) I was afraid of that.  You are running unRAID 4.4... right?  4.4 mounts the flash differently, so the access to the config files is hosed.  I was hoping that was just a glitch on my system, but I guess not.

 

Compare the output of the "mount" command in 4.3:

 

/dev/sda1 on /boot type vfat (rw)

 

with 4.4:

 

/dev/sda1 on /boot type vfat (rw,umask=066,shortname=mixed)

 

This prevents the php code from accessing the config files.

 

[NLS]

is there anything we can do?

 

manually set umask for your files? I dunno

 

 

[NLS]

...me is eager to make this work.

 

Seems to be close to perfect for my needs - when it works.

[bubbaQ]

Yea... got me stumped too.  I can work around it by copying the /bubba dir from the flash to the array, and symlinking it, but that isn't a real solution.

[NLS]

...so... are you looking into it?

(sorry I just see the empty screens and I am full of torrents I want to move on it)

 

I think copying your (let say) "system files" from flash to array (and syncing back vital config files etc.?) is not such a bad idea.

If you can make the path configurable during install at least.

 

Plus it could allow bubbaRAID to grow large (with the assistance of during-install-downloads) without being too afraid of the stick size.

 

 

[bubbaQ]

No, it's not a solution, as Bubba needs access to those files BEFORE the array loads.

 

I've been able to get it working by copy/symlinking to my boot drive on a system that boots from HD, but that won't work for you.

 

If you really want Bubba, you need to go back to 0.0.7 and 4.3.3 for now.

[NLS]

OK clear, I just hope you are not abandoning this...

 

 

[bubbaQ]

Not abandoning... I just can't spend unlimited time on it right now.

 

I use my own BubbaRaid myself, for torrents.  I don't want to muck up my main unRAID server with Torrents, but I do occasionally use them.  Plus because of the inbound port fowarding security issues, I like having them on a separate server.

[NLS]

Being a network guy (ok I want to say I am - that's a closer to reality statement) I would be very interested to hear what security issues come from port forwarding.

(urban oversimplistic scary legends, nothing material - if you ask me, that is)

 

I really don't see how port forwarding towards torrent service can compromise the rest of my server (except things that don't mean ACCESSING data, but D.o.S. and such - for which I have the proper protection anyway).

 

Matter of another thread possibly.

 

I will move to 4.3.3 both because of your system and because I read about some issues in 4.4.

But I really hope we see a nice bubba-magic update "soonish".

 

 

[NLS]

a REALLY important on-topic question now... reverting to the old images (4.3.3 over 4.4) will be enough for your system to start working?

 

should I run enable again? (I doubt it)

 

thanks

 

[bubbaQ]

I would be very interested to hear what security issues come from port forwarding.

 

It is not insecure per se.  But as a practical matter, I seem to have fat fingers, and often select the wrong rules or otherwise fubar it.  Operator error.  It's simpler to just have no incoming ports open at all... and easier to confirm it and police it.  Intrusion detection can be set on high, etc.

 

a REALLY important on-topic question now... reverting to the old images (4.3.3 over 4.4) will be enough for your system to start working?

 

As far as I know, and based on feedback here, 0.0.7 on 4.3.3 was pretty stable and functional.... so wipe out your flash (except for /config), put unRAID 4.3.3 on it, then expand Bubba 0.0.7, run "enable_bubba" and boot it.

[NLS]

OK I found a bug now that I just almost scre*ed myself.

 

Your enable script (although correct) somehow does NOT keep our original go and syslinux.cfg!!!

I check the .original files and I see the GENERIC versions not MY versions!

 

Luckily an undelete utility recovered them (plus searching in the forum to find my old issues and "re-resolve" them).

 

---

 

Second thing.

In my own go script I have these additional lines:

 

for i in /dev/md*
do
    blockdev --setra 2048 $i
done
echo "nameserver 192.168.0.11" >/etc/resolv.conf
installpkg /boot/packages/ntfs-3g-1.2310-i486-1McD.tgz
cp /boot/smtp_check_unraid.sh /etc/cron.daily/

 

in other words:

- the blockdev trick,

- the nameserver in resolv.conf

- install of ntfs-3g package (probably old nowadays, need to check for new version)

- a cron job for sending email to me

 

is ANY of the above in conflict with your system?

if not can I freely re-add them at the end of your own go script?

(and btw, are they something to consider for your bubbaRAID?)

 

---

 

EDIT:

 

0.0.7 on my 4.3.3 (which I first booted and works ok), fails miserably!

My system does not get a working network (cannot see web, shares or even ping it) and seems to have great problems discovering correctly my hard disks! (many failures and retries). Thankfully I have my unRAID VGA connector on my KVM so I can see all these things.

 

Reverting to normal unRAID (4.3.3) works.

I remind you that 0.0.9 on 4.4 didn't have THOSE problems (unRAID part of your system was working fine then).

 

Can I help you help me somehow?

 

[bubbaQ]

OK.... lets take if from the top.

 

If you wipe out your flash, other than /config, and go to a plain unRAID 4.3.3 with your go scrip modifications, does it boot properly?

 

Take that stick, and in a Windows box, expand 0.0.7 to it, and run "enable_bubba"

 

- the blockdev trick,
- the nameserver in resolv.conf
- install of ntfs-3g package (probably old nowadays, need to check for new version)
- a cron job for sending email to me

 

BubbaRaid should have no problem with those.