RFQ: USB Flash Creator Rework


jonp

Recommended Posts

Hi Unraid Community!

 

We have a special request for anyone who is familiar with the work required and wants to make a little $cash!

 

A few years back we released our own USB flash creator tool for Unraid OS.  For those of you who remember, installation of Unraid used to require a manual process (documented here), but we wanted this new tool to be a far easier way to get up and running.

 

Here we are a few years later and the tool desperately needs an update, especially the macOS version.  Our problem is that the development team is heads down focused right now on getting 6.9 and 6.10 out the door.  As such, we wanted to throw out a request to our Community to see if anyone has the tools and talent to help us with this.

 

This is a formal RFQ (Request for Quote) to correct issues in the current USB flash creator for Unraid OS, for both Windows and Mac platforms.  We're not necessarily looking for any increased functionality at this time, though creative ideas on how to make it better will be considered.

 

To respond to this RFQ, please email [email protected] with your bid and time estimate for the work.  We will update this post once a bid has been accepted.  If you have questions regarding the RFQ, please post them here so our responses can be made in the post publicly for all to see.  Thanks everyone!!

 

All the best,

 

Team Lime Tech

  • Like 6
  • Thanks 1
Link to comment

I think that is a good idea. Maybe the tool could also execute a health check on the pendrive and/or look for it's ID, I have some cheap pendrives here that present their selves like this in lsusb:

 

Bus 001 Device 008: ID abcd:1234 Unknown

 

Going beyond, the tool could help our fellow unraiders that want to switch the license from one pendrive to another.

  • Like 1
Link to comment
9 hours ago, agarkauskas said:

I think that is a good idea. Maybe the tool could also execute a health check on the pendrive and/or look for it's ID, I have some cheap pendrives here that present their selves like this in lsusb:

 

Bus 001 Device 008: ID abcd:1234 Unknown

 

Going beyond, the tool could help our fellow unraiders that want to switch the license from one pendrive to another.

I just don't see the point in going down the same path, Big Sur has some substantial changes under the hood, they could rewrite the app to work on Catalina and then find themselves back in the same position after Big Sur is released.  

 

There are tools out there Balena Etcher and as also mentioned Rufus which are pretty much bullet proof, used by lots of high profile projects (Raspberry Pi official method of flashing) that are well supported, documented, maintained.

 

Just provide the images on the website and a link to the etcher downloads and forget about having to worry about it until the end of the universe.

 

 

Link to comment

Is it out of scope to discuss a different method of license enforcement than the USB key serial number?  It's a scary single point of failure and kinda restricting to require a USB stick at all.

If it has to be hardware, maybe the check could be when starting the array instead of on boot, and base it on one or more array member serial numbers?  Or the MAC address of the NIC?  

 

Link to comment
2 hours ago, warren17c said:

For me, the people who need to a dedicated installer are not the people who Unraid is aimed at.

Totally disagree. It is all about user experience and having a tool available, which makes it extremely easy to set up Unraid the first time is very beneficial.

 

This is also acknowledged by reviewers, who praise the easiness of Unraid installations.

 

  • Like 2
Link to comment
11 minutes ago, jammin said:

Is it out of scope to discuss a different method of license enforcement than the USB key serial number?  It's a scary single point of failure and kinda restricting to require a USB stick at all.

If it has to be hardware, maybe the check could be when starting the array instead of on boot, and base it on one or more array member serial numbers?  Or the MAC address of the NIC?  

 

Yes, out of scope for this topic.

Link to comment
15 hours ago, Fizzyade said:

Without reinventing the wheel, why not just supply an compatible img and then tell people to use balena etcher which is pretty much the go-to image writer, you just have to supply an img in a suitable format.

This is like going to the shop to buy a bicycle and the shop owner is offering a lovely motorcycle, which is more performant, looks better and can even carry a passenger ... but it's not a bicyle 🙃

  • Like 1
Link to comment
51 minutes ago, bonienl said:

This is like going to the shop to buy a bicycle and the shop owner is offering a lovely motorcycle, which is more performant, looks better and can even carry a passenger ... but it's not a bicyle 🙃

Or getting the bicycle and finding in 6 months time the wheels don't work anymore, and you need a new bicycle to get up and cycling again!. ;)

 

This to me just seems like a massive waste of effort for little gain, even with outsourcing.

 

Produce the image.

Link to etcher.

Donate the money to a charity.

 

Maybe I'm massively missing something, but I've hated the fact that I've had to use the Unraid tool and have longed to have an image that I could just flash with my favourite tool.

 

Link to comment
On 6/29/2020 at 6:25 PM, Fizzyade said:

Without reinventing the wheel, why not just supply an compatible img and then tell people to use balena etcher which is pretty much the go-to image writer, you just have to supply an img in a suitable format.

 

There are additional things that our creator does that a simple IMG file does not.  Our tool validates the USB flash GUID as usable (most of the time ;-), it allows the user to toggle EFI boot mode, as well as customize hostname and networking options.  It even lets the user select which release to install (from a backup, the current available release, or from our Next branch).  These are features that are important to ease of use for new users and while we can appreciate that not everyone needs this, those that do really appreciate it.

 

13 hours ago, jammin said:

Is it out of scope to discuss a different method of license enforcement than the USB key serial number?  It's a scary single point of failure and kinda restricting to require a USB stick at all.

If it has to be hardware, maybe the check could be when starting the array instead of on boot, and base it on one or more array member serial numbers?  Or the MAC address of the NIC?  

 

It is out of scope for the purpose of this RFQ, but know that we are investigating other licensing methods for future inclusion.  Changing licensing is always a real iceberg of a problem.  Seems small and simple from above the water line, but below it is a gigantic thing just waiting to sink your ship ;-).  That is going to have to be another battle for another day.

 

11 hours ago, Fizzyade said:

Maybe I'm massively missing something, but I've hated the fact that I've had to use the Unraid tool and have longed to have an image that I could just flash with my favourite tool.

 

While we definitely appreciate what certain users want, we have to address the wider market of users that aren't as savvy.  While I definitely agree if you're savvy enough to build a computer, you're probably savvy enough to figure out how to image a USB flash using some generic tool, but we're not just targeting that kind of customer and perhaps longer term users won't be building their own servers at all.  The point is, our flash creator tool should work fine, but it's been a bit more of a bear to maintain than we'd like, so we're looking for offers from developers that want to earn a little extra cash to help build this thing.  And yeah, we probably will have to fix it again after the next Mac release comes out, but that's fine and something we're also willing to accept.

  • Like 2
  • Thanks 1
Link to comment

Would you be open to it being made with Python? Seems to me that python would be a good choice since it is easily extensible, cross platform, and easier to maintain. Last I looked into it, you could easily build for Linux, Windows, and OS X. The only stipulation is that the OS X executable needs to be made on OS X.

Edited by bamhm182
Link to comment
2 hours ago, bamhm182 said:

Would you be open to it being made with Python? Seems to me that python would be a good choice since it is easily extensible, cross platform, and easier to maintain. Last I looked into it, you could easily build for Linux, Windows, and OS X. The only stipulation is that the OS X executable needs to be made on OS X.

 

We're not opposed to it.  The big thing is the user can't have to download and install any additional "components" in order to make it work.  So it has to be a self-contained executable on all platforms.

Link to comment

Etcher is open source, if you absolutely need extra functionality then fork that and add what you need.

 

Personally, I’m still not convinced with the justification for your own tool, if licensing is an issue then when it boots it should just stop saying “this disk cannot be used for Unraid, a disk with a serial number is required.”. The users wasted a couple of minutes at worst.

 

I don’t have the numbers, but I can’t imagine many people install Unraid on incompatible usb sticks, surely most people buy a reputable brand especially if they’re going to be using it on a system such as Unraid, which is definitely not for technophobes.

 

Link to comment
18 hours ago, bonienl said:

Totally disagree. It is all about user experience and having a tool available, which makes it extremely easy to set up Unraid the first time is very beneficial.

 

This is also acknowledged by reviewers, who praise the easiness of Unraid installations.

 

 

I may of come off a little to brash there. Using Etcher / Rufus is the go to for creating boot media for the Raspberry Pi, which isn't as easy to use as Unraid I will admit - despite the number guides saying otherwise - that its become more mainstream now more than ever.

 

I may be biased as on my last install the USB installer didn't work, so did it via the manual method...

 

Personally, having to use a 3rd party tool vs using an inbuilt is so minor these days, that its a "nice to have" but not an essential.

Link to comment

As someone who's never had the USB creator tool work correctly for them ever (Windows only... I don't do macs, or *nix for that matter really :) ), is it really that cumbersome to extract a ZIP file to an empty stick, apply the correct label to same, and "run as administrator" the make_bootable.bat file (or equivalent for other host OSes?)  I can some what understand the desire to attempt to appeal to the lowest common denominator, but given the general need to peek behind the curtain to fully unleash the power of unRAID, wouldn't it make more sense to expect the user base to have just a bit more skill beyond the One Ping Click Only mentaility?

Link to comment
9 hours ago, jonp said:

There are additional things that our creator does that a simple IMG file does not. 

Don't know if things have changed, but when I first started out with Unraid - or when I did some additional playing around - I found that the tool didn't set the keyboard, which caused me issues when using the GUI mode (a temporary thing). Some of us get used to the " and @ keys being in certain places... Still not sure that was ever resolved.

Totally get that Unraid is a GUI, and should start off as such with the installer. The options setting in the imager are important in helping new (and old) users avoid the command line, in the same way the browser interface sorts out (nearly?) all the configuration.

I also understand that there are very good imaging tools available, so providing a default image for users that want it and understand that it doesn't do the nice things an installer does will satisfy users that prefer their own imager.

Edited by jsebright
typo
Link to comment
9 hours ago, warren17c said:

 

I may of come off a little to brash there. Using Etcher / Rufus is the go to for creating boot media for the Raspberry Pi, which isn't as easy to use as Unraid I will admit - despite the number guides saying otherwise - that its become more mainstream now more than ever.

Interesting that the official Rasberry Pi site now has its own disk imager program on the downloads page and no longer links to Etcher like it used to.

  • Like 1
Link to comment
On 7/1/2020 at 1:26 PM, jonp said:

 

Our tool validates the USB flash GUID as usable (most of the time ;-),

Not smiling here! I hope someone who takes this up, offers assistance for those 1% people who are on a budget and want to use unraid but are crippled by its installer as I bought 2 new USB from Sandisk - Ultra fit 32GB to use for unraid to trial and I can't even do that because this installer doesn't recognise it?

 

How can I even test unraid when the installer is whacked and I have to try and use a manual method where I can't even grab a beta version to test?


Sux Balls as spent money on USB sticks in preparation to use and purchase unraid and when asked here for help I'm told buy another USB stick.

 

Given my currency is shit compared to the USD, this is not a cheap purchase for me wanting the pro version comes in just under $200..

 

 

 

Link to comment

One request: Leave the license model as is...

Linking the license to the hardware of the machine would be more than just an "iceberg problem" - that would be more of a volcano...

The GUID model is not optimal, but it is still easier to use than anything else.

My little Lexar USB stick has been working since 11.2012 and is still working (tap on wood)... 😉

Edited by Zonediver
  • Like 3
  • Thanks 1
Link to comment
  • 3 weeks later...

For those who are interested in this thread...

 

We performed various fixes and it should now work on Catalina.  We gave the fixes to the lime-tech folks, and hopefully a new USB creator will be made available to the community shortly.

 

 

BTW, the reason some USB sticks are not "recognized" is because they lack a unique identifier and hence unraid is unable to generate a valid license file (even a trial license), but the user will at least be informed about it now.

 

Basically the changes are: 

 

- Now works on Catalina (and hopefully upcoming Big Sur, will look into it if it doesn't -- if anyone wants to test it that'd be great)

 

- USB keys that can't be used will now be listed in the dropdown, but marked as incompatible. If you try to write to it, a message will appear to inform the user for the reason it can't be used (you still won't be able to write to it)

 

- If the key is larger than 4gb, a warning will appear saying the key is needlessly large for no added benefit  (you can still write to it).

 

- If more than 1 key is found to be inserted, it will be recommended to remove other keys so that you don't inadvertently write to the wrong one (you can still write to the selected key)

 

- Fixed an issue where switch internal state (downloading, checking key, writing, etc) could put the usb key creator in an unstable state if the process is cancelled.

 

- Fixed a thread cleanup issue which caused a potential crash on exit.

 

Edited by Dr. Justice
Link to comment
1 hour ago, Dr. Justice said:

If the key is larger than 4gb, a warning will appear saying the key is needlessly large for no added benefit  (you can still write to it).

Agree in principle, however I think in practice the warning is going to apply to virtually everybody, and the net result is going to be confusion. I'd prefer to see the warning at 32GB, since that seems to be the current sweet spot for pricing.

 

Don't get me wrong, the 4GB is completely accurate, I just don't think it's going to be productive, given that the price difference between a 4GB and a 32GB is very small, thus very little dollar benefit for staying low capacity, especially since the warning is issued AFTER someone already made a decision on some level to use that specific drive.

 

User thoughts, "Purchasing a 4GB drive because the install tool told me to seems like a waste after I already purchased a 16GB stick that was the same price."

 

@jonp, @limetech?

  • Thanks 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.