ISCSI Support


Recommended Posts

Ok, I see what you mean about the reboot thing, I had to shut down the server to add some discs and when I rebooted the entire iSCSI plugin was gone XD

I added the fileIO as new, just used the same name in the GUI and the data was there :)

Edited by isvein
Link to comment
5 minutes ago, isvein said:

Ok, I see what you mean about the reboot thing, I had to shut down the server to add some discs and when I rebooted the entire iSCSI plugin was gone XD

Is there a way in the GUI to add the existing fileIO or do I need to use the terminal for that?

The iscsi plugin was gone? That should't happen, the plugin should stay in place.

Uodate the plugin and you will have now a save button on the first page after you've created all you disks, volumes,... Click the save button and everythin should be saved and is restored after a reboot.

You can always use the unraid termnial...

 

Btw please mark me in a post or quote me otherwiseI don't se your comments here.

  • Like 1
Link to comment
13 minutes ago, ich777 said:

The iscsi plugin was gone? That should't happen, the plugin should stay in place.

Uodate the plugin and you will have now a save button on the first page after you've created all you disks, volumes,... Click the save button and everythin should be saved and is restored after a reboot.

You can always use the unraid termnial...

 

Btw please mark me in a post or quote me otherwiseI don't se your comments here.

Yup, it was gone from the Unraid GUI at least, I did not check the drive if the files was there.
Now Im testing out BlockIO as I have a spare 1TB I can test with, it also works right out of the box as long as the drive was not formatted in Unraid it worked as I expected it. 

But the unassigned dives  plugin can still format the drive, so I guess its best to be careful and remember what drive you use for what until this is a part of Unraid officially.

 

Oh, and yes, I get the expected write speed to the BlockIO LUN :)

It is not a problem connecting different LUNs to the same IQN as long as its the same computer they are connected to?
I know you should not use same IQN for different computers, that may/will result in data corruption.

Link to comment
10 minutes ago, isvein said:

Yup, it was gone from the Unraid GUI at least, I did not check the drive if the files was there.

That's really strange since I installed it from the beginning on and never had such a problem. Have you done a upgrade to another version of Unraid or anything?

 

11 minutes ago, isvein said:

But the unassigned dives  plugin can still format the drive

That's by default, think of it like if you pass it through to a VM, it will be the same, just turn on the Passthrough switch and it will be not editable.

 

12 minutes ago, isvein said:

Oh, and yes, I get the expected write speed to the BlockIO LUN :)

Nice

 

13 minutes ago, isvein said:

It is not a problem connecting different LUNs to the same IQN as long as its the same computer they are connected to?
I know you should not use same IQN for different computers, that may/will result in data corruption.

Yes that's no problem.

You should never connect one iSCSI target to multiple initiators (computers).

Link to comment
7 hours ago, ich777 said:

That's really strange since I installed it from the beginning on and never had such a problem. Have you done a upgrade to another version of Unraid or anything?

 

That's by default, think of it like if you pass it through to a VM, it will be the same, just turn on the Passthrough switch and it will be not editable.

 

Nice

 

Yes that's no problem.

You should never connect one iSCSI target to multiple initiators (computers).

No, no upgrade or anything like that.

 

So the drive should be formatted in Unraid before setting it up for iSCSI, because that is the only way to set the pass-through flag.

I tried that and it showed up in Windows10 as formatted and ready to use, so seems to work as intended :)

Link to comment
5 hours ago, isvein said:

No, no upgrade or anything like that.

That's really strange, since it only tries to install it on boot and the files should be in place on the USB Boot device.

 

5 hours ago, isvein said:

So the drive should be formatted in Unraid before setting it up for iSCSI, because that is the only way to set the pass-through flag.

Tried it now with a spare ext4 formatted disk and the passthrough flag works right out of the box and I also can use this disk as a target, the only downside to this is that you have to format it in the windows Disk Manager directly.

  • Like 1
Link to comment

Hi All,

Thanks for all your hard work on this.  I've noticed something a little bit odd with read/write speeds.  Just wondering if anyone has seen anything similar?

Just to confirm all network speeds are solid, all over CAT6/6a and IPerf to and from server is consistent 1Gb both ways.  I am using 5x 1Gb files in a single folder for all tests.  Ill list below what I am seeing:

PC transfer to SMB share on Cache = Transfers at full speed (112Mb)
PC read from SMB share from Array/Cache = Transfer at full speed (112Mb)

PC write to iSCSI on block volume on dedicated SSD = Transfer at full speed. (112Mb)
PC read from iSCI on block volume on dedicated SSD = Transfer at full speed (112Mb)

PC write to iSCSI fileIO Blockstore volume on Cache Drives = Transfer at a flat 17Mb
PC read from iSCSI fileIO Blockstore volume on Cache Drives = Transfer at a flat 20Mb

I have moved the Blockstore from the Cache drives to some spinning disks in the array and its the same result.  I did expect some overheads/loss in terms of throughput put not 95Mb(ish).  Run similar tests using TrueNAS as a VM and it was perfect, wire speed all the way.

Any pointers would be welcome.

Thanks,

Link to comment
12 hours ago, scubalee said:

PC write to iSCSI fileIO Blockstore volume on Cache Drives = Transfer at a flat 17Mb
PC read from iSCSI fileIO Blockstore volume on Cache Drives = Transfer at a flat 20Mb

Can you try to set 'Write Back' to 'true', this will cache smaller files onto RAM and writes it after it caught up to the img itself.

 

ATTENTION: Please keep in mind if you restart all your fileIO images will be gone from the iSCSI GUI they are still there but you have to assign them new to the LUN, after that it works just like normal.

There is currently a bug in the Plugin (iSCSI-Target) that removes the images from the config, I will push a fix ASAP.

 

12 hours ago, scubalee said:

Run similar tests using TrueNAS as a VM and it was perfect, wire speed all the way.

Can you give me the configuration of the TrueNAS iSCSI Target? Write Back enabled? On which filesystem is the img located?

Sorry for that many questions, but I'm not too familar with TrueNAS.

 

Also I will open up a proper support thread for this if the plugin is working correctly.

Link to comment
  • 1 month later...
50 minutes ago, tknx said:

So reading through all of this has left me more confused - iSCSI kernel support is in the 6.9.0 rcs, but I need a plugin?

Exactly.

 

50 minutes ago, tknx said:

Which plugin?

The iSCSI Plugin which I created, you can only see it in the CA App if you are on 6.9.0rc2, also @SimonF made an addon to my Plugin (nicer GUI) because he got much better webcoding skills than me (someday I will integrate the GUI from @SimonF when I got more time to implement it, but further testing is needed).

 

50 minutes ago, tknx said:

How stable is this?

I don't understand, this is as stable as it can get on Linux. Limetech already included the Kernel-Modules for iSCSI and I wrote a plugin for the necessary tools to interact with it and have a basic GUI (the Plugin is in Beta state because, let's say the GUI is pretty basic because I got not the best webcoding skills... :D ), you always have the ability to use it from the CLI).

I use it on a daily basis with my SSD in my server and connected to my Gaming rig where my games are on, no problems so far...

 

50 minutes ago, tknx said:

Will Unraid be adding a non-plugin, supported method?

That's a question Limetech has to answer, I think someday they will implement it but for now you have to stick with the plugin that will install all the necessary tools to interact with targetcli be it from the command line or from the GUI.

 

EDIT: But think of other Plugins like the Nvidia-Driver, DVB-Driver, Mellanox Firmware Tools, Unassigned Devices, Unassigned Devices Plus,... they are all Plugins and they work just fine, much easier to install these days... ;)

  • Like 1
Link to comment
On 12/16/2020 at 11:41 PM, isvein said:

with multiply cache pools coming up, is it possible to set an pool as an iSCSI LUN?
I guess that is no problem with an fileIO at least.
Is there any benefits with DiscIO over FileIO or the other way around?

You can't setup a DiscIO with a Cache Pool because if you want to setup a DiscIO you have to pass through a whole disk that is dedicated to iSCSI and outside the Array or a Cache Pool.

FileIO is no problem and can be on the Array or a Cache Pool but is also much slower.

 

For example I use a SATA SSD and get the full speed out of it over my 10Gbit/s link to the Server without an issue.

  • Like 1
Link to comment
23 hours ago, ich777 said:

You can't setup a DiscIO with a Cache Pool because if you want to setup a DiscIO you have to pass through a whole disk that is dedicated to iSCSI and outside the Array or a Cache Pool.

FileIO is no problem and can be on the Array or a Cache Pool but is also much slower.

 

For example I use a SATA SSD and get the full speed out of it over my 10Gbit/s link to the Server without an issue.

So the main difference between a FileIO and DiscIO is the speed. Thanks :)

Edited by isvein
Link to comment
Just now, isvein said:

So the main difference between a FileIO and DiscIO is the speed you can get. Thanks :)

This is not the whole story if you turn on Write Back then it's not that different since it caches everything into RAM and will then write it later to the file, if you got enough RAM this should not be a problem but keep in mind if there is a power outage this could possibly lead to data loss.

  • Like 1
Link to comment
  • 3 weeks later...

Thanks Simon, looks like that's the one:

 

0x4d (LOG SENSE), presumably because the Solaris initiator is trying to check for disk errors. Fortunately we are already doing that on the iSCSI backends (well, usually).

 

Not sure exactly how or why ESX is trying to 'check for errors', but I suspect that's what it is.  

 

Good news though, despite those errors/warning messages, going into 24 hours with rock solid connection via iSCSI.  No drops, vMotion is working fine, VM itself has nothing troubling reported at all.  Working very well!

Link to comment
21 hours ago, Overtaxed said:

unraid kernel: iSCSI/iqn.1998-01.com.vmware:esx1-5b059529: Unsupported SCSI Opcode 0x4d, sending CHECK_CONDITION.

This message can be safely ignored.

 

EDIT: I think I should be creating a real support thread for the iSCSI plugin...

Link to comment
15 minutes ago, ich777 said:

This message can be safely ignored.

 

EDIT: I think I should be creating a real support thread for the iSCSI plugin...

You can use my gui one also if you want could change titleto iscsi

Edited by SimonF
  • Like 1
Link to comment
Quote

EDIT: I think I should be creating a real support thread for the iSCSI plugin...

 

Well, the support I'm getting here is lightyears beyond what I get for commercial software, so, while it might be a good idea, you guys are killing it.  And thank you for that and this contribution to the UnRAID community!  Even if it breaks and takes all the VMs with it, you guys have shown it's doable and I'm sure this functionality will wind up mainstream in the future, that's a huge step forward for all of us idiots doing stuff like running ESX clusters at home.  ;)

  • Like 1
Link to comment

So, doing a little more testing, things are still working very well.  Not a single blip to my ESX servers, and, even better, I've rebooted UnRAID while a VM was up, using iSCSI and it recovered from that gracefully (VM froze until storage came back, which is what it should do).  

 

One thing I've noticed, it appears that the iSCSI traffic all runs on CPU 0.  Here's what I'm seeing running an FIO from the VM using an iSCSI disk; 93% utilized on CPU0, everything else pretty much flatlined.  Now, the good news, this is pushing 3-4Gb/s of traffic down that iSCSI pipe, not too shabby!  

 

Is iSCSI bound to dom0 or restricted from the rest of CPUs (either by design, or perhaps by something I did)?

 

iSCSI all on CPU1.jpg

Link to comment

Hi All!! 

i've installed the iscsi plugin, in order to create a iscsi target for a windows machine.

Unfortunatelly on windows i have the server status as "inactive", i tried in several ways but all was not good.

 

Can you help to find the issue?

 

this is the plugin settings

 

o- / ......................................................................................................................... [...]
o- backstores .............................................................................................................. [...]
| o- block .................................................................................................. [Storage Objects: 0]
| o- fileio ................................................................................................. [Storage Objects: 1]
| | o- iscsi ......................................................... [/mnt/user/iscsi//iscsi.img (20.0GiB) write-back activated]
| | o- alua ................................................................................................... [ALUA Groups: 1]
| | o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
| o- pscsi .................................................................................................. [Storage Objects: 0]
| o- ramdisk ................................................................................................ [Storage Objects: 0]
o- iscsi ............................................................................................................ [Targets: 1]
| o- iqn.2003-01.org.linux-iscsi.unraid.x8664:sn.7eedff846bbb .......................................................... [TPGs: 1]
| o- tpg1 ............................................................................................... [no-gen-acls, no-auth]
| o- acls .......................................................................................................... [ACLs: 0]
| o- luns .......................................................................................................... [LUNs: 1]
| | o- lun0 ................................................... [fileio/iscsi (/mnt/user/iscsi//iscsi.img) (default_tg_pt_gp)]
| o- portals .................................................................................................... [Portals: 1]
| o- 0.0.0.0:3260 ..................................................................................................... [OK]
o- loopback ......................................................................................................... [Targets: 0]
o- vhost ............................................................................................................ [Targets: 0]
o- xen-pvscsi ....................................................................................................... [Targets: 0]

Link to comment
22 minutes ago, gigo90 said:

Can you help to find the issue?

You have to do all steps not just only creating a IQN, a FileIO Volume and a LUn, you also have to authorize your windows machine in the tab ACL.

 

Should look something like this:

grafik.thumb.png.efc45d208a3f59f3fa72903a26796c02.png

Link to comment

Thank you very much, after complete the ACL with the IQN(of windows pc) it works perfectly.

 

One more question, my unraid has a cache pool, is possible to use this pool, for caching write to iSCSI File IO?

 

This is my folders, inside "iscsi" i put my FileIO Volume and "write back" option is enabled.

image.thumb.png.fc2533d7fc4515cd618b3a7aa92ce60e.png

 

Many thanks

Edited by gigo90
Link to comment
10 hours ago, gigo90 said:

FileIO Volume

With the way you have it set at present the file will be moved to the array if it is not inuse if on the cache drive. If already on the array nothing will change.

 

Do you want the img to be on the cache and run from the cache or do you want it to run from the array.  You cannot cache writes to the img.

 

Think of it more like a vdisk you would assign to an VM, its a single file.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.