{"id":4633,"date":"2025-01-23T18:45:32","date_gmt":"2025-01-23T23:45:32","guid":{"rendered":"https:\/\/dft.wiki\/?p=4633"},"modified":"2026-06-08T10:18:35","modified_gmt":"2026-06-08T14:18:35","slug":"iscsi-cheat-sheet","status":"publish","type":"post","link":"https:\/\/dft.wiki\/?p=4633","title":{"rendered":"iSCSI Cheat Sheet"},"content":{"rendered":"<p><strong>iSCSI<\/strong> (Internet Small Computer Systems Interface) is a protocol that allows SCSI commands to be transmitted over a TCP\/IP network. It enables the creation of SANs (Storage Area Networks) by allowing servers (called initiators) to access storage devices (called targets) shared over the network as if they were locally attached.<\/p>\n<p>Among various storage solutions, iSCSI plays a critical role in highly available virtualization environments. When a hypervisor host fails and its running VMs go down, another host (or node) in the cluster can quickly take over and restart those VMs. This is possible because the VM volumes are stored in centralized iSCSI storage, ensuring seamless access across all cluster nodes.<\/p>\n<p>Important Concepts and Nomenclatures<\/p>\n<ul>\n<li><strong>TPGs<\/strong> = Target Portal Groups\n<ul>\n<li>A group of one or more network portals associated with an iSCSI target.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Portal<\/strong> = Network Endpoint\n<ul>\n<li>Composed of a binding IP and port. E.g. <strong>192.168.1.100:3260<\/strong><\/li>\n<\/ul>\n<\/li>\n<li><strong>LUNs<\/strong> = Logical Units\n<ul>\n<li>A unique storage resource identifier for a partition or volume. E.g. <strong>lun0<\/strong>, <strong>lun1<\/strong>, etc.<\/li>\n<\/ul>\n<\/li>\n<li><strong>IQN<\/strong> = iSCSI Qualified Name\n<ul>\n<li>Used by both initiators and targets. Can be freely chosen as long as it follows the format: <strong>iqn.yyyy-mm.reverse_domain_name:unique_id<\/strong><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<hr \/>\n<p><strong>TARGET \/ SERVER<\/strong><\/p>\n<p><strong>targetcli<\/strong> is the tool used to manage shared volumes. Changes take effect immediately with no need to reload or run additional commands.<\/p>\n<p>When exiting, it writes the configuration to disk in JSON format for persistence, backup, or source control. The location may vary by distribution, but it is typically found at <strong>\/etc\/target\/saveconfig.json<\/strong> or <strong>\/etc\/rtslib-fb-target\/saveconfig.json<\/strong>. At runtime, configuration is also held under <strong>\/sys\/kernel\/config\/target\/<\/strong>.<\/p>\n<p><strong>Installation on Ubuntu<\/strong><\/p>\n<pre>sudo apt install targetcli-fb -y\r\nsudo systemctl enable target --now\r\nsudo mkdir -p \/iscsi_disks\r\nsudo targetcli<\/pre>\n<p><strong>Walkthrough<\/strong><\/p>\n<pre>\/&gt; ls\r\n\/&gt; cd backstores\/fileio\r\n\/&gt; create disk01 \/iscsi_disks\/disk01.img 1G<\/pre>\n<pre>\/&gt; cd \/iscsi\r\n\/&gt; set discovery_auth userid=username\r\n\/&gt; set discovery_auth password=secret\r\n\/&gt; set discovery_auth enable=1<\/pre>\n<pre>\/&gt; create\r\n\/&gt; delete iqn.2003-01.org.linux-iscsi.u24.x8664:sn.0dcd420db0df\r\n\/&gt; create iqn.2003-01.lan.srv:disk01\r\n\/&gt; cd \/iscsi\/iqn.2003-01.lan.srv:disk01\/tpg1<\/pre>\n<pre>\/&gt; portals\/ create\r\n\/&gt; portals\/ delete 0.0.0.0 3260\r\n\/&gt; portals\/ create 192.168.10.10 3260\r\n\/&gt; luns\/ create \/backstores\/fileio\/disk01<\/pre>\n<pre>\/&gt; get attribute authentication\r\n\/&gt; set attribute authentication=1<\/pre>\n<pre>\/&gt; acls\/ create iqn.2003-01.lan.client\r\n\/&gt; cd acls\/iqn.2003-01.lan.client\/\r\n\/&gt; set auth userid=username\r\n\/&gt; set auth password=secret\r\n\/&gt; set auth mutual_userid=username\r\n\/&gt; set auth mutual_password=secret<\/pre>\n<hr \/>\n<p><strong>INITIATOR \/ CLIENT<\/strong><\/p>\n<p>On Linux hosts that use volumes or disks, <strong>iscsiadm<\/strong> is the tool used to discover and connect to storage resources.<\/p>\n<p><strong>Installation and Configuration on Ubuntu<\/strong><\/p>\n<pre>sudo apt install open-iscsi -y\r\nsudo iscsiadm -m discovery -t sendtargets -p 192.168.10.10\r\nsudo nano \/etc\/iscsi\/initiatorname.iscsi<\/pre>\n<p>Edit or paste the IQN.<\/p>\n<pre>InitiatorName=iqn.2003-01.lan.client<\/pre>\n<p>Rebooting after changing the IQN is optional but recommended.<\/p>\n<pre>#sudo reboot\r\nsudo nano \/etc\/iscsi\/iscsid.conf<\/pre>\n<p>Uncomment the relevant lines based on your authentication configuration on the target side. Remember to replace the values.<\/p>\n<pre># *************\r\n# CHAP Settings\r\n# *************\r\n## For Discovery Auth\r\ndiscovery.sendtargets.auth.authmethod = <strong>CHAP<\/strong>\r\ndiscovery.sendtargets.auth.username = <strong>username<\/strong>\r\ndiscovery.sendtargets.auth.password = <strong>secret<\/strong>\r\n## For 1-way Auth\r\nnode.session.auth.authmethod = <strong>CHAP<\/strong>\r\nnode.session.auth.username = <strong>username<\/strong>\r\nnode.session.auth.password = <strong>secret<\/strong>\r\n## For Mutual Auth\r\ndiscovery.sendtargets.auth.username_in = <strong>username<\/strong>\r\ndiscovery.sendtargets.auth.password_in = <strong>secret<\/strong><\/pre>\n<p><strong>Test it out<\/strong><\/p>\n<pre>sudo iscsiadm -m node --targetname iqn.2003-01.lan.srv:disk01 --portal 192.168.10.10:3260 --login\r\nsudo iscsiadm -m session\r\nsudo iscsiadm -m node -T iqn.2003-01.lan.srv:disk01 -p 192.168.10.10:3260 --logout\r\nsudo iscsiadm -m node -o delete -T iqn.2003-01.lan.srv:disk01 -p 192.168.10.10:3260<\/pre>\n<hr \/>\n<p><strong>BONUS<\/strong><\/p>\n<p>As an alternative to iSCSI, <strong>Network Block Device<\/strong> (NBD) offers a simpler and more versatile solution for accomplishing essentially the same goals [<a href=\"https:\/\/github.com\/NetworkBlockDevice\/nbd\">Link<\/a>]. It is a routable TCP-based protocol that runs on port 10809 by default.<\/p>\n<p>When would you choose <strong>NBD over iSCSI<\/strong> or other solutions?<\/p>\n<ul>\n<li>NBD has a simpler protocol than iSCSI, making it easier to use in software development. See <code>libnbd<\/code> [<a href=\"https:\/\/gitlab.com\/nbdkit\/libnbd\">Link<\/a>].<\/li>\n<li>It is fully integrated into QEMU (client and server), with native TLS encryption [<a href=\"https:\/\/www.qemu.org\/\">Link<\/a>].<\/li>\n<li>NFS does not allow the client to define the filesystem because it is not a block device.<\/li>\n<li>Fibre Channel over Ethernet is not routable, and ATA over Ethernet has not been actively developed for a long time.<\/li>\n<li>In summary, the right choice depends on your requirements.<\/li>\n<\/ul>\n<pre>sudo apt install nbd-client -y\r\nsudo nbd-client 10.0.0.10 -N data_drive_1 \/dev\/nbd0<\/pre>\n<p>On the server side, use <code>nbdkit<\/code>, a multithreaded NBD server implementation [<a href=\"https:\/\/gitlab.com\/nbdkit\/nbdkit\">Link<\/a>].<\/p>\n<pre>nbdkit file file=disk0.img\r\nnbdkit file -N data_drive_1 file=disk1.img\r\nnbdkit file -N data_drive_2 file=\/dev\/sdb\r\nnbdkit qemu disk=disk.qcow2\r\nnbdkit memory size=1G -i 10.0.0.10<\/pre>\n<p>Or to export multiple block devices at once:<\/p>\n<pre>nbdkit --exportname data_drive_1 file disk1.img --exportname data_drive_2 file disk2.img<\/pre>\n<p>Ideally, run it as a service managed by <code>systemd<\/code>.<\/p>\n<pre>\/etc\/systemd\/system\/nbdkit-disk0.service<\/pre>\n<pre>[Unit]\r\nDescription=NBD export disk0\r\n\r\n[Service]\r\nExecStart=\/usr\/bin\/nbdkit file file=\/srv\/nbd\/disk0.img\r\nUser=nbdkit\r\nGroup=nbdkit\r\n\r\n[Install]\r\nWantedBy=multi-user.target<\/pre>\n<pre>sudo systemctl daemon-reload\r\nsudo systemctl enable --now nbdkit-data1.service<\/pre>\n<p>Make sure the <code>nbdkit<\/code> user exists and has read\/write permission on the image file <code>\/srv\/nbd\/disk0.img<\/code>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>iSCSI (Internet Small Computer Systems Interface) is a protocol that allows SCSI commands to be [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[],"class_list":["post-4633","post","type-post","status-publish","format-standard","hentry","category-linux"],"_links":{"self":[{"href":"https:\/\/dft.wiki\/index.php?rest_route=\/wp\/v2\/posts\/4633","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dft.wiki\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dft.wiki\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dft.wiki\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dft.wiki\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4633"}],"version-history":[{"count":7,"href":"https:\/\/dft.wiki\/index.php?rest_route=\/wp\/v2\/posts\/4633\/revisions"}],"predecessor-version":[{"id":5616,"href":"https:\/\/dft.wiki\/index.php?rest_route=\/wp\/v2\/posts\/4633\/revisions\/5616"}],"wp:attachment":[{"href":"https:\/\/dft.wiki\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4633"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dft.wiki\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4633"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dft.wiki\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4633"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}