{"id":4838,"date":"2025-06-01T13:35:04","date_gmt":"2025-06-01T17:35:04","guid":{"rendered":"https:\/\/dft.wiki\/?p=4838"},"modified":"2026-06-08T09:45:55","modified_gmt":"2026-06-08T13:45:55","slug":"ceph-cheat-sheet","status":"publish","type":"post","link":"https:\/\/dft.wiki\/?p=4838","title":{"rendered":"CEPH Deployment and Concepts"},"content":{"rendered":"<p><strong>Ceph<\/strong> is an open-source, distributed storage system designed to provide high performance, reliability, and scalability. It supports object, block, and file storage in a unified platform, though not all storage types are available out of the box (more on that later) [<a href=\"https:\/\/ceph.io\/\">Link<\/a>] [<a href=\"https:\/\/github.com\/ceph\/ceph\">Link<\/a>].<\/p>\n<p>Ceph differs from the traditional model of large (usually proprietary) storage appliances and the Client-Server architecture by leveraging Layer 3 networks to implement a Client-Cluster model instead.<\/p>\n<p>Unlike ZFS and NFS, which are centralized and contained within a single appliance, Ceph coordinates between multiple devices and nodes. While Ceph is POSIX compliant, NFS is not fully compliant, though it handles most workloads without issue.<\/p>\n<p>Core concepts and nomenclature:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-4853\" src=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/ceph-diagram-1.drawio.png\" alt=\"\" width=\"711\" height=\"156\" srcset=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/ceph-diagram-1.drawio.png 711w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/ceph-diagram-1.drawio-300x66.png 300w\" sizes=\"auto, (max-width: 711px) 100vw, 711px\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>Refer to the official documentation for more details [<a href=\"https:\/\/docs.ceph.com\/en\/latest\/architecture\/\">Link<\/a>].<\/p>\n<ul>\n<li><strong>OSD<\/strong>\n<ul>\n<li>Object Storage <span style=\"text-decoration: underline;\">Device<\/span>\n<ul>\n<li>Each discrete HDD or SSD.<\/li>\n<\/ul>\n<\/li>\n<li>Object Storage <span style=\"text-decoration: underline;\">Daemon<\/span>\n<ul>\n<li>A process that manages the Object Storage <span style=\"text-decoration: underline;\">Device<\/span>.<\/li>\n<\/ul>\n<\/li>\n<li>This is where data is actually stored.<\/li>\n<\/ul>\n<\/li>\n<li><strong>MON<\/strong>\n<ul>\n<li>Monitors OSDs and controls authentication.<\/li>\n<\/ul>\n<\/li>\n<li><strong>RADOS<\/strong>\n<ul>\n<li>Manages OSDs.<\/li>\n<\/ul>\n<\/li>\n<li><strong><span style=\"color: #777777; font-size: 1rem;\">LIBRADOS<\/span><\/strong>\n<ul>\n<li><span style=\"color: #777777; font-size: 1rem;\">LIBRADOS APIs are the native interface for communicating directly with RADOS.<\/span><\/li>\n<li><span style=\"color: #777777; font-size: 1rem;\">Client applications must use LIBRADOS libraries [<a href=\"https:\/\/docs.ceph.com\/en\/latest\/rados\/api\/librados-intro\/\">Link<\/a>].<\/span><\/li>\n<\/ul>\n<\/li>\n<li><strong><span style=\"color: #777777; font-size: 1rem;\">GATEWAY<\/span><\/strong>\n<ul>\n<li><span style=\"color: #777777; font-size: 1rem;\">Exports Ceph as iSCSI, SMB, NFS, S3, etc. using LIBRADOS under the hood.<\/span><\/li>\n<li>Client applications are unaware that Ceph is running behind the scenes.<\/li>\n<\/ul>\n<\/li>\n<li><strong>RBD<\/strong>\n<ul>\n<li>RADOS Block Devices.<\/li>\n<\/ul>\n<\/li>\n<li><strong><span style=\"color: #777777; font-size: 1rem;\">MDS<\/span><\/strong>\n<ul>\n<li><span style=\"color: #777777; font-size: 1rem;\">Metadata Services for the Ceph File System (not used by Block Devices or Object Store).<\/span><\/li>\n<li><span style=\"color: #777777; font-size: 1rem;\">Offloads OSDs from the task of indexing file metadata.<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>How is data allocated?<\/p>\n<ul>\n<li><strong><span style=\"color: #777777; font-size: 1rem;\">Pool<\/span><\/strong>\n<ul>\n<li><span style=\"color: #777777; font-size: 1rem;\">Each Pool is mapped to a group of disks.<\/span>\n<ul>\n<li>Replicated &#8211; defines the number of replicas\/copies of each object, similar to RAID 1.\n<ul>\n<li>A replica count of 3 means 200% overhead.<\/li>\n<\/ul>\n<\/li>\n<li>Erasure Coding &#8211; stripes the data and creates encoded redundancy parity. Redundancy of 1 and 2 are equivalent to RAID 5 and 6, respectively.\n<ul>\n<li>A <strong>4<\/strong>-stripe with parity of <strong>2<\/strong> is called <strong>4+2<\/strong>, and results in 50% overhead.<\/li>\n<li>Storage efficiency comes at the cost of CPU overhead.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<li><strong><span style=\"color: #777777; font-size: 1rem;\">PG<\/span><\/strong>\n<ul>\n<li><span style=\"color: #777777; font-size: 1rem;\">Placement Groups are mapped to OSDs.<\/span><\/li>\n<li>The general rule of thumb from Ceph documentation is approximately 100 PGs per OSD.<\/li>\n<li>Starting with Nautilus, PGs can be merged or split, enabling the PG auto-scaler (recommended).<\/li>\n<\/ul>\n<\/li>\n<li><strong>Objects<\/strong>\n<ul>\n<li>By default, objects are 4 MB fragments of data sets (e.g., files, block devices, etc.).<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>So, <strong>Files<\/strong> are split into <strong>Objects<\/strong>, which are sent to a <strong>PG<\/strong> (within a <strong>Pool<\/strong>) to be distributed across its <strong>OSDs<\/strong>.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-4854\" src=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/ceph-disgram-2.drawio.png\" alt=\"\" width=\"553\" height=\"340\" srcset=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/ceph-disgram-2.drawio.png 553w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/ceph-disgram-2.drawio-300x184.png 300w\" sizes=\"auto, (max-width: 553px) 100vw, 553px\" \/><\/p>\n<p>&nbsp;<\/p>\n<ul>\n<li>KVM virtualization uses the RBD library for direct communication with OSDs and MONs.<\/li>\n<\/ul>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-4855\" src=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/ceph-diagram-3.drawio.png\" alt=\"\" width=\"170\" height=\"272\" \/><\/p>\n<ul>\n<li>When using LIBRADOS, client applications can fetch the Cluster Map from the MONs and communicate directly with the OSDs responsible for the data.<\/li>\n<\/ul>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-4862\" src=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/ceph-diagram-4.drawio.png\" alt=\"\" width=\"320\" height=\"264\" srcset=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/ceph-diagram-4.drawio.png 320w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/ceph-diagram-4.drawio-300x248.png 300w\" sizes=\"auto, (max-width: 320px) 100vw, 320px\" \/><\/p>\n<hr \/>\n<p><strong>FACTS OF CEPH<\/strong><\/p>\n<p>When a Pool reaches 85% capacity, a warning is raised.<\/p>\n<p>If utilization reaches 90%, the volume becomes read-only until additional OSDs are added.<\/p>\n<p>If an OSD Daemon stops communicating with its peers, it takes 5 minutes for the quorum to consider it offline. Only then does self-healing begin, with no human intervention required.<\/p>\n<p>For data consistency, a client write is only acknowledged once all copies have been written. The client sends data to a single OSD over the Public Network, and that OSD is responsible for replicating it to the other OSDs over the Internal Network.<\/p>\n<p>Docker or Podman must be available on all nodes, as Ceph processes (OSDs, MONs, etc.) run as containerized applications.<\/p>\n<hr \/>\n<p><strong>DESIGNING A CLUSTER<\/strong><\/p>\n<ul>\n<li>MONs can coexist with OSDs in small and medium-sized clusters.\n<ul>\n<li>As a rule of thumb, this works for up to 10 to 16 nodes, depending on the number of OSDs per node.<\/li>\n<\/ul>\n<\/li>\n<li>Gateways should have dedicated hardware.\n<ul>\n<li>Gateways require high throughput due to their role as traffic forwarders, similar to a proxy.<\/li>\n<\/ul>\n<\/li>\n<li>Public (Front) Network and Internal (Back) Network\n<ul>\n<li>Do not share the Front and Back networks, as doing so will severely impact throughput and latency.<\/li>\n<li>The Back network should be at least as fast as the Front network to avoid becoming a bottleneck for client traffic.<\/li>\n<\/ul>\n<\/li>\n<li>Failure Domains\n<ul>\n<li>Each Failure Domain (e.g., a rack) should be self-sufficient, containing its own MONs, MGRs, MDSs, Gateways, etc.<\/li>\n<\/ul>\n<\/li>\n<li>Resource Allocation per Node (Bare Metal Host)\n<ul>\n<li>Reserve 4 CPU cores and 16 GB of RAM for the OS.<\/li>\n<li>The remaining resources can be allocated as follows:\n<ul>\n<li>1 CPU core (or thread) per OSD.<\/li>\n<li>5 GB of RAM per OSD.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Note:<\/strong> each HDD\/SSD will have its own OSD Daemon, so the calculation above determines the maximum optimal number of drives per node.<\/li>\n<li>If MONs coexist with OSDs, account for their resource usage in the calculation above.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>Capacity Calculation by Node Resources<\/p>\n<p><input id=\"cpu\" type=\"number\" value=\"64\" data-ddg-inputtype=\"unknown\" \/> Cores \/ Threads<br \/>\n<input id=\"ram\" type=\"number\" value=\"256\" data-ddg-inputtype=\"unknown\" \/> GB of RAM<br \/>\nOperating System Reserved<br \/>\n<input id=\"cpures\" type=\"number\" value=\"4\" data-ddg-inputtype=\"unknown\" \/> Cores \/ Threads<br \/>\n<input id=\"ramres\" type=\"number\" value=\"16\" data-ddg-inputtype=\"unknown\" \/> GB of RAM<\/p>\n<p>Number of Processes: <b><span id=\"maxosds\">15<\/span> OSDs<\/b><\/p>\n<p><script>\nfunction recalculateMaxOSDs() {\n  const cpu = parseFloat(document.getElementById('cpu').value) || 1;\n  const cpures = parseFloat(document.getElementById('cpures').value) || 1;\n  const cpu_maxosds = Math.floor((cpu - cpures) \/ 4);  \n  const ram = parseFloat(document.getElementById('ram').value) || 1;\n  const ramres = parseFloat(document.getElementById('ramres').value) || 1;\n  const ram_maxosds = Math.floor((ram - ramres) \/ 4);  \n  document.getElementById('maxosds').textContent = cpu_maxosds >= ram_maxosds ? ram_maxosds : cpu_maxosds ;\n}\n['cpu', 'cpures', 'ram', 'ramres'].forEach(id => {\n  document.getElementById(id).addEventListener('input', recalculateMaxOSDs);\n});\n<\/script><\/p>\n<p><strong>Note:<\/strong> the value above is a guide for properly sizing a node. Decisions on how much to under- or over-provision require further evaluation of the hardware and environment.<\/p>\n<hr \/>\n<p><strong>CEPH DEPLOYMENT<\/strong><\/p>\n<p>If you find documentation or tutorials that reference the <code>ceph-deploy<\/code> tool, note that it is deprecated and has been replaced by <code>cephadm<\/code>.<\/p>\n<p><strong>On Debian 12, fresh install nodes.<\/strong><\/p>\n<p>Make sure <span style=\"text-decoration: underline;\">all hosts<\/span> are up to date, have all dependencies installed, and have their clocks synchronized with <code>chrony<\/code> or <code>ntpd<\/code>.<\/p>\n<pre>sudo su -\r\napt update &amp;&amp; apt upgrade -y\r\napt install gpg chrony openssh-server lvm2 curl nano docker.io -y<\/pre>\n<p>Optionally, make all nodes aware of each other by editing <code>\/etc\/hosts<\/code> or by using a DNS server.<\/p>\n<pre>10.1.1.11 ceph-node-1\r\n10.1.1.12 ceph-node-2\r\n10.1.1.13 ceph-node-3\r\n10.1.1.14 ceph-node-4\r\n10.1.1.15 ceph-node-5<\/pre>\n<p>On the <span style=\"text-decoration: underline;\">admin host<\/span>, install <code>cephadm<\/code>.<\/p>\n<pre>apt install lsb-release gnupg -y\r\ncurl --silent --remote-name https:\/\/download.ceph.com\/keys\/release.asc\r\ngpg --no-default-keyring --keyring .\/ceph-release.gpg --import release.asc\r\ngpg --no-default-keyring --keyring .\/ceph-release.gpg --export &gt; \/etc\/apt\/trusted.gpg.d\/ceph.gpg\r\necho deb https:\/\/download.ceph.com\/debian-squid\/ $(lsb_release -sc) main | tee \/etc\/apt\/sources.list.d\/ceph.list\r\napt update &amp;&amp; apt install cephadm -y<\/pre>\n<p><strong>Note:<\/strong> not all versions of Debian or Ubuntu have pre-built packages in the repository. As of this writing (June 2025), there is no <code>noble<\/code> build available, so use <code>focal<\/code> instead. Check for the latest release [<a href=\"https:\/\/docs.ceph.com\/en\/latest\/releases\/index.html\">Link<\/a>], then confirm packages are available for your distribution [<a href=\"https:\/\/download.ceph.com\/debian-squid\/dists\/\">Link<\/a>].<\/p>\n<p>Bootstrap the Ceph Cluster:<\/p>\n<pre>cephadm bootstrap --mon-ip <strong>192.168.1.11<\/strong> --cluster-network <strong>10.1.1.0\/24<\/strong> --ssh-user root<\/pre>\n<p><strong>Note:<\/strong> the Public network, where clients access Ceph resources and services, is <strong>10.10.10.0\/24<\/strong>. The Internal network, <strong>10.1.1.0\/24<\/strong>, is used exclusively for communication between nodes, specifically for OSD data replication.<\/p>\n<p>Sample output, including the UI credentials and instructions for opening the shell.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-4863\" src=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/ceph-output-e1748994955767.png\" alt=\"\" width=\"800\" height=\"392\" srcset=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/ceph-output-e1748994955767.png 938w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/ceph-output-e1748994955767-300x147.png 300w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/ceph-output-e1748994955767-768x377.png 768w\" sizes=\"auto, (max-width: 800px) 100vw, 800px\" \/><\/p>\n<p>Before opening the Dashboard, copy the generated SSH public key.<\/p>\n<pre>cat \/etc\/ceph\/ceph.pub<\/pre>\n<p>Then paste it into the root authorized keys file on all other nodes.<\/p>\n<pre>sudo nano \/root\/.ssh\/authorized_keys<\/pre>\n<hr \/>\n<p><strong>WEB-UI SETUP<\/strong><\/p>\n<p>Navigate to <span style=\"text-decoration: underline;\"><strong>https:\/\/10.10.10.11:8443<\/strong><\/span> to complete the setup. Use the credentials generated in the previous step.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-4865\" src=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-02_17-30-53.png\" alt=\"\" width=\"600\" height=\"145\" srcset=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-02_17-30-53.png 995w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-02_17-30-53-300x73.png 300w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-02_17-30-53-768x186.png 768w\" sizes=\"auto, (max-width: 600px) 100vw, 600px\" \/><\/p>\n<p>You will be prompted to change the password.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-4867\" src=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-02_17-33-18.png\" alt=\"\" width=\"600\" height=\"154\" srcset=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-02_17-33-18.png 837w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-02_17-33-18-300x77.png 300w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-02_17-33-18-768x197.png 768w\" sizes=\"auto, (max-width: 600px) 100vw, 600px\" \/><\/p>\n<p>After signing in, an Expand Cluster button will appear. This wizard will guide you through the setup process.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-large wp-image-4879\" src=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20-01-02-1024x610.png\" alt=\"\" width=\"640\" height=\"381\" srcset=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20-01-02-1024x610.png 1024w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20-01-02-300x179.png 300w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20-01-02-768x457.png 768w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20-01-02.png 1345w\" sizes=\"auto, (max-width: 640px) 100vw, 640px\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>The first step is to add nodes to the cluster.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-4880\" src=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20-01-21.png\" alt=\"\" width=\"800\" height=\"364\" srcset=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20-01-21.png 1082w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20-01-21-300x136.png 300w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20-01-21-1024x466.png 1024w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20-01-21-768x349.png 768w\" sizes=\"auto, (max-width: 800px) 100vw, 800px\" \/><\/p>\n<p>In this example, Nodes 2 through 5 were assigned the MON and OSD roles.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-4881\" src=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20-02-35.png\" alt=\"\" width=\"800\" height=\"459\" srcset=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20-02-35.png 1085w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20-02-35-300x172.png 300w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20-02-35-1024x587.png 1024w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20-02-35-768x440.png 768w\" sizes=\"auto, (max-width: 800px) 100vw, 800px\" \/><\/p>\n<p>Node 1 was then edited to include all roles except OSD and _no_schedule.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-4884\" src=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20_16_17.png\" alt=\"\" width=\"800\" height=\"580\" srcset=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20_16_17.png 1067w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20_16_17-300x217.png 300w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20_16_17-1024x742.png 1024w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20_16_17-768x556.png 768w\" sizes=\"auto, (max-width: 800px) 100vw, 800px\" \/><\/p>\n<p>On the following steps, select the options that best suit your requirements.<\/p>\n<p>&nbsp;<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-4885\" src=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20_16_39.png\" alt=\"\" width=\"800\" height=\"390\" srcset=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20_16_39.png 1081w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20_16_39-300x146.png 300w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20_16_39-1024x499.png 1024w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20_16_39-768x374.png 768w\" sizes=\"auto, (max-width: 800px) 100vw, 800px\" \/><\/p>\n<p>In this example, no changes were made.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-4886\" src=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20_16_47.png\" alt=\"\" width=\"800\" height=\"336\" srcset=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20_16_47.png 1081w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20_16_47-300x126.png 300w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20_16_47-1024x430.png 1024w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20_16_47-768x323.png 768w\" sizes=\"auto, (max-width: 800px) 100vw, 800px\" \/><\/p>\n<p>Click Expand Cluster to complete the cluster creation.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-4887\" src=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20_17_05.png\" alt=\"\" width=\"800\" height=\"636\" srcset=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20_17_05.png 1080w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20_17_05-300x239.png 300w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20_17_05-1024x814.png 1024w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20_17_05-768x611.png 768w\" sizes=\"auto, (max-width: 800px) 100vw, 800px\" \/><\/p>\n<p>It may take several minutes for all components to be deployed and reach a healthy state.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-4888\" src=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20-43-39.png\" alt=\"\" width=\"800\" height=\"755\" srcset=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20-43-39.png 1090w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20-43-39-300x283.png 300w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20-43-39-1024x967.png 1024w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-06-03_20-43-39-768x725.png 768w\" sizes=\"auto, (max-width: 800px) 100vw, 800px\" \/><\/p>\n<p>The graph below shows internal network (10.1.1.0\/24) traffic during a data rebalance, which occurs when a failed drive is replaced or a new host is added to the cluster.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-5128\" src=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-08-06_21-28-48.png\" alt=\"\" width=\"800\" height=\"94\" srcset=\"https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-08-06_21-28-48.png 1116w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-08-06_21-28-48-300x35.png 300w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-08-06_21-28-48-1024x120.png 1024w, https:\/\/dft.wiki\/wp-content\/uploads\/sites\/15\/2025\/06\/Screenshot_2025-08-06_21-28-48-768x90.png 768w\" sizes=\"auto, (max-width: 800px) 100vw, 800px\" \/><\/p>\n<p><strong>Note:<\/strong> no interactivity or intervention is required. This all happens silently and autonomously.<\/p>\n<hr \/>\n<p>In the next post, I will walk through setting up volumes, services, and accessing the storage remotely.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Ceph is an open-source, distributed storage system designed to provide high performance, reliability, and scalability. [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4,1],"tags":[],"class_list":["post-4838","post","type-post","status-publish","format-standard","hentry","category-linux","category-ccna"],"_links":{"self":[{"href":"https:\/\/dft.wiki\/index.php?rest_route=\/wp\/v2\/posts\/4838","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dft.wiki\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dft.wiki\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dft.wiki\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dft.wiki\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4838"}],"version-history":[{"count":35,"href":"https:\/\/dft.wiki\/index.php?rest_route=\/wp\/v2\/posts\/4838\/revisions"}],"predecessor-version":[{"id":5608,"href":"https:\/\/dft.wiki\/index.php?rest_route=\/wp\/v2\/posts\/4838\/revisions\/5608"}],"wp:attachment":[{"href":"https:\/\/dft.wiki\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4838"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dft.wiki\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4838"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dft.wiki\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4838"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}