VMware VSAN & Boot Devices considerations

Recently, I have been engaged in multiple VSAN HCI projects designs. One of the most interesting decisions to take was the kind of boot devices that will hold the ESXi installation as this will introduce a bunch of discussions that have to be taken into consideration in the design phase.

In this post, I will summarize what are the available options along with the considerations to be taken.

Host Memory <=512GB

If the memory of the ESXi host has 512 GB of memory or less, you can boot the host from a USB, SD, or SATADOM device. When you boot a vSANhost from a USB device or SD card, the size of the boot device must be at least 4 GB (8GB recommended).

You can boot the host from disks as well. From a capacity and endurance standpoint, this meets the requirements to store logs, traces and dumps but you end up losing a drive slot.

Host Memory >512GB

You can boot the host from a SATADOM or disk device with a size of at least 16 GB. When you use a SATADOM device, use a single-level cell (SLC) device.

If you are using vSAN 6.5 or later, you must resize the coredump partition on ESXi hosts to boot from USB/SD devices. For more information, see the VMware knowledge base article at http://kb.vmware.com/kb/2147881.

If you want to boot from disks, all logs and dumps are retained in this case. The recommended configuration to gain the best performance in the case is to have:

One storage controller operating in RAID mode (RAID1 disks for ESXi installation).
Another controller operating in HBA mode (pass-through disks for VSAN).

Log Considerations

When you boot ESXi from a USB or SD device, log information and stack traces are lost on host reboot.

Consider the following options for persistent log storage:

Redirect logs to a syslog server such as vRealize log insight (VRLI). NOTE: all users with a supported vCenter Server license are entitled to a 25-OSI pack of vRealize Log Insight for vCenter Server, at no charge
Core dumps should be redirected to a core dump collector such as vCenter server.

Do not store log information on the vSAN datastore. This configuration is not supported because a failure in the vSAN cluster could impact the accessibility of log information.

Comparison of Boot Devices for Virtual SAN