Architecture Guide for QuantaStor with VMware@SoftLayer

You can order and configure the OSNexus QuantaStor shared storage solution for a VMware ESXi 5 environment. Use the following information in conjunction with the Advanced Single-Site VMware Reference Architecture cookbook to set up one of these storage options in your VMware environment.

Step 1: Order QuantaStor Shared Storage

Before placing the order for storage, you will want to determine the necessary specifications that will meet your capacity and I/O requirements. These specifications include, but are not limited to, server RAM, type and number of disk drives, SSDs for caching, RAID configuration, and networking configuration. Reference the QuantaStor Solution Design Guide for Virtualization for more guidance on choosing the correct QuantaStor configuration in your environment.

For the example environment, we will be selecting a smaller configuration capable of delivering enough capacity and I/O needed for our virtual machines (VMs). Make sure you understand the capacity and I/O requirements of your VMs before placing your order. While the QuantaStor server is expandable, it is important that you size your hardware initially to avoid delays in configuration and deployment.

Ordering QuantaStor

Use the following steps to order a QuantaStor server.

  1. Log in to control.softlayer.com and click Account > Place an Order.
  2. Select Bare Metal Servers, Monthly on the pop-up screen.
  3. Select the appropriate server that has the ability to store the amount of disks needed from your environment from the Server List page. [For our environment, we will select a 12-core system (i.e., dual hex core) with 12 drive bays.]
  4. Enter the following configuration options:
    • Data Center: Location of the VLANs and ESXi hosts previously created
    • Server: Dual Processor Hex Core Xeon
    • RAM: 64 GB
    • Operating System: OSNexus QuantaStor 3.x (4TB)
    • Hard Drives:
      Disks 1 & 2: 500GB SATA in RAID 1
      Partition Template: Linux Basic
      Disks 3-10: 600GB SAS 15K in RAID-10
    • Public Bandwidth: Private Network Only
    • Uplink Port Speeds: 10Gbps Redundant Private Network Uplinks
  5. Click on the Continue Your Order button.

Note that we configured the storage server with two network interfaces that are unbonded so that we can use two different subnets to load balance traffic to the storage array.

Finish Configuration

You should now have a QuantaStor appliance in your shopping cart. You will need to assign the private VLAN, hostname, and domain to the device in order for the it to be provisioned correctly.

  1. Assign the following VLANs and create hostnames for the devices: QuantaStor host – Backend VLAN: (e.g., 1102 in our environment)
  2. Select your payment method, agree to the Terms and Conditions, and click the Finalize Your Order button.

Do not proceed to Step 3 until the server has been provisioned and is accessible via VPN or virtual server.

Step 2: Enable iSCSI Software Adapter for Management and Capacity Hosts

The iSCSI Software Adapter must be enabled on each management and capacity host and its iSCSI Qualified Name (IQN) collected before we can mount iSCSI volumes. Use the following steps to enable the iSCSI software adapter:

  1. Navigate to an ESXi management or capacity host and select Manage, Storage, Storage Adapters.
  2. Click on the green plus sign (+) and add the iSCSI Software Adapter.
  3. Click on the vmhba that corresponds to the iSCSI Software Adapter and record the iSCSI Name (Figure 1) in the Adapter Details section after it is enabled.
  4. Do steps 1 to 3 for each iSCSI Software Adapter on each ESXi management and capacity host.

Step 3: Configure QuantaStor

We are ready to set up our shared storage device once the server is provisioned. Specifically, we will set up networking and configure iSCSI volumes, and assign volumes to hosts. The iSCSI setup configuration will look similar to Figure 2.

Configure Networking

  1. Open a web browser and navigate to the QuantaStor IP address listed on the Devices page on control.softlayer.com.
  2. Enter the Admin Username and Password (found in the Passwords section on the Device Details page). Before proceeding, also note the private network devices that are utilized by the QuantaStor server, e.g. eth4 in Figure 3 below.
  3. Navigate to Storage System > Network Ports.
  4. Select the private adapter that is currently active (e.g., eth4) in the list of network adapters, right-click and select Modify Network Port from the pop-up menu.
  5. Uncheck the iSCSI Enabled checkbox to disable iSCSI connections to this adapter and click OK.
  6. Select the private adapter that is not currently active, but is assigned to the private network (e.g., eth6).
  7. Right-click on this adapter and select Modify Network Port from the pop-up menu.
  8. Select a Config Type of Static on the Modify Network Port screen
  9. Enter a primary private IP address, subnet, and gateway for this adapter. Use an address from the Storage row if you’re using the VLAN worksheet from the Advanced Single-Site VMware Reference Architecture cookbook.
  10. Uncheck the iSCSI Enabled checkbox to disable iSCSI connections to this adapter. (Figure 4 below)
  11. Right-click on the private adapter and select Enable Network Port to bring the adapter online after configuring it with an IP address.
  12. Click OK to enable the port upon verifying the IP address.

Figure 3

Figure 4

Open Ticket with SoftLayer

You will need to open a ticket with SoftLayer when done configuring the second adapter with a primary private IP address. Opening the ticket will help ensure the IP address you used will not be taken if another machine is provisioned on the VLAN.

  1. Click Support, Add Ticket and enter the following:
    • Subject: Private network question
    • Title: Please reserve and assign private IP address
    • Associate Devices: 'Select the QuantaStor server'
    • Details: Please reserve and assign on VLAN to . This IP address is being used for the second adapter on the QuantaStor server.

Configure QuantaStor

Now that the array has the ability to accept connections from both adapters, the adapters’ virtual interfaces that reside on the Storage Path A and Storage Path B subnets must be assigned.

  1. Right-click on the initial private adapter interface (e.g., eth4) and select Create Virtual Interface on the pop-up menu.
  2. Enter a portable private IP address and subnet for this adapter on the resulting pop-up screen, and make sure iSCSI Enabled is checked.
  3. Use an address from the Storage Path A row if you are using the VLAN worksheet.
  4. Record the IP address as used in the Notes section on the Portable IP Address page.
  5. Select the initial private adapter interface (e.g., eth4) to bind the virtual interface and click OK.
  6. Right-click on the other private adapter interface (e.g., eth6) and select Create Virtual Interface on the pop-up menu.
  7. Enter a portable private IP address and subnet for this adapter and make sure iSCSI Enabled in checked on the resulting pop-up screen.
  8. Use an address from the Storage Path B row if you are using the VLAN worksheet.
  9. Record this IP address as used in the Notes section on the Portable IP Address page.
  10. Select the other private adapter interface (e.g., eth6) to bind the virtual interface and click OK.

Once the QuantaStor server is configured with IP addresses and virtual interfaces, the routing needs to be configured to make sure the outgoing traffic uses the correct interface (because there are two NICs on different subnets).

  1. SSH into the QuantaStor server with your root credentials and append the following lines to /etc/network/interfaces (assuming eth4 and eth6 are the NICs on the private network):
    • post-up ip route add 10.0.0.0/8 via dev eth4
    • post-up ip route add 10.0.0.0/8 via dev eth6 table eth6
    • post-up ip rule add from table eth6

Figure 5

Create a Storage Pool

Next, we must create a storage pool that will be used to allocate volumes before we can create volumes or shares.

  1. Right-click on Storage Pools and select Create Storage Pool.
  2. Enter the following on the Create Storage Pool screen:
    • Name: A name for the storage pool, e.g., StoragePool-01
    • Pool Type: Default (zfs)
    • I/O Profile: Virtualization
    • Disks to Utilize for the Storage Pool: sdb
  3. Click OK.

Figure 6

If sdb is not available to add to a pool, it is due to a partition existing and being tagged as bootable. You will need to utilize gdisk on the Quantastor console to remove the partition and any /etc/fstab entries. Next, we will create volumes for ESXi to consume.

Create iSCSI Storage Volumes

We will create two storage volumes - one for management VMs on the management cluster and one for VMs on the capacity cluster. Use the following steps to create the iSCSI storage volumes:

  1. Right-click on Storage Volumes and select Create Storage Volume on the pop-up menu.
  2. Enter the information for the storage volumes. (Note: While your configuration may differ depending on workload and storage capacity, we chose the values in Table 1 and Table 2 for each storage volume.

Management Volume

Field Value
Name Mgmt-LUN0
Storage Pool <Storage Pool Name configured in the previous step>
Size 500GB
% Reserved 50
Compression Enabled

Capacity Volume

Field Value
Name Capacity-LUN0
Storage Pool <Storage Pool Name configured in the previous step>
Size 3TB
% Reserved 50
Compression Enabled

Assign Host Access to Volumes

QuantaStor needs to be configured to allow access from the ESXi hosts via each host’s IQN once the volumes have been set up.

  1. Navigate to the QuantaStor administration page and right-click on the Hosts menu and select Add Host.
  2. Enter the following on the Add Host screen:
    • Host Name: A host name; it does not have the fully qualified domain name (FQDN), but is should describe the host, e.g., MyESXiHostName.
    • Operating System Type: VMware
    • Initator: iSCSI Qualified Name (IQN) radio button
    • iSCSI Qualified Name: The IQN for the respective host, e.g., IQNofESXIHost.
  3. Click OK.

Do this for each management and capacity host in ESXi environment.

 

After adding each host in the management and capacity clusters: 

  1. Right-click on the Host Groups menu and select Create Host Group….
  2. Enter ManagementCluster in the Name field and select all the hosts in the management cluster.
  3. Click OK. A host group will be created that we can assign to a particular volume. 

Repeat this process for the capacity cluster.

  1. Click on the Storage Volumes menu.
  2. Right-click on the Mgmt-Lun0 volume and select Assign/Unassign Host Access.
  3. Confirm Mgmt-Lun0 is an option in the drop-down menu and select the host group that you created in the previous step. This now allows each ESXi host in the management cluster to access the Mgmt-Lun0 volume. Do the same for Capacity-LUN0 

Step 4: Mount Volume(s) on Management / Capacity Clusters 

Log in to the vSphere Web Client and navigate to Hosts under the vCenter Inventory Lists

Use the following steps to mount the volume on the ESXi hosts:

  1. Select a host and click Manage, Storage, and Storage Adapters.
  2. Select Targets, click Dynamic Discovery, and click the Add… button.
  3. Enter the IP address assigned to the QuantaStor storage device on Storage Path A on the Add Send Target Server screen.
  4. Leave 3260 as the Port and click OK.
  5. Click Dynamic Discovery, clickthe Add… button again. Repeat steps 4 and 5 for Storage Path B. 

The ESXI host is ready to rescan the iSCSI Software Adapter to discover the Mgmt-Lun0 volume. 

  1. Select the Rescan Storage Adapters icon (Figure 9) on the Storage Adapters screen.
  2. Leave the default option on the resulting pop-up screen and click OK.
  3. Click Actions, New Datastore once the volume has been discovered and format it as a VMFS-5.
  4. Confirm the formatting is complete and click Storage Devices, Device Details, Edit Multipathing….
  5. Select Round Robin for the Path Selection Policy and click OK.

Now that the Mgmt-Lun0 volume is attached to a single host, you must go back to the other management hosts in the cluster and repeat the process to add the volume. However, you do not need to format the volume as it already formatted with VMFS-5 and will be displayed as such after discovery.

Step 5: Disable Delayed ACK in vSphere ESXi hosts

The delayed ACK needs to be disabled once the storage LUNs have been attached to the management and capacity hosts. 

  1. Go to the vSphere environment and select Storage Adapters, iSCSI Software Adapter, Advanced Options. 
  2. Click the Edit and scroll all the way to the bottom of the Advanced Options screen.
  3. Uncheck the DelayedAck checkbox, and click OK. 

Visit KB on VMware’s site for more details on delayed ACK with respect to VMware.

You can now return to Advanced Single-Site VMware Reference Architecture cookbook and complete the environment setup.