Install FreeNAS on VMware Fusion for use in your vSphere Virtual Lab

This guide will show you how to create a FreeNAS VM with VMware Fusion and configure it to work with ESXi so that you can test features like VMotion in your virtual lab. We will also setup multipathing and make sure failover works when an iSCSI NIC or switch fails.

Overview

In this tutorial, we will use VMware Fusion* to demonstrate how FreeNAS and ESXi can be configured in a production environment.

We will configure the iSCSI networks so that there are multiple redundant paths from the ESXi host to the FreeNAS server. This will allow VMs located on the FreeNAS datastore to stay running when an iSCSI switch or network adapter fails.

The diagram below shows the network topology we will setup.

Notice if we take out the switch labeled iscsi-1, there is still a path from esxi01 to the freenas server. We will demonstrate this at the end by disabling an iscsi network adapter on either the ESXi or FreeNAS VM.

Prerequisites

Since we will be using nested virtualization to run ESXi inside a VM, you’ll need a modern Mac which has a CPU that supports hardware virtualizaion and has at least 16 GB of RAM. You’ll also need to download and install VMware Fusion* because it allows the hardware assisted virtualization feature of the CPU to be used in a virtual machine.

The following steps have been tested on a 2017 MacBook Pro that has 16 GB of RAM running macOS Catalina and VMware Fusion 11.5*.

Software required for this tutorial:

• VMware Fusion 11.5 - You can purchase it from the VMware Store* or download the free trial.
• VMware vSphere ESXi 6.7 or 7.0 - You can get it by purchasing the vSphere Essentials Kit* or by downloading the free trial.
• FreeNAS-11.3 - You can download it for free from here.

Okay, so you’ve got a modern Mac, installed VMware Fusion* and downloaded FreeNAS and ESXi, we’re now ready to create the custom VMware Fusion networks then build our FreeNAS VM.

Create VMware Fusion Custom Networks

In the following steps we will be using VMware Fusion Pro* to configure three custom networks. If you haven’t got the professional version, you can edit the config file to create the networks. See this tutorial to learn how.

Click VMware Fusion, Preferences then click the Network tab. Click the lock icon so that you can make changes then click on the + to add a new custom network. Call it vSphere, enable NAT, disable DHCP and assign the Subnet IP as 10.1.1.0.

Create another called iscsi-1 with NAT disabled and a Subnet IP of 10.10.1.0.

Create another called iscsi-1 with NAT disabled and a Subnet IP of 10.10.1.0.

Notice the networks we just created have the same names and subnets as the switches in the network diagram in the overview.

If we were to set this topology up in production, the custom networks would be the physical switches. However, you’d probably only use two switches and configure the iSCSI networks so they use VLANS.

Create FreeNAS VM

Okay, that’s the network configured. Let’s create the FreeNAS VM by following the steps below:

1. Open VMware Fusion*, click File then New…
2. Select Create a custom virtual machine then click Continue.
3. Choose Other, FreeBSD 11 64-Bit then click Continue.
4. Select Legacy BIOS then click Continue.
5. Select Create a new virtual disk then click Continue.
6. Click Customize Settings.
7. Choose a location to save the VM, enter a name, e.g freenas and click Save.

After clicking Save the VM should be created and the Settings screen will load.

From here we need to configure the virtual hardware before powering on the VM for the first time. We will assign more memory, add an extra network adapter, attach the FreeNAS ISO, and configure the network adapters to use the custom VMware Fusion networks we created at the start.

Configure Processor & Memory

1. Click Processors & Memory.
2. For the Processor assign 2 processor cores.
3. For Memory assign 8192 MB (8 GB).
4. Click Show All to save processor and memory settings.

Configure Network Adapters

1. Click on the Network Adapter.
2. Connect the Network Adapter to the Custom Network vSphere.
3. Click Add Device…
4. Select Network Adapter then click Add…
5. Connect the Network Adapter to the Custom Network iscsi-1.
6. Repeat steps 3, 4 and 5, choosing iscsi-2 for the third adapter.

Add Extra Storage

We need to add at least one more extra hard disk to the VM that will be used to create the FreeNAS storage pool. FreeNAS will be installed on the first disk and data will be installed on the extra disks. For this tutorial we will create two extra 40 GB disks and we will configure them later as a mirror volume.

1. Click Add Device…
2. Select New Hard Disk then click Add…
3. Change the disk size to 40 GB then click Apply.
4. Repeat steps 1 to 3 so that you have two 40 GB hard disks.

Connect FreeNAS ISO

The last thing we need to do before we can power the VM on is attach the FreeNAS ISO with the following steps.

1. Click CD/DVD (IDE).
2. Tick the Connect CD/DVD Drive box.
3. Select Choose disk or a disc image…, browse to the FreeNAS ISO located on your Mac then click Open.

Close the Settings screen. We’re now ready to install FreeNAS.

Install FreeNAS

Double click on the FreeNAS VM to start booting it up.

Select Install/Upgrade then OK.

Select the first hard disk by pressing Space then press OK.

Say Yes to the warning message about erasing data on the drive.

Enter the root password.

Select Boot via BIOS.

Wait for the install to finish.

Click OK on the succeeded message.

Select Reboot System then OK.

FreeNAS will reboot to the console setup screen.

Configure FreeNAS Management Network

From the console setup screen, you will notice the IP address is 0.0.0.0. This is because we are using a custom VMware Fusion network that doesn’t have DHCP enabled. In the following steps, we will assign a static IP address so we can connect to the web user interface.

1. Go through the options in section 1) Configure Network Interfaces to assign a static IP of 10.1.1.201 to the first interface (em0).
2. Go through the options in section 4) Configure Default Route to assign 10.1.1.2 as the gateway to the internet.
3. Go through the options in section 6) Configure DNS to assign 10.1.1.2 as the DNS server.

Once finished, you should see the web user interface address has changed to http://10.1.1.201 as seen in the following screenshot.

Now you should be able to ping the FreeNAS VM from your Mac.

We’re now ready to login to the web interface and finish the FreeNAS configuration.

Login to FreeNAS

Open a web browser and navigate to http://10.1.1.201. Login to the root account using the password you set when installing FreeNAS.

Click on Network then Interfaces. Edit em1 and assign a static IP of 10.10.1.201. Edit em2 and assign it a static IP of 10.10.2.201.

After applying the changes, you should be able to ping both addresses from your Mac.

Create a Storage Pool

Click the Storage menu option, then click on Pools. Click ADD to start the pool creation wizard.

Select Create new pool then click the CREATE POOL button.

Enter a name for the pool, eg pool then click the SUGGEST LAYOUT button. Since we only have two disks, the wizard will create a mirror pool (RAID-1).

Scroll to the bottom and click CREATE.

Check the Confirm box and click CREATE POOL.

With the pool now created. We need to configure iSCSI and create our datastore that will be used by VMware ESXi to store VMs.

Configure iSCSI

Click the Services menu option then scroll to iSCSI.

Flick the switch to enable iSCSI, tick the checkbox to make it start with the system, then click the pencil icon to edit the settings.

Click the WIZARD button.

For step one, we need to create a block device and store it on the pool we created earlier. The following example shows a device named datastore01 which is 20 GiB in size. Fill out the details then click NEXT.

Click Create New for the portal, select the IP 10.10.1.201 for the first IP, click ADD to create another, then select 10.10.2.201 for the second IP. Click NEXT.

Click NEXT again.

Click SUBMIT

Everything is now configured on the FreeNAS side. After configuring ESXi, we should be able to see a disk which is just less than 20 GB in size. We will see this in a later step, once we configure ESXi.

Create ESXi VM

The next thing we need to do is create an ESXi 6.x or 7 VM that has three network adapters. The first should be connected to the vSphere custom network. The second should be connected to the iscsi-1 network, and the third should be connected to the iscsi-2 network.

I’ve already written a tutorial on how to install ESXi on VMware Fusion. After following the steps in the tutorial, you should have an ESXi 6.x or 7 VM up and running with a management IP address of 10.1.1.11. Don’t forget to add the extra network adapters to the VM and assign them to the iscsi networks. See images below:

Configure ESXi Networks

Now that we have an ESXi VM up and running, we will configure the iSCSI networks so that there are multiple paths to the FreeNAS datastore, so that if one fails, the other will takeover immediately.

Let’s start by logging into ESXi by navigating to https://10.1.1.11 and entering the root login credentials.

Once logged in, click Networking, Virtual Switches then Add standard virtual switch.

Give the vSwitch a name of iscsi-1, select vmnic1 for the uplink then click Add.

Add another vSwitch, this time call it iscsi-2, select vmnic2 as the uplink and click Add.

Click on the VMkernel NICs tab then click Add VMkernel NIC.

Give the port group a name of iscsi-1, select iscsi-1 for the Virtual switch, assign a static IP of 10.10.1.11 then click Create.

Click Add VMkernel NIC.

Give the port group a name of iscsi-2, select iscsi-2 for the Virtual switch, assign a static IP of 10.10.2.11 then click Create.

You should now be able to ping 10.10.1.11 and 10.10.2.11 from your Mac. You should also be able to ping 10.10.1.201 from the ESXi host.

Enable iSCSI

Once you know you can ping both iSCSI network addresses of the FreeNAS machine, it’s time to enable and configure iSCSI so that we can see the datastore.

Click Storage, Adapters then Software iSCSI.

Select Enabled and wait for the extra settings to load.

Scroll down to Dynamic targets and add the IP addresses of the FreeNAS iSCSI networks (10.10.1.201 and 10.10.2.201) then click Save configuration.

Click the Devices tab and you should now see the FreeNAS iSCSI disk in the list. If you don’t see it try clicking the Rescan button.

Test Path Failover

First of all, if you connect to the ESXi host via SSH (ssh root@10.1.1.11) and run the following command, you should see two paths to the iSCSI disk that have the State of active.

esxcli storage core path list

Now that we know there are two active paths, we will disconnect a network adapter and see if the disk still works.

Open the Settings screen of either the ESXi or FreeNAS VM and click on one of the iSCSI network adapters then disable it by de-selecting the Connect Network Adapter box.

Click Refresh and you will see that the FreeNAS disk has changed to Normal, Degraded.

Running the esxcli storage core path list command again should also show a path with a state of dead.

Reconnect the network adapter then click Refresh and you should see the status has changed back to Normal.

The only thing left to do now is create the VMFS datastore which you can do by clicking New datastore and selecting the default options throughout the wizard.

Conclusion

And that’s it, you’ve successfully used VMware Fusion* to test iSCSI mulipathing using FreeNAS and ESXi. We also simulated what would happen if a network adapter or switch failed in a production environment.

As a further test, try creating a VM on the datastore and see if it stays active when disconnecting one of the iSCSI network adapters.