Use port groups and vSwitches for networking

[dezull]

I'm not sure whether you will want this and there are some more work to do on this one, but please don't merge this yet. If you are interested, I may be able to make some changes.

Essentially, it adds networks like VirtualBox provider, by using port groups and VLANs.

Details

This creates port groups (networks) and vSwitches automatically on vagrant up. The networks are created like virtualbox provider. It also optionally destroys the networks and the vSwitches on machine destroy.

Helper command destroy-networks is added for convenience, to destroy all networks matching a pattern {vSwitchName}-{network-address}-{network-prefix}

It also ignores esxi_virtual_network option, but I can make it compatible.

Example Vagrantfile

Vagrant.configure("2") do |config|
  config.vm.box = "hashicorp/bionic64"

  config.vm.provider :vmware_esxi do |esxi|
    esxi.esxi_hostname = "x.x.x.x"
    esxi.esxi_username = "root"
    esxi.esxi_password = "password"
    esxi.guest_memsize = "512"

    # esxi.destroy_unused_port_groups= true

    # Defaults
    # exsi.default_vswitch = "vSwitch0"
    # exsi.default_port_group = "VM Network"
    # esxi.destroy_unused_port_groups= false
    # esxi.destroy_unused_vswitches = false
    # esxi.destroy_unused_networks = false     # sets to the previous 2 options 
  end

  # creates port group `vSwitch0-192.168.100.0-24` on `vSwitch0`
  config.vm.network "private_network", ip: "192.168.100.2"

  # creates port group `vSwitch0-10.10.10.0-16` on `vSwitch0`
  config.vm.network "private_network", ip: "10.10.10.2", netmask: "16"

  # creates port group `Port Group A` on `vSwitch0`
  config.vm.network "private_network", ip: "172.16.10.20", esxi__port_group: "Port Group A"

  # creates port group `Port Group B` on `vSwitch0`
  config.vm.network "private_network", dhcp: true, esxi__port_group: "Port Group B"

  # should fail, default_port_group not in esxi__vswitch
  # config.vm.network "private_network", ip: "192.168.10.2", esxi__vswitch: "Should Fail 1"

  # creates port group `Port Group C` on `vSwitch 1`
  config.vm.network "private_network", ip: "10.20.20.2", esxi__vswitch: "vSwitch 1", esxi__port_group: "Port Group C"

  # should fail, default_port_group not in esxi__vswitch
  # config.vm.network "private_network", dhcp: true, esxi__vswitch: "Should Fail 2"

  # creates port group `Port Group D` on `vSwitch 2`
  config.vm.network "private_network", dchp: true, esxi__vswitch: "vSwitch 2", esxi__port_group: "Port Group D"

  config.vm.synced_folder('.', '/vagrant', type: 'rsync')
end

Example vagrant up and vagrant destroy

Click to expand!

ninja@ubuntu2004-1:~/vagrantesxi$ vagrant up --provider vmware_esxi                                                                                                                 
                                                                                                                                                                                        
Bringing machine 'default' up with 'vmware_esxi' provider... 
==> default: --- Default network on Adapter 1: vSwitch: vSwitch0, port group: VM Network                                                                                                
==> default: --- Creating other networks...                                                  
    default: --- Adapter 2: vSwitch: vSwitch0, port group: vSwitch0-192.168.100.0-24         
    default: --- Adapter 3: vSwitch: vSwitch0, port group: vSwitch0-10.10.0.0-16             
    default: --- Adapter 4: vSwitch: vSwitch0, port group: Port Group A                      
    default: --- Adapter 5: vSwitch: vSwitch0, port group: Port Group B                      
    default: --- Adapter 6: vSwitch: vSwitch 1, port group: Port Group C                     
    default: --- Adapter 7: vSwitch: vSwitch 2, port group: Port Group D                     
==> default: Virtual Machine will be built.                                                  
VMware ovftool 4.4.0 (build-16360108)                                                        
==> default: --- WARNING         : esxi_disk_store not set, using "--- Least Used ---"       
==> default: ---   --- ESXi Summary ---                                                                                                                                                 
==> default: --- ESXi host       : x.x.x.x                                  
==> default: --- Virtual Network : ["VM Network", "vSwitch0-192.168.100.0-24", "vSwitch0-10.10.0.0-16", "Port Group A", "Port Group B", "Port Group C", "Port Group D"]                 
==> default: --- Disk Store      : datastore1                                                
==> default: --- Resource Pool   : /                                                         
==> default: ---  --- Guest Summary ---                                                      
==> default: --- VM Name         : V-ubuntu2004-1-ninja-vagrantesxi                          
==> default: --- Box             : hashicorp/bionic64                                        
==> default: --- Box Ver         : 1.0.282                                                   
==> default: --- Memsize (MB)    : 512                                                       
==> default: --- CPUS            : 1                                                         
==> default: --- Guest OS type   : ubuntu-64                                                                                                                                            
==> default: --- Resource Pool   : / 
==> default: ---  --- Guest Summary --- 
==> default: --- VM Name         : V-ubuntu2004-1-ninja-vagrantesxi 
==> default: --- Box             : hashicorp/bionic64 
==> default: --- Box Ver         : 1.0.282 
==> default: --- Memsize (MB)    : 512 
==> default: --- CPUS            : 1 
==> default: --- Guest OS type   : ubuntu-64 
==> default: ---   --- Guest Build --- 
Opening VMX source: /home/ninja/.vagrant.d/boxes/hashicorp-VAGRANTSLASH-bionic64/1.0.282/vmware_desktop/ZZZZ_V-ubuntu2004-1-ninja-vagrantesxi.vmx 
Opening VI target: vi://[email protected]:443/ 
Deploying to VI: vi://[email protected]:443/ 
Transfer Completed                     
Completed successfully 
==> default: --- VMID            : 72 
==> default: --- VM has been Powered On... 
==> default: --- Waiting for state "running" 
==> default: --- Success, state is now "running" 
==> default: --- Configuring     : 192.168.100.2/24 on vSwitch0-192.168.100.0-24 
==> default: --- Configuring     : 10.10.10.2/16 on vSwitch0-10.10.0.0-16 
==> default: --- Configuring     : 172.16.10.20/24 on Port Group A 
==> default: --- Configuring     : dhcp on Port Group B 
==> default: --- Configuring     : 10.20.20.2/24 on Port Group C 
==> default: --- Configuring     : dhcp on Port Group D 
    default:  
    default: Vagrant insecure key detected. Vagrant will automatically replace 
    default: this with a newly generated keypair for better security. 
    default:  
    default: Inserting generated public key within guest... 
    default: Removing insecure key from the guest if it's present... 
    default: Key inserted! Disconnecting and reconnecting using new SSH key... 
==> default: Rsyncing folder: /home/ninja/vagrantesxi/ => /vagrant 
ninja@ubuntu2004-1:~/vagrantesxi$ vagrant destroy 
 
==> default: powered off 
==> default: --- VM has been destroyed... 
==> default: --- Destroying unused port groups that were created automatically... 
    default: --- Destroying port group 'vSwitch0-192.168.100.0-24' 
    default: --- Destroying port group 'vSwitch0-10.10.0.0-16' 
    default: --- Destroying port group 'Port Group A' 
    default: --- Destroying port group 'Port Group B' 
    default: --- Destroying port group 'Port Group C' 
    default: --- Destroying port group 'Port Group D' 
==> default: --- Destroying unused vSwitches that were created automatically... 
    default: --- Destroying vswitch 'vSwitch 1' 
    default: --- Destroying vswitch 'vSwitch 2' 
ninja@ubuntu2004-1:~/vagrantesxi$  

[josenk]

Thanks for the work you are doing, but do you really think there's a demand for this for Vagrant? I see some risk here... The option to destroy or not destroy the network will lead to dangling resources. Also, if you create a network using vagrant that could lead to dependency issues if those networks are used for other VMs.

Terraform is a much better tool to build your infrastructure. There is work being done in the terraform provider to create vSwitches and networking.

[dezull]

I can't speak for others, but the project that I'm working on have a third party component that depends on Vagrant (with support for VirtualBox & oVirt providers), switching it would be hard, so simply adding the features I need to the provider seems like the easiest route for me. So I thought it might be useful for others too.

Regarding the networking, the code checks if the network is in use before destroying it. So, it may not be possible to destroy all networks initially created by this provider. The additional destroy-networks command may help in this case, but still there is no way to prevent another VM manually created outside Vagrant, using the same network.

But if one sticks to just the default vSwitch, and let the port group be automatically created by IP address (like VirtualBox), it should work just fine.

I do understand your concern, just wanted to know if you are interested in this, I'll spend a bit more time to polish things.

[scaleoutsean]

I like this idea, I think it's pretty useful.

1. I haven't noticed this before, it's likely unrelated but once I installed this plugin vagrant automatically removed my vagrant-vsphere plugin (maybe due to some dependency issue?)

Fetching vagrant-vmware-esxi-2.5.0.gem
Successfully uninstalled vagrant-vsphere-1.13.5
Installed the plugin 'vagrant-vmware-esxi (2.5.0)'!

2. Changing default vSwitch setting made me hit errors and spend some time on troubleshooting:

    # Defaults
    exsi.default_vswitch = "vSwitch1"
    exsi.default_port_group = "VM Network"
    # esxi.destroy_unused_port_groups= false

Turns out those should be esxi, not exsi.

3. I tried to work with these two set to true:

    # Defaults
    # exsi.default_vswitch = "vSwitch0"
    # exsi.default_port_group = "VM Network"
    esxi.destroy_unused_port_groups = true
    esxi.destroy_unused_vswitches = false
    # esxi.destroy_unused_networks = false     # sets to the previous 2 options

But that doesn't seem to work:

vagrant destroy 

==> default: powered off
==> default: --- VM has been destroyed...
==> default: --- Destroying unused port groups that were created automatically...
There was an error talking to ESXi.
  Unable to get port groups for vm ''

Maybe get port groups before destroying the VM, then destroy them one by one if no other VMs use those port groups?

I also tried a different scenario, which is:

• Destroy VM first
• Change esxi.destroy_unused_port_groups to true and run vagrant destroy again

I got the same error. I didn't expect to get a different error, but I was hoping it might work becuse .vagrant/machines/default/vmware_esxi/networks still has that information ({"port_groups":["vSwitch0-192.168.100.0-24","vSwitch0-10.10.0.0-16","Port Group A","Port Group B","Port Group C","Port Group D"],"vswitches":[]}).

4. One thing that should probably be documented (or even better, exposed in port group settings) is how VLAN IDs get created. Is it just unused IDs in increasing order? I happen to not have VLAN IDs 1-6 (those were created) but if I had existing VLANs, would the patch pick non-conflicting VLAN IDs?