Deploy and Configure a VCF Management Domain Using VMware Cloud Builder PDF

VMware Cloud Foundation Deployment Guide VMware Cloud Foundation 5.1 VMware Cloud Foundation Deployment Guide You can find the most up-to-date technical documentation on the VMware by Broadcom website at: https://docs.vmware.com/ VMware by Broadcom 3401 Hillview Ave. Palo Alto, CA 94304 www.vmware.com © Copyright 2015-2024 Broadcom. All Rights Reserved. The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. For more information, go to https://www.broadcom.com. All trademarks, trade names, service marks, and logos referenced herein belong to their respective companies. VMware by Broadcom 2 Contents About the VMware Cloud Foundation Deployment Guide 4 1 Preparing your Environment for VMware Cloud Foundation 5 2 Deploying VMware Cloud Foundation 6 Deploy VMware Cloud Builder Appliance 7 Prepare ESXi Hosts for VMware Cloud Foundation 9 Create a Custom ISO Image for ESXi 10 Create a Custom ESXi ISO Image Using VMware PowerCLI 10 Create a Custom ESXi ISO Image Using vSphere Lifecycle Manager 12 Install ESXi Interactively and Configure Hosts for VMware Cloud Foundation 12 Install ESXi on VMware Cloud Foundation Hosts Using the ISO 13 Configure the Network on VMware Cloud Foundation Hosts 14 Configure the Virtual Machine Network Port Group on VMware Cloud Foundation Hosts 15 Configure NTP on VMware Cloud Foundation Hosts 16 Regenerate the Self-Signed Certificate on All Hosts 17 Configure ESXi Hosts with Signed Certificates 17 Deploy the Management Domain Using VMware Cloud Builder 19 About the Deployment Parameter Workbook 20 Credentials Worksheet 21 Hosts and Networks Worksheet 23 Deploy Parameters Worksheet: Existing Infrastructure Details 27 Deployment Parameters Worksheet: License Keys 28 Deploy Parameters Worksheet: vSphere Infrastructure 29 Deploy Parameters Worksheet: VMware NSX 33 Deploy Parameters Worksheet: SDDC Manager 33 Deploy the Management Domain Using ESXi Hosts with External Certificates 33 3 Troubleshooting VMware Cloud Foundation Deployment 35 VMware Cloud Builder Log Files 35 Using the SoS Utility on VMware Cloud Builder 36 4 VMware Cloud Foundation Glossary 41 VMware by Broadcom 3 About the VMware Cloud Foundation Deployment Guide The VMware Cloud Foundation Deployment Guide provides information about installing VMware ESXi™ software on VMware Cloud Foundation™ servers and deploying the management domain using the VMware Cloud Builder appliance™. Intended Audience The VMware Cloud Foundation Deployment Guide is intended for data center cloud administrators who deploy a VMware Cloud Foundation system in their organization's data center. The information in this guide is written for experienced data center cloud administrators who are familiar with: n Concepts of virtualization and software-defined data centers n Networking and concepts such as uplinks, NICs, and IP networks n Hardware components such as top-of-rack (ToR) switches, inter-rack switches, servers with direct attached storage, cables, and power supplies n Methods for setting up physical racks in your data center ® n Using the VMware vSphere Client™ to work with virtual machines Related Publications Getting Started with VMware Cloud Foundation document provides a high-level overview of the VMware Cloud Foundation product The Planning and Preparation Workbook provides detailed information about the software, tools, and external services that are required for Cloud Foundation. The VMware Cloud Foundation Administration Guide contains detailed information about how to administer and operate a VMware Cloud Foundation system in your data center. Your VMware Cloud Foundation system includes various VMware software products and components. You can find the documentation for those VMware software products at docs.vmware.com. VMware Cloud Foundation Glossary The VMware Cloud Foundation Glossary defines terms specific to VMware Cloud Foundation. VMware by Broadcom 4 Preparing your Environment for VMware Cloud Foundation 1 Before you start the automated deployment of the management domain using VMware Cloud Builder, your environment must meet target prerequisites and be in a specific starting state. Prepare the platform by deploying and configuring the necessary infrastructure components. For detailed prerequisites, see the Planning and Preparation Workbook. VMware by Broadcom 5 Deploying VMware Cloud Foundation 2 You begin the VMware Cloud Foundation deployment process by deploying the VMware Cloud Builder appliance. After imaging your servers, you download and complete the deployment parameters workbook from the VMware Cloud Builder appliance to define your network information, host details, and other required information. During the deployment process, this workbook is uploaded to the VMware Cloud Builder appliance, where a JSON file is generated to drive the bring-up process. The provided information is validated, and the automated phase of the bring-up process begins. You can perform bring-up with certificates generated by an external CA, in which case ESXi certificates are not replaced with vCenter Server signed certificates. If you use external certificates for ESXi hosts in the management domain, hosts added after bring-up must also be added with external certificates. This feature is supported only through APIs. For more information, see Deploy the Management Domain Using ESXi Hosts with External Certificates. Prerequisites You must prepare your environment for deploying VMware Cloud Foundation. See the Planning and Preparation Workbook. Procedure 1 Deploy VMware Cloud Builder Appliance VMware Cloud Builder is a virtual appliance that is used to deploy and configure the first cluster of the management domain and transfer inventory and control to SDDC Manager. During the deployment process, the VMware Cloud Builder appliance validates network information you provide in the deployment parameter workbook such as DNS, network (VLANS, IPs, MTUs), and credentials. 2 Prepare ESXi Hosts for VMware Cloud Foundation 3 Deploy the Management Domain Using VMware Cloud Builder The VMware Cloud Foundation deployment process is referred to as bring-up. You specify deployment information specific to your environment such as networks, hosts, license keys, and other information in the deployment parameter workbook and upload the file to the VMware Cloud Builder appliance to initiate bring-up of the management domain. VMware by Broadcom 6 VMware Cloud Foundation Deployment Guide Deploy VMware Cloud Builder Appliance VMware Cloud Builder is a virtual appliance that is used to deploy and configure the first cluster of the management domain and transfer inventory and control to SDDC Manager. During the deployment process, the VMware Cloud Builder appliance validates network information you provide in the deployment parameter workbook such as DNS, network (VLANS, IPs, MTUs), and credentials. You must deploy the VMware Cloud Builder appliance on a suitable platform. This can be on a laptop running VMware Workstation or VMware Fusion, or on an ESXi host. The VMware Cloud Builder appliance must have network access to all hosts on the management network. This procedure describes how to deploy the VMware Cloud Builder appliance directly to an ESXi host. Prerequisites Before you deploy the VMware Cloud Builder appliance, verify that your environment fulfills the requirements for this process. Prerequisite Value Environment n Verify that your environment is configured for deployment of VMware Cloud Builder and the management domain. n Verify that you have available virtual infrastructure that has access to the management network that will be used by the management domain. You deploy VMware Cloud Builder on that virtual infrastructure. Resource Requirements n 4 CPUs n 4 GB of Memory n 279 GB of Storage n 25.1 GB (thin provisioned) n 253.8 GB (thick provisioned) Installation Packages Verify that you download the OVA file(s) for VMware Cloud Builder. Network n Verify that the static IP address and FQDN for the VMware Cloud Builder appliance are available. n Verify that connectivity is in place from the VMware Cloud Builder appliance and the management VLAN used in the deployment. To image servers and automate the deployment, the VMware Cloud Builder appliance must be on the same management network as the hosts to be used. It must also be able to access all required external services, such as DNS and NTP. Procedure 1 In a web browser, log in to the ESXi host using the VMware Host Client. 2 In the navigation pane, select Host, and click Create/Register VM. VMware by Broadcom 7 VMware Cloud Foundation Deployment Guide 3 On the Select creation type dialog box, select Deploy a virtual machine from an OVF or OVA file and click Next. 4 On the Select OVF and VMDK files page, enter a name for the virtual machine, select the VMware Cloud Builder.ova file, and click Next. 5 On the Select Storage page, select a datastore and click Next. 6 On the License agreements dialog box, click I agree and then click Next. 7 On the Select networks dialog box, enter the following values and click Next. Setting Value Network mappings your_portgroup Disk provisioning Thin Power on automatically Selected 8 On the Additional settings dialog box, expand Application, enter the following values, and click Next. Setting Details Admin Username Accept the default admin user name, admin. Admin Password/Admin Password The admin password must be a minimum of 8 characters and include at confirm least one uppercase, one lowercase, one digit, and one special character. Supported special characters: @ ! # $ % ? ^ Note A password cannot be based on a dictionary word (for example, VMware1!) Root password/Root password The root password must be a minimum of 8 characters and include at confirm least one uppercase, one lowercase, one digit, and one special character. Supported special characters: @ ! # $ % ? ^ Note A password cannot be based on a dictionary word (for example, VMware1!) Hostname Enter the hostname for the VMware Cloud Builder appliance. Network 1 IP Address Enter the IP address for the VMware Cloud Builder appliance. Network 1 Subnet Mask Enter the subnet mask for the VMware Cloud Builder appliance. Default Gateway Enter the default gateway for the VMware Cloud Builder appliance. DNS Servers Enter the IP address of the primary and secondary DNS servers (comma separated). Do not specify more than two servers. DNS Domain Name Enter the DNS domain name. For example, vsphere.local. VMware by Broadcom 8 VMware Cloud Foundation Deployment Guide Setting Details DNS Domain Search Paths Enter the DNS domain search path(s). Use a comma if entering multiple search paths. For example vsphere.local, sfo.vsphere.local. NTP Servers Enter the NTP server(s). Use a comma if entering multiple NTP servers. NTP servers can be entered using FQDNs or IP addresses. 9 On the Ready to complete page, review the virtual machine configuration and click Finish. Note Make sure your passwords meet the requirements specified above before clicking Finish or your deployment will not succeed. 10 After the VMware Cloud Builder appliance is deployed, SSH in to the VM with the admin credentials provided in step 9. 11 Ensure that you can ping the ESXi hosts. 12 Verify that the VMware Cloud Builder appliance has access to the required external services, such as DNS and NTP by performing forward and reverse DNS lookups for each host and the specified NTP servers. Prepare ESXi Hosts for VMware Cloud Foundation Before you can begin the process of deploying VMware Cloud Foundation you must prepare the ESXi hosts that will form the management domain. Preparing the ESXi hosts involves installing the correct version of ESXi and performing some basic configuration tasks. For the supported ESXi version, see the Bill of Materials (BOM) section of the VMware Cloud Foundation Release Notes. Prerequisites The management domain requires a minimum of four ESXi hosts. To use vSAN Express Storage Architecture (ESA), your hosts must be ESA-compatible. Tip See the vSAN ESA VCG for information about compatible hardware. n Create a Custom ISO Image for ESXi When your environment requires a custom ISO file for ESXi, you can create one using VMware PowerCLI or vSphere Lifecycle Manager. n Install ESXi Interactively and Configure Hosts for VMware Cloud Foundation You can interactively install ESXi on all the hosts that will form the first cluster in the management domain, then you configure the management network, DNS, and NTP services. You can use the same process to add more hosts to the management domain later, or to install and configure hosts for VI workload domains. VMware by Broadcom 9 VMware Cloud Foundation Deployment Guide n Regenerate the Self-Signed Certificate on All Hosts Once you have configured the ESXi hosts' identity by providing a hostname you must regenerate the self-signed certificate to ensure the correct common name is defined. n Configure ESXi Hosts with Signed Certificates If corporate policy requires that you use external CA-signed certificates instead of VMCA- signed certificates for ESXi hosts, you can manually add external certificates to the hosts. Create a Custom ISO Image for ESXi When your environment requires a custom ISO file for ESXi, you can create one using VMware PowerCLI or vSphere Lifecycle Manager. You might need to create a custom ISO image for ESXi in the following situations: n The ESXi version specified in the VMware Cloud Foundation BOM does not have an associated ISO file on VMware Customer Connect. This can be the case for ESXi patch releases. n You need an async patch version of ESXi. n You need a vendor-specific (OEM) ISO file. Prerequisites Download the zip files for the following: n ESXi patch for the ESXi version specified in the VMware Cloud Foundation BOM or in the list of supported async patches in KB 88287. You can download patches from https:// customerconnect.vmware.com/patch#search. Note If you are preparing hosts for a VI workload domain where the ESXi hosts have been async patched to a later version of ESXi than the version listed in the BOM, the new hosts must use the later version of ESXi. n OEM add-on for ESXi from VMware Customer Connect. If the ESXi version specified in the BOM is not available in the Select Version drop-down menu, contact your vendor to determine which OEM add-on version to use. Create a Custom ESXi ISO Image Using VMware PowerCLI You can use VMware Power CLI to create a custom ISO. Prerequisites VMware PowerCLI 12.0 or later. VMware by Broadcom 10 VMware Cloud Foundation Deployment Guide Procedure 1 Gather the required information for the software spec that is used to create the custom ISO. a In VMware PowerCLI, use the Get-DepotBaseImages cmdlet to get the base image version from the zip file for the ESXi patch that you downloaded from the patches portal. For example: Get-DepotBaseImages “c:\temp\VMware-ESXi-7.0U1d-17551050-depot.zip” b Use the Get-DepotAddons cmdlet to get the add-on name and version from the zip file for the OEM add-on for ESXi that you downloaded from VMware Customer Connect. (if applicable) For example: Get-DepotAddons “c:\temp\HPE-701.0.0.10.6.5.12-Jan2021-Synergy-Addon-depot.zip” 2 Create the software spec using the information you gathered in step 1. The software spec is a JSON file that contains information about the ESXi version and vendor add-on (if applicable). For example: { "add_on": { "name": "HPE-Custom-Syn-AddOn", "version": "701.0.0.10.6.5-12" }, "base_image": { "version": "7.0.1-0.30.17551050" }, "components": null, "hardware_support": null, "solutions": null } 3 In VMware PowerCLI, use the New-IsoImage cmdlet to generate a custom ISO. For example: New-IsoImage -SoftwareSpec “c:\temp\HPE-70U1d-custom.JSON” -Depots “c:\temp\VMware- ESXi-7.0U1d-17551050-depot.zip” , “c:\temp\HPE-701.0.0.10.6.5.12-Jan2021-Synergy-Addon- depot.zip” -Destination “c:\temp\HPE-70U1d-custom.iso” Provide the path to the software spec you created in step 2. The depot(s) include the path to the zip files for the supported ESXi version and vendor add-on. The destination include the path and file name for the custom ISO file. For more information about the New-IsoImage cmdlet, see https://code.vmware.com/docs/ 11794/cmdletreference//doc/New-IsoImage.html. VMware by Broadcom 11 VMware Cloud Foundation Deployment Guide Create a Custom ESXi ISO Image Using vSphere Lifecycle Manager If you have access to a vCenter Server environment, you can use vSphere Lifecycle Manager to create and export a custom ISO. Prerequisites Import the ESXi patch and vendor add-on (if applicable) zip files to the vSphere Lifecycle Manager depot. See Import Updates to the vSphere Lifecycle Manager Depot. Procedure 1 Log in to vCenter Server using the vSphere Client. 2 Create a new temporary cluster, selecting the Manage all hosts in the cluster with a single image check box. 3 Select the ESXi version and vendor add-on (optional) and click OK. 4 Export the vSphere Lifecycle Manager image as an ISO. See Export an Image. 5 Delete the temporary cluster. Install ESXi Interactively and Configure Hosts for VMware Cloud Foundation You can interactively install ESXi on all the hosts that will form the first cluster in the management domain, then you configure the management network, DNS, and NTP services. You can use the same process to add more hosts to the management domain later, or to install and configure hosts for VI workload domains. VMware by Broadcom 12 VMware Cloud Foundation Deployment Guide Prerequisites n Download the ESXi ISO from VMware Customer Connect. For the supported ESXi versions, see the Bill of Materials (BOM) section of the VMware Cloud Foundation Release Notes and the list of supported async patches in KB 88287. If the required version of ESXi does not have an ISO available on VMware Customer Connect, you can create one. See Create a Custom ISO Image for ESXi. Note If you are preparing hosts for a VI workload domain where the ESXi hosts have been async patched to a later version of ESXi than the version listed in the BOM, the new hosts must use the later version of ESXi. n Make sure that you have a host machine for SDDC access. You use this host to connect to the data center and perform configuration steps. n Verify that you have the completed Planning and Preparation Workbook. n Verify the Prerequisite Checklist sheet in the Planning and Preparation Workbook. Procedure 1 Install ESXi on VMware Cloud Foundation Hosts Using the ISO Install ESXi on all hosts in the first cluster in the management domain interactively. You can use the same process to install ESXi on additional hosts for the management domain, or on hosts for a VI workload domain. 2 Configure the Network on VMware Cloud Foundation Hosts After the initial boot, use the ESXi Direct Console User Interface (DCUI) for host network configuration and administrative access. 3 Configure the Virtual Machine Network Port Group on VMware Cloud Foundation Hosts You perform configuration of the Virtual Machine Network port group for each ESXi host by using the VMware Host Client. 4 Configure NTP on VMware Cloud Foundation Hosts Complete the initial configuration of all ESXi hosts by configuring the NTP service to avoid time synchronization issues in the SDDC. Install ESXi on VMware Cloud Foundation Hosts Using the ISO Install ESXi on all hosts in the first cluster in the management domain interactively. You can use the same process to install ESXi on additional hosts for the management domain, or on hosts for a VI workload domain. Repeat this procedure for all hosts in the first cluster in the management domain. Procedure 1 Mount the ESXi ISO on the host and restart the machine. VMware by Broadcom 13 VMware Cloud Foundation Deployment Guide 2 Set the BIOS or UEFI to boot from the mounted ISO. Note If your system has supported data processing units (DPUs), you can only use UEFI to install and boot ESXi on the DPUs. See your hardware vendor documentation for information on changing boot order. After scanning for available devices completes, if your system has supported DPUs, you see them listed with their respective PCI slots. 3 If your system has supported DPUs, select the DPU on which you want to install ESXi and press Enter. In the DPU Details screen, you see all properties of the DPU device. 4 On the Select a Disk to Install or Upgrade screen, select the drive on which to install ESXi on and press Enter. 5 Select the keyboard type for the host. You can change the keyboard type after installation in the direct console. 6 Enter the root password for the host. 7 Press F11 to start the installation. 8 On the Installation Complete screen, press Enter to reboot the host. 9 Set the first boot device to be the drive on which you installed ESXi. 10 Repeat this procedure for all remaining hosts. Configure the Network on VMware Cloud Foundation Hosts After the initial boot, use the ESXi Direct Console User Interface (DCUI) for host network configuration and administrative access. Perform the following tasks to configure the host network settings: n Configure the network adapter (vmk0) and VLAN ID for the Management Network. n Configure the IP address, subnet mask, gateway, DNS server, and FQDN for the ESXi host. Repeat this procedure for all hosts that you are adding to the first cluster of the management domain. Enter the respective values from the completed Planning and Preparation Workbook. Procedure 1 Open the DCUI of the ESXi host. a Open a console window to the host. b Press F2 to enter the DCUI. c Log in by using the esxi_root_user_password. VMware by Broadcom 14 VMware Cloud Foundation Deployment Guide 2 Configure the network. a Select Configure Management Network and press Enter. b Select VLAN (Optional) and press Enter. c Enter the VLAN ID for the Management Network and press Enter. d Select IPv4 Configuration and press Enter. e Select Set static IPv4 address and network configuration and press the Space bar. f Enter the IPv4 Address, Subnet Mask and Default Gateway and press Enter. g Select DNS Configuration and press Enter. h Select Use the following DNS Server address and hostname and press the Space bar. i Enter the Primary DNS Server, Alternate DNS Server and Hostname (FQDN) and press Enter. j Select Custom DNS Suffixes and press Enter. k Ensure that there are no suffixes listed and press Enter. 3 Press Escape to exit and press Y to confirm the changes. 4 Repeat this procedure for all remaining hosts. Configure the Virtual Machine Network Port Group on VMware Cloud Foundation Hosts You perform configuration of the Virtual Machine Network port group for each ESXi host by using the VMware Host Client. You configure the VLAN ID of the VM Network port group on the vSphere Standard Switch. This configuration provides connectivity to the Management network to allow communication to the vCenter Server Appliance during the automated deployment. Repeat this procedure for all hosts in the first cluster of the management domain. Enter the respective values from the completed Planning and Preparation Workbook. Procedure 1 In a web browser, log in to the ESXi host using the VMware Host Client. 2 Click OK to join the Customer Experience Improvement Program. VMware by Broadcom 15 VMware Cloud Foundation Deployment Guide 3 Configure a VLAN for the VM Network port group. a In the navigation pane, click Networking. b Click the Port groups tab, select the VM network port group, and click Edit Settings. c On the Edit port group - VM network page, enter the Management Network VLAN ID, and click Save. 4 Repeat this procedure for all remaining hosts. Configure NTP on VMware Cloud Foundation Hosts Complete the initial configuration of all ESXi hosts by configuring the NTP service to avoid time synchronization issues in the SDDC. Repeat this procedure for all hosts in the first cluster of the management domain. Enter the respective values from the completed Planning and Preparation Workbook. Procedure 1 In a web browser, log in to the ESXi host using the VMware Host Client. 2 Configure and start the NTP service. a In the navigation pane, click Manage, and click the System tab. b Click Time & date and click Edit NTP Settings. c On the Edit NTP Settings page, select the Use Network Time Protocol (enable NTP client) radio button, and change the NTP service startup policy to Start and stop with host. VMware by Broadcom 16 VMware Cloud Foundation Deployment Guide d In the NTP servers text box, enter the NTP Server FQDN or IP Address, and click Save. e To start the service, click Actions, select NTP service, and click Start. 3 Repeat this procedure for all remaining hosts. Regenerate the Self-Signed Certificate on All Hosts Once you have configured the ESXi hosts' identity by providing a hostname you must regenerate the self-signed certificate to ensure the correct common name is defined. During the installation of ESXi, the installer generates a self-signed certificate for each ESXi host but the process is performed prior to the ESXi identity being configured. This means all ESXi hosts have a common name in their self-signed certificate of localhost.localdomain. All communication between VMware Cloud Builder and the ESXi hosts is performed securely over HTTPS and as a result it validates the identify when making a connection by comparing the common name of the certificate against the FQDN provided within the VMware Cloud Builder configuration file. To ensure that the connection attempts and validation does not fail, you must manually regenerate the self-signed certificate after hostname has been configured. Note VMware Cloud Foundation supports the use of signed certificates. If your organization's security policy mandates that all ESXi hosts must be configured with a CA-signed certificate, see Configure ESXi Hosts with Signed Certificates. Procedure 1 In a web browser, log in to the ESXi host using the VMware Host Client. 2 In the navigation pane, click Manage and click the Services tab. 3 Select the TSM-SSH service and click Start if not started. 4 Log in to the ESXi host using an SSH client such as Putty. 5 Regenerate the self-signed certificate by executing the following command: /sbin/generate-certificates 6 Restart the hostd and vpxa services by executing the following command: /etc/init.d/hostd restart && /etc/init.d/vpxa restart 7 In the VMware Host Client, select the TSM-SSH service for the ESXi host and click Start. 8 Repeat this procedure for all remaining hosts. Configure ESXi Hosts with Signed Certificates If corporate policy requires that you use external CA-signed certificates instead of VMCA-signed certificates for ESXi hosts, you can manually add external certificates to the hosts. VMware by Broadcom 17 VMware Cloud Foundation Deployment Guide When you install ESXi software on a server to create an ESXi host, the host initially has an autogenerated certificate. By default, when the host is added to a vCenter Server system during bring-up of the management domain or other operations involving hosts (for example, host commissioning, VI workload domain creation, and so on), the autogenerated certificate is replaced with a certificate that is signed by the VMware Certificate Authority (VMCA). When you use external certificates during bring-up, they are not replaced by VMCA-signed certificates. Once you perform bring-up with external certificates for ESXi hosts, all future hosts added to VMware Cloud Foundation must also use external certificates. Prerequisites External CA-signed certificate and key are available. Procedure 1 In a web browser, log in to the ESXi host using the VMware Host Client. 2 In the navigation pane, click Manage and click the Services tab. 3 Select the TSM-SSH service and click Start if not started. 4 Log in to the ESXi Shell for the first host, either directly from the DCUI or from an SSH client, as a user with administrator privileges. 5 In the directory /etc/vmware/ssl, rename the existing certificates using the following commands: mv rui.crt orig.rui.crt mv rui.key orig.rui.key 6 Copy the external certificate and key that you want to use to /etc/vmware/ssl. 7 Rename the external certificate and key to rui.crt and rui.key. VMware by Broadcom 18 VMware Cloud Foundation Deployment Guide 8 Restart the host management agents by running the following commands: /etc/init.d/hostd restart /etc/init.d/vpxa restart 9 In the VMware Host Client, select the TSM-SSH service for the ESXi host and click Stop. 10 Repeat for all the ESXi hosts that you are adding to VMware Cloud Foundation. What to do next See Deploy the Management Domain Using ESXi Hosts with External Certificates. Deploy the Management Domain Using VMware Cloud Builder The VMware Cloud Foundation deployment process is referred to as bring-up. You specify deployment information specific to your environment such as networks, hosts, license keys, and other information in the deployment parameter workbook and upload the file to the VMware Cloud Builder appliance to initiate bring-up of the management domain. During bring-up, the management domain is created on the ESXi hosts specified in the deployment parameter workbook. The VMware Cloud Foundation software components are automatically deployed, configured, and licensed using the information provided. The deployment parameter workbook can be reused to deploy multiple VMware Cloud Foundation instances of the same version. The following procedure describes how to perform bring-up of the management domain using the deployment parameter workbook. You can also perform bring-up using a custom JSON specification. See the VMware Cloud Foundation API Reference Guide for more information. Some use cases are only available using a custom JSON specification. For example, using custom CA-signed certificates for ESXi hosts. See Deploy the Management Domain Using ESXi Hosts with External Certificates. Procedure 1 In a web browser, log in to the VMware Cloud Builder appliance administration interface: https://Cloud_Builder_VM_FQDN. 2 Enter the admin credentials you provided when you deployed the VMware Cloud Builder appliance and then click Log In. 3 On the End-User License Agreement page, select the I Agree to the End User License Agreement check box and click Next. 4 Select VMware Cloud Foundation and click Next. VMware by Broadcom 19 VMware Cloud Foundation Deployment Guide 5 Review and acknowledge the prerequisties and click Next. If there are any gaps, ensure they are fixed before proceeding to avoid issues during the bring-up process. You can download or print the prerequisite list for reference. 6 Download the deployment parameter workbook from VMware Customer Connect and fill it in with the required information. See About the Deployment Parameter Workbook. 7 Click Next. 8 Click Select File, browse to the completed workbook, and click Open to upload the workbook. 9 Click Next to begin validation of the uploaded file. To access the bring-up log file, SSH to the VMware Cloud Builder appliance as admin and open the /opt/vmware/bringup/logs/vcf-bringup-debug.log file. If there is an error during the validation and the Next button is grayed out, you can either make corrections to the environment or edit the deployment parameter workbook and upload it again. Then click Retry to perform the validation again. If any warnings are displayed and you want to proceed, click Acknowledge and then click Next. 10 Click Deploy SDDC. During the bring-up process, the vCenter Server, NSX, and SDDC Manager appliances are deployed and the management domain is created. The status of the bring-up tasks is displayed in the UI. After bring-up is completed, a green bar is displayed indicating that bring-up was successful. A link to the SDDC Manager UI is also displayed. If there are errors during bring-up, see Chapter 3 Troubleshooting VMware Cloud Foundation Deployment. 11 Click Download to download a detailed deployment report. This report includes information on assigned IP addresses and networks that were configured in your environment. 12 After bring-up is completed, click Finish. 13 Click Launch SDDC Manager. 14 Power off the VMware Cloud Builder appliance. About the Deployment Parameter Workbook The deployment parameter workbook contains worksheets categorizing the information required for deploying VMware Cloud Foundation. The information provided is used to create the management domain using the VMware Cloud Builder appliance. Before you begin filling in the deployment parameter workbook, download the workbook from VMware Customer Connect. VMware by Broadcom 20 VMware Cloud Foundation Deployment Guide The fields in yellow contain sample values that you should replace with the information for your environment. If a cell turns red, the required information is missing, or validation input has failed. Important The deployment parameter workbook is not able to fully validate all inputs due to formula limitations of Microsoft Excel. Some validation issues may not be reported until you upload the deployment parameter workbook to the VMware Cloud Builder appliance. Note Do not copy and paste content between cells in the deployment parameter workbook, since this may cause issues. The Introduction worksheet in the deployment parameter workbook contains an overview of the workbook and guidance on how to complete it. For information about the prerequisites for deploying the management domain, see the Planning and Preparation Workbook. Credentials Worksheet The Credentials worksheet details the accounts and initial passwords for the VMware Cloud Foundation components. You must provide input for each yellow box. A red cell may indicate that validations on the password length has failed. Input Required Update the Default Password field for each user (including the automation user in the last row). Passwords can be different per user or common across multiple users. The tables below provide details on password requirements. Table 2-1. Password Complexity Password Requirements ESXi Host root account This is the password which you configured on the hosts during ESXi installation. Default Single-Sign on domain 1 Length 8-20 characters administrator user 2 Must include: n mix of upper-case and lower-case letters n a number n a special character, such as @ ! # $ % ^ or ? 3 Must not include * { } [ ] ( ) / \ ' " ` ~ , ; :. < > vCenter Server virtual appliance root 1 Length 8-20 characters account 2 Must include: n mix of upper-case and lower-case letters n a number n a special character, such as @ ! # $ % ^ or ? 3 Must not include: * { } [ ] ( ) / \ ' " ` ~ , ; :. < > VMware by Broadcom 21 VMware Cloud Foundation Deployment Guide Table 2-1. Password Complexity (continued) Password Requirements NSX virtual appliance root account 1 Length 12-127 characters 2 Must include: n mix of uppercase and lowercase letters n a number n a special character, such as @ ! # $ % ^ or ? n at least five different characters 3 Must not include: * { } [ ] ( ) / \ ' " ` ~ , ; :. < > NSX user interface and default CLI admin 1 Length 12-127 characters account 2 Must include: n mix of uppercase and lowercase letters n a number n a special character, such as @ ! # $ % ^ or ? n at least five different characters 3 Must not include: * { } [ ] ( ) / \ ' " ` ~ , ; :. < > NSX audit CLI account 1 Legnth 12-127 characters 2 Must include: n mix of uppercase and lowercase letters n a number n a special character, such as @ ! # $ % ^ or ? n at least five different characters 3 Must not include: * { } [ ] ( ) / \ ' " ` ~ , ; :. < > SDDC Manager appliance root account 1 Length 8-20 characters 2 Must include: n mix of uppercase and lowercase letters n a number n a special character, such as @ ! # $ % ^ or ? 3 Must not include: n *{}[]()/\'"`~,;:. n A dictionary word (for example, VMware1!) VMware by Broadcom 22 VMware Cloud Foundation Deployment Guide Table 2-1. Password Complexity (continued) Password Requirements SDDC Manager super user (vcf) 1 Length 8-20 characters 2 Must include: n mix of uppercase and lowercase letters n a number n a special character, such as @ ! # $ % ^ or ? 3 Must not include: n *{}[]()/\'"`~,;:. n A dictionary word (for example, VMware1!) SDDC Manager local account (admin@local) 1 Length 12-20 characters 2 Must include: n mix of uppercase and lowercase letters n a number n a special character, such as @ ! # $ % ^ or ? 3 Must not include: * { } [ ] ( ) / \ ' " ` ~ , ; :. < > Hosts and Networks Worksheet The Hosts and Networks worksheet specifies the details for all networks and hosts. This information is configured on the appropriate VMware Cloud Foundation components. Management Domain Networks This section covers the VLANs, gateways, MTU, and expected IP ranges and subnet mask for each network you have configured on the Top of Rack switches in your environment. VMware Cloud Foundation 5.1 introduces the ability to create separate distibuted port groups for management VM (for example, vCenter Server and NSX Manager) traffic and ESXi host management traffic. n If you enter information for the VM Management Network, VMware Cloud Foundation creates a distibuted port group for the VM Management Network using the information you provide. n If you do not enter information for the VM Management Network, VMware Cloud Foundation still creates a distibuted port group for VM Management Network, but uses the Management Network information (gateway, VLAN, MTU). Network Type VLAN Portgroup Name CIDR Notation Gateway MTU VM Management Enter the VLAN Enter a Enter the CIDR Enter the Enter MTU for Network ID. portgroup name. notation for the gateway IP for the network. network. network. The MTU can Management be between 1500 Network and 9000. VMware by Broadcom 23 VMware Cloud Foundation Deployment Guide Network Type VLAN Portgroup Name CIDR Notation Gateway MTU vMotion Network The VLAN ID can be between 0 vSAN Network and 4094. Note The VLAN ID for Uplink 1 and Uplink 2 Networks must be unique and not used by any other network type. Virtual Networking The deployment parameter workbook provides three vSphere Distributed Switch profiles that allow you to perform bring-up of hosts with two or four pNICs and to create up to two vSphere Distributed Switches for isolating VMkernel traffic. The information that you are required to provide depends on the profile that you select. Note You can use the VMware Cloud Foundation API to perform bring-up with other combinations of vSphere Distributed Switches and pNICs that are not available using the vSphere Distributed Switch profiles. vSphere Distributed Switch Profile Description Profile 1 n One vSphere Distributed Switch (vDS): Traffic for Management, vMotion, vSAN, and Host Overlay networks using specified pNICs. n Two or four physical NICs (pNICs) Profile 2 n Two vSphere Distributed Switches (vDS) n Four physical NICs (pNICs) n Primary vDS: Traffic for Management, vMotion, and Host Overlay networks using specified pNICs. n Secondary vDS: Traffic for vSAN network using specified pNICs. Profile 3 n Two vSphere Distributed Switches (vDS) n Four physical NICs (pNICs) n Primary vDS: Traffic for Management, vMotion, and vSAN networks using specified pNICs. n Secondary vDS: Traffic for Host Overlay network using specified pNICs. After you select a vSphere Distributed Switch Profile, enter the required information for that profile. VMware by Broadcom 24 VMware Cloud Foundation Deployment Guide vSphere Standard Switch Name Enter a name for the vSphere Standard Switch. Primary vSphere Distributed Switch - Name Enter a name for the primary vSphere Distributed Switch (vDS). You can modify the portgroup names of the management domain networks to make it clear which vDS each network uses. Primary vSphere Distributed Switch - pNICs Select the physical NICs to assign to the primary vDS. Primary vSphere Distributed Switch - MTU Size Enter the MTU size for the primary vDS. Default value is 9000. Primary vSphere Distributed Switch - Transport Zone Type Select Overlay or VLAN. Secondary vSphere Distributed Switch - Name Enter a name for the secondary vSphere Distributed Switch (vDS). You can modify the portgroup names of the management domain networks to make it clear which vDS each network uses. Note If you are not creating a secondary vDS, enter n/a. Secondary vSphere Distributed Switch - Transport Zone Select Overlay or VLAN. Type Secondary vSphere Distributed Switch - pNICs Select the physical NICs to assign to the secondary vDS. Secondary vSphere Distributed Switch - MTU Size Enter the MTU size for the secondary vDS. Default value is 9000. Management Domain ESXi Hosts Specify the IP addresses of the ESXi hosts for the management domain. In a standard deployment, only four hosts are required in the management domain. VMware Cloud Foundation can also be deployed with a consolidated architecture. In a consolidated deployment, all workloads are deployed in the management domain instead of to separate workload domains. As such, additional hosts may be required to provide the capacity needed. In this section, only enter values for the number of hosts desired in the management domain. Host Name IP Address Enter host names for each of the four ESXi hosts. Enter IP Address for each of the four ESXi hosts. Inclusion Ranges Specify IP inclusion ranges for the vSAN and vMotion networks of the management domain. IP addresses from the specified range are automatically assigned to hosts. Ensure that the IP ranges include sufficient IP addresses for the initial deployment. The number of IP addresses must be at least equal to the number of hosts deployed as part of VMware Cloud Foundation. As an example, if you specify the range start value as 192.168.1.1 and end as 192.168.1.20, a total of 20 IP addresses would be used. Do not use special IP addresses, such as the network or broadcast address. VMware by Broadcom 25 VMware Cloud Foundation Deployment Guide IPs for the vMotion range must be part of the VLAN configured with the vMotion portgroup. IPs for the vSAN range must be part of the VLAN configured for the vSAN portgroup. All IPs within the range must be available for use or IP conflicts will occur. It is a good practice to validate this prior to starting a deployment. Table 2-2. Input Required Network Start IP End IP vMotion Enter start of IP address range for Enter end of IP address range. vMotion network. VSAN Enter start of IP address range for Enter end of IP address range. vMotion network. ESXi Host Security Thumbprints If you want bring-up to validate the SSH fingerprint and SSL thumbprints of the ESXi hosts before connecting to them to reduce the chance of Man In The Middle (MiTM) attack, select Yes in the Validate Thumbprints field. If you set Validate Thumbprints to Yes, follow the steps below. 1 In a web browser, log in to the ESXi host using the VMware Host Client. 2 In the navigation pane, click Manage and click the Services tab. 3 Select the TSM-SSH service and click Start if not started. 4 Connect to the VMware Cloud Builder appliance using an SSH client such as Putty. 5 Enter the admin credentials you provided when you deployed the VMware Cloud Builder appliance. 6 Retrieve the SSH fingerprint by entering the following command replacing hostname with the FQDN of your host: ssh-keygen -lf /dev/null) 7 Retrieve the SSL thumbprint by entering the following command replacing hostname with the FQDN of your host: openssl s_client -connect hostname:443 < /dev/null 2> /dev/null | openssl x509 -sha256 -fingerprint -noout -in /dev/stdin 8 In the VMware Host Client, select the TSM-SSH service for the ESXi host and click Stop. 9 Repeat for each ESXi host and then enter the information in the deployment parameter workbook. VMware by Broadcom 26 VMware Cloud Foundation Deployment Guide NSX Host Overlay Network By default, VMware Cloud Foundation uses DHCP for the management domain Host Overlay Network TEPs. For this option, a DHCP server must be configured on the NSX host overlay (Host TEP) VLAN of the management domain. When NSX creates TEPs for the VI workload domain, they are assigned IP addresses from the DHCP server. For the management domain and VI workload domains with uniform L2 clusters, you can choose to use static IP addresses instead. Make sure the IP range includes enough IP addresses for the number of hosts that will use the static IP Pool. The number of IP addresses required depends on the number of pNICs on the ESXi hosts that are used for the vSphere Distributed Switch that handles host overlay networking. For example, a host with four pNICs that uses two pNICs for host overlay traffic requires two IP addresses in the static IP pool.. Table 2-3. DHCP Settings Parameter Value VLAN ID Enter a VLAN ID for the NSX host overlay network. The VLAN ID can be between 0 and 4094. Configure NSX Host Overlay Using a Static IP Pool Select No to use DHCP. Table 2-4. Static IP Pool Settings Parameter Value VLAN ID Enter a VLAN ID for the NSX host overlay network. The VLAN ID can be between 0 and 4094. Configure NSX Host Overlay Using a Static IP Pool Select Yes to use a static IP pool. Pool Description Enter a description for the static IP pool. Pool Name Enter a name for the static IP pool. CIDR Notation Enter CIDR notation for the NSX Host Overlay network. Gateway Enter the gateway IP address for the NSX Host Overlay network. NSX Host Overlay Start IP Enter the first IP address to include in the static IP pool. NSX Host Overlay End IP Enter the last IP address to include in the static IP pool. Deploy Parameters Worksheet: Existing Infrastructure Details Your existing DNS infrastructure is used to provide forward and reverse name resolution for all hosts and VMs in the VMware Cloud Foundation SDDC. External NTP sources are also utilized to synchronize the time between the software components. VMware by Broadcom 27 VMware Cloud Foundation Deployment Guide Table 2-5. Infrastructure Parameter Value DNS Server #1 Enter IP address of first DNS server. DNS Server #2 Enter IP address of second DNS server. Note If you have only one DNS server, enter n/a in this cell. NTP Server #1 Enter IP address or FQDN of first NTP server. NTP Server #2 Enter IP address or FQDN of second NTP server. Note If you have only one NTP server, enter n/a in this cell. Table 2-6. DNS Zone Parameter Value DNS Zone Name Enter root domain name for your SDDC management components. Note VMware Cloud Foundation expects all components to be part of the same DNS zone. Table 2-7. Customer Experience Improvement Program Parameter Value Enable Customer Select an option to activate or deactivate CEIP across vSphere, NSX, and vSAN during bring- Experience up. Improvement Program (“CEIP”) Table 2-8. Enable FIPS Security Mode on SDDC Manager Parameter Value Enable FIPS Security Select an option to activate or deactivate FIPS security mode during bring-up. VMware Mode on SDDC Cloud Foundation supports Federal Information Processing Standard (FIPS) 140-2. FIPS Manager 140-2 is a U.S. and Canadian government standard that specifies security requirements for cryptographic modules. When you enable FIPS compliance, VMware Cloud Foundation enables FIPS cipher suites and components are deployed with FIPS enabled. To learn more about support for FIPS 140-2 in VMware products, see https:// www.vmware.com/security/certifications/fips.html. Note This option is only available for new VMware Cloud Foundation installations and the setting you apply during bring-up will be used for future upgrades. You cannot change the FIPS security mode setting after bring-up. Deployment Parameters Worksheet: License Keys Provide licensing information for VMware Cloud Foundation. VMware by Broadcom 28 VMware Cloud Foundation Deployment Guide The License Keys options are dependent on the version of VMware Cloud Foundation you are deploying: n For VMware Cloud Foundation 5.1: a Select No for Use Keyless Licensing. Important Deploying VMware Cloud Foundation with keyless licensing is no longer supported. Do not select Yes for Use Keyless Licensing. b In the License Keys section, update the red fields with your license keys. Ensure the license key matches the product listed in each row and that the license key is valid for the version of the product listed in the VMware Cloud Foundation BOM. The license key audit during bring-up validates both the format and validity of the key. n For VMware Cloud Foundation 5.1.1: a Select Yes or No for License Now. b If you select Yes, in the License Keys section, update the red fields with your license keys. Ensure the license key matches the product listed in each row and that the license key is valid for the version of the product listed in the VMware Cloud Foundation BOM. The license key audit during bring-up validates both the format and validity of the key. c If you select No, the VMware Cloud Foundation components are deployed in evaluation mode. Important After bring-up, you must switch to licensed mode by adding component license keys in the SDDC Manager UI or adding and assigning a solution license key in the vSphere Client. See the VMware Cloud Foundation Administration Guide for information about adding component license keys in the SDDC Manager UI. See Managing vSphere Licenses for more information about adding and applying a solution license key for VMware ESXi and vCenter Server in the vSphere Client. If you are using a solution license key, you must also add a separate VMware vSAN license key for vSAN clusters. See Configure License Settings for a vSAN Cluster. Deploy Parameters Worksheet: vSphere Infrastructure The vSphere infrastructure section of the Deploy Parameters Worksheet details how you want to configure the vCenter Server and its related objects. This section of the deployment parameter workbook contains sample configuration information, but you can update them with names that meet your naming standards. Note All host names entries within the deployment parameter workbook expect the short name. VMware Cloud Builder takes the host name and the DNS zone provided to calculate the FQDN value and performs validation prior to starting the deployment. The specified host names and IP addresses must be resolvable using the DNS servers provided, both forward (hostname to IP) and reverse (IP to hostname), otherwise the bring-up process will fail. VMware by Broadcom 29 VMware Cloud Foundation Deployment Guide Table 2-9. vCenter Server Parameter Host Name IP Address vCenter Server Enter a host name for the vCenter Enter the IP address for the Server. vCenter Server that is part of the management VLAN. Note This is the same VLAN and IP address space where the ESXi management VMKernels reside. vCenter Server Appliance Size This parameter defines the size of the vCenter Server to be deployed. Default (Default Small) size is Small. Additional options are: Tiny, Medium, Large, and X-large. See Hardware Requirements for the vCenter Server Appliance. vCenter Server Appliance Storage The amount of storage depends on the vCenter Server appliance size. See Size Storage Requirements for the vCenter Server Appliance. Table 2-10. vCenter Datacenter and Cluster Parameter Value Datacenter Name Enter a name for the management datacenter. Cluster Name Enter a name for the management cluster. Enable vLCM Cluster Image Select Yes to use vSphere Lifecycle Manager images for managing the lifecycle of ESXi hosts in the primary cluster of management domain. VMware Cloud Builder extracts a vSphere Lifecycle Manager image from the first ESXi host and applies that image to all the hosts in the cluster. The vSphere Lifecycle Manager image is also imported into SDDC Manager (available at Lifecycle Management > Image Management. Note vSAN Express Storage Architecture (ESA) requires vSphere Lifecycle Manager images. Select No to use vSphere Lifecycle Manager baselines for managing the lifecycle of ESXi hosts in the primary cluster of management domain. Cluster EVC Setting To enable EVC on the management cluster, select the CPU chipset that should be applied to enhance vMotion compatability. Note If you don't want to enable EVC, enter n/a in this cell. Select the architecture model you plan to use. If you choose Consolidated, specify the names for the vSphere resource pools. You do not need to specify resource pool names if you are using the standard architecture model. See Introducing VMware Cloud Foundation for more information about these architecture models. VMware by Broadcom 30 VMware Cloud Foundation Deployment Guide Table 2-11. vSphere Resource Pools Parameter Value Resource Pool SDDC Management Specify the vSphere resource pool name for management VMs. Resource Pool SDDC Edge Specify the vSphere resource pool name for NSX VMs. Resource Pool User Edge Specify the vSphere resource pool name for user deployed NSX VMs in a consolidated architecture. Resource Pool User VM Specify the vSphere resource pool name for user deployed workload VMs. Note Resource pools are created with Normal CPU and memory shares. Table 2-12. vSphere Datastore Parameter Value vSAN Datastore Name Enter vSAN datastore name for your management components. Enable vSAN Deduplication and Compression Select Yes to turn on Dedupe and Compression capabilities of vSAN. Note This option is only available with vSAN OSA. If you enable vSAN ESA, deduplication and compression settings can be specified in the vSAN storage policies using the vSphere Client. VMware by Broadcom 31 VMware Cloud Foundation Deployment Guide Table 2-12. vSphere Datastore (continued) Parameter Value Enable vSAN-ESA Select Yes to use vSAN Express Storage Architecture (ESA) for the first cluster in the management domain. After bringup, you can create additional clusters (vSAN ESA or vSAN OSA) in the management domain. Note vSAN ESA requires the use of vLCM images and is not supported with vLCM baselines. Important You cannot stretch clusters that use vSAN ESA. vSAN ESA is designed for high-performance NVMe based TLC flash devices and high performance networks. Each host that contributes storage contains a single storage pool of four or more flash devices. Each flash device provides caching and capacity to the cluster. Select No to use vSAN Original Storage Architecture (OSA) for the first cluster in the management domain. After bringup, you can create additional clusters (vSAN ESA or vSAN OSA) in the management domain, but you can create vSAN ESA clusters only if the management domain is using vLCM images. For an overview of the differences between vSAN OSA and vSAN ESA, see Building a vSAN Cluster in the vSphere documentation.. Path to HCL JSON File vSAN ESA requires a current version of the vSAN HCL JSON file to ensure that your ESXi hosts are ESA- compatible. If the VMware Cloud Builder appliance is not able to connect to the internet (either directly or through a proxy server), download the latest vSAN HCL JSON file from https://partnerweb.vmware.com/service/vsan/all.json and copy it to the VMware Cloud Builder appliance. Enter to path to the vSAN HCL JSON file on the VMware Cloud Builder appliance. For example: /opt/vmware/ bringup/tmp/all.json If the VMware Cloud Builder appliance does not have direct internet access, you can configure a proxy server to download the vSAN HCL JSON. A recent version of the HCL JSON file is required for vSAN ESA. Table 2-13. Proxy Server Configuration Parameter Value Proxy Server Configuration Select Yes to configure a proxy server. Proxy Server Enter the proxy server FQDN or IP address. Proxy Port Enter the proxy server port. VMware by Broadcom 32 VMware Cloud Foundation Deployment Guide Deploy Parameters Worksheet: VMware NSX The NSX section of the Deploy Parameters Worksheet specifies the details you want to use for deploying VMware NSX components. Table 2-14. NSX Management Cluster Parameter Value NSX Management Cluster VIP Enter the host name and IP address for the NSX Manager VIP. The host name can match your naming standards but must be registered in DNS with both forward and reverse resolution matching the specified IP. Note This is the same VLAN and IP address space where the vCenter and ESXi management VMKernels reside. NSX Virtual Appliance Node #1 Enter the host name and IP address for the first node in the NSX Manager cluster. NSX Virtual Appliance Node #2 Enter the host name and IP address for the second node in the NSX Manager cluster. NSX Virtual Appliance Node #3 Enter the host name and IP address for the third node in the NSX Manager cluster. NSX Virtual Appliance Size Select the size for the NSX Manager virtual appliances. The default is medium. Deploy Parameters Worksheet: SDDC Manager The SDDC Manager section of the Deploy Parameters Worksheet specifies the details for deploying SDDC Manager. Table 2-15. SDDC Manager Parameter Value SDDC Manager Hostname Enter a host name for the SDDC Manager VM. SDDC Manager IP Address Enter an IP address for the SDDC Manager VM. Network Pool Name Enter the network pool name for the management domain network pool. Cloud Foundation Management Domain Name Enter a name for the management domain. This name will appear in Inventory > Workload Domains in the SDDC Manager UI. Deploy the Management Domain Using ESXi Hosts with External Certificates VMware Cloud Foundation supports vCenter Server's Custom Certificate Authority mode during bring-up using the VMware Cloud Foundation API. Use this mode if you want to use only external certificates that are signed by a third-party or enterprise CA. In this mode, you are responsible VMware by Broadcom 33 VMware Cloud Foundation Deployment Guide for managing the certificates. You cannot refresh and renew external certificates from the SDDC Manager or vSphere Client. To use external ESXi certificates, you must create a custom JSON file for bring-up. You cannot use the deployment parameter workbook. Deploying the management domain with external ESXi certificates enables Custom Certificate Authority mode, so all future hosts that you add to a workload domain (management or VI) must also use external ESXi certificates. Prerequisites See Configure ESXi Hosts with Signed Certificates. Procedure 1 Create a JSON file populated with the bring-up information for your environment. You can see a sample JSON specification in the VMware Cloud Foundation API Reference Guide. 2 Update the securitySpec section, choosing Custom for the esxiCertsMode and entering your signing CA chain for certChain. For example: "securitySpec" : { "esxiCertsMode" : "Custom", "rootCaCerts" : [ { "alias" : "Rainpole-CA", "certChain" : [ "-----BEGIN CERTIFICATE----- MIIDczCCAlugAwIBAgIQI9xwbTkI9J5GhMffcP5CHDANBgkqhkiG9w0BAQsFADBM MRIwEAYKCZImiZPyLGQBGRYCaW8xGDAWBgoJkiaJk/IsZAEZFghyYWlucG9sZTEc MBoGA1UEAxMTcmFpbnBvbGUtZGMwMXJwbC1DQTAeFw0yMDAzMzAxNDQ2MTNaFw0y NTAzMzAxNDU2MTNaMEwxEjAQBgoJkiaJk/IsZAEZFgJpbzEYMBYGCgmSJomT8ixk ARkWCHJhaW5wb2xlMRwwGgYDVQQDExNyYWlucG9sZS1kYzAxcnBsLUNBMIIBIjAN BgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAzpwkz7aPlQcfevcCelHc9DPswHkd kjY96Vh3GvYlesaVEcy/q/BOvvh3KgLMLy8r7cy2cNPO3FANKOfqVdVx3ghfEUyL g61W9BskAlwryzJRmjhOJJVqvB8CWjy+eCp7MejHGdEud6WdEvK8CaBcPngEg0KM eLRNLGe8OCw8yY4GTrjU+H7PYQZtyD0kxxy5f48ueaDXat4ENRGcAuHEfCoMGfaR bDue1OO4diHd900bCym5ggBNX0jhRudNULXPTayZl2ksImV0+QkaVeptQImXfCgb kgnHQJ5CxK26up7fB5eAsmGLAsJLBnHuM7P9xvV09EvWjFCgLX/oBBDYTQIDAQAB o1EwTzALBgNVHQ8EBAMCAYYwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHQ4EFgQU7oOq QBK8yg8mHnAfb+u6/GO0ZUcwEAYJKwYBBAGCNxUBBAMCAQAwDQYJKoZIhvcNAQEL BQADggEBALYxZGj4vWjFDN1atOUsBx2jrmxbExgMAyRpNlSc2aj+7vzxHxUW5VbX x9nc/BfkTiCK6c7Y9VYb+mgjb8z0kNv58sT4ar1yIl1n63VOCoyyLcaFB8HyEJpD wUhz4RNPoSijZMpm+M5EuSLfWlhEJo7N8sLqHgvvk1dFpbK8fIHbPS5KJwJibbPe w9UuNRdcxN9hFWKBC0SvfgX+1CJxVdvgfi65rSHPuWinJzrXXdH999DfpDESRzwH 0pqE3GtMCt1Nqalp2QJFdahbT+kxj7QWHTjUylSENDHjdln7a8WH8RGxvEy/97YZ +crXmxvQ/bAgHk9vcRERbRjfyIs7v88= -----END CERTIFICATE-----" ] } ] } 3 Follow the steps outlined in the VMware Cloud Foundation API Reference Guide to deploy the management domain. VMware by Broadcom 34 Troubleshooting VMware Cloud Foundation Deployment 3 During the deployment stage of VMware Cloud Foundation you can use log files and the Supportability and Serviceability (SoS) Tool to help with troubleshooting. Read the following topics next: n VMware Cloud Builder Log Files n Using the SoS Utility on VMware Cloud Builder VMware Cloud Builder Log Files VMware Cloud Builder contains various log files for different components of the system. VMware Cloud Builder has a number of components which are used during the bring-up process, each component generates a log file which can be used for the purpose of troubleshooting. The components and their purpose are: n JsonGenerator: Used to convert the deployment parameter workbook into the required configuration file (JSON) that is used by the Bringup Validation Service and Bringup Service. n Bringup Service: Used to perform the validation of the configuration file (JSON), the ESXi hosts and infrastructure where VMware Cloud Foundation will be deployed, and to perform the deployment and configuration of the management domain components and the first cluster. n Supportability and Serviceability (SoS) Utility: A command line utility for troubleshooting deployment issues. The following table describes the log file locations: Component Log Name Location JsonGenerator jsongenerator-timestamp /var/log/vmware/vcf/sddc-support/ Bringup Service vcf-bringup.log /var/log/vmware/vcf/bringup/ vcf-bringup-debug.log /var/log/vmware/vcf/bringup/ rest-api-debug.log /var/log/vmware/vcf/bringup/ SoS Utility sos.log /var/log/vmware/vcf/sddc-support/ sos-timestamp/ VMware by Broadcom 35 VMware Cloud Foundation Deployment Guide Using the SoS Utility on VMware Cloud Builder You can run the Supportability and Serviceability (SoS) Utility on the VMware Cloud Builder appliance to generate a support bundle, which you can use to help debug a failed bring-up of VMware Cloud Foundation. Note After a successful bring-up, you should only run the SoS Utility on the SDDC Manager appliance. See Supportability and Serviceability (SoS) Tool in the VMware Cloud Foundation Administration Guide. The SoS Utility is not a debug tool, but it does provide health check operations that can facilitate debugging a failed deployment. To run the SoS Utility in VMware Cloud Builder, SSH in to the VMware Cloud Builder appliance using the admin administrative account, then enter su to switch to the root user, and navigate to the /opt/vmware/sddc-support directory and type./sos followed by the options required for your desired operation../sos --option-1 --option-2... --option-n SoS Utility Help Options Use these options to see information about the SoS tool itself. Option Description --help Provides a summary of the available SoS tool options -h --version Provides the SoS tool's version number. -v SoS Utility Generic Options These are generic options for the SoS Utility. Option Description --configure-sftp Configures SFTP for logs. --debug-mode Runs the SoS tool in debug mode. --force Allows SoS operations from theVMware Cloud Builder appliance after bring- up. Note In most cases, you should not use this option. Once bring-up is complete, you can run the SoS Utility directly from the SDDC Manager appliance. --history Displays the last twenty SoS operations performed. --log-dir LOGDIR Specifies the directory to store the logs. VMware by Broadcom 36 VMware Cloud Foundation Deployment Guide Option Description --log-folder LOGFOLDER Specifies the name of the log directory. --setup-json SETUP_JSON Custom setup-json file for log collection. SoS prepares the inventory automatically based on the environment where it is running. If you want to collect logs for a pre-defined set of components, you can create a setup.json file and pass the file as input to SoS. A sample JSON file is available on the VMware Cloud Builder in the /opt/vmware/ sddc-support/ directory. --skip-known-host-check Skips the specified check for SSL thumbprint for host in the known host. --zip Creates a zipped tar file for the output. SoS Utility Log File Options Option Description --api-logs Collects output from APIs. --cloud-builder-logs Collects Cloud Builder logs. --esx-logs Collects logs from the ESXi hosts only. Logs are collected from each ESXi host available in the deployment. --no-clean-old-logs Use this option to prevent the tool from removing any output from a previous collection run. By default, before writing the output to the directory, the tool deletes the prior run's output files that might be present. If you want to retain the older output files, specify this option. --no-health-check Skips the health check executed as part of log collection. --nsx-logs Collects logs from the NSX Manager instances only. --rvc-logs Collects logs from the Ruby vSphere Console (RVC) only. RVC is an interface for ESXi and vCenter. Note If the Bash shell is not enabled in vCenter, RVC log collection will be skipped. Note RVC logs are not collected by default with./sos log collection. --sddc-manager-logs Collects logs from the SDDC Manager only. --test Collects test logs by verifying the files. --vc-logs Collects logs from the vCenter Server instances only. Logs are collected from each vCenter server available in the deployment. --vm-screenshots Collects screen shots from all VMs. VMware by Broadcom 37 VMware Cloud Foundation Deployment Guide SoS Utility JSON Generator Options The JSON generator options within the SoS Utility provide a method to execute the creation of the JSON file from a completed deployment parameter workbook. To run the JSON generator, you must provide, as a minimum, a path to the deployment parameter workbook and the design type using the following syntax:./sos --jsongenerator --jsongenerator-input JSONGENERATORINPUT --jsongenerator-design JSONGENERATORDESIGN Option Description --jsongenerator Invokes the JSON generator utility. --jsongenerator-input Specify the path to the input file to be used by the JSON generator utility. JSONGENERATORINPUT For example: /tmp/vcf-ems-deployment-parameter.xlsx. --jsongenerator-design Use vcf-ems for VMware Cloud Foundation. JSONGENERATORDESIGN --jsongenerator-supress Supress confirmation to force cleanup directory. (optional) --jsongenerator-logs Set the directory to be used for logs. (optional) JSONGENERATORLOGS SoS Utility Health Check Options The SoS Utility can be used to perform health checks on various components or services, including connectivity, compute, and storage. Note The health check options are primarily designed to run on the SDDC Manager appliance. Running them on the VMware Cloud Builder appliance requires the --force parameter, which instructs the SoS Utility to identify the SDDC Manager appliance deployed by VMware Cloud Builder during the bring-up process, and then execute the health check remotely. For example:./sos --health-check --force Option Description --certificate-health Verifies that the component certificates are valid (within the expiry date). --connectivity-health Performs a connectivity health check to inspect whether the different components of the system such as the ESXi hosts, vCenter Servers, NSX Manager VMs, and SDDC Manager VM can be pinged. --compute-health Performs a compute health check. --general-health Verifies ESXi entries across all sources, checks the Postgres DB operational status for hosts, checks ESXi for error dumps, and gets NSX Manager and cluster status. --get-host-ips Returns server information. VMware by Broadcom 38 VMware Cloud Foundation Deployment Guide Option Description --health-check Performs all available health checks. --ntp-health Verifies whether the time on the components is synchronized with the NTP server in the VMware Cloud Builder appliance. --services-health Performs a services health check to confirm whether services are running --run-vsan-checks Runs proactive vSAN tests to verify the ability to create VMs within the vSAN disks. Sample Output The following text is a sample output from an --ntp-health operation. root@cloud-builder [ /opt/vmware/sddc-support ]#./sos --ntp-health --skip-known-host --force Welcome to Supportability and Serviceability(SoS) utility! User passed --force flag, Running SOS from Cloud Builder VM, although Bringup is completed and SDDC Manager is available. Please expe ct failures with SoS operations. Health Check : /var/log/vmware/vcf/sddc-support/healthcheck-2020-02-11-23-03-53-24681 Health Check log : /var/log/vmware/vcf/sddc-support/healthcheck-2020-02-11-23-03-53-24681/ sos.log SDDC Manager : sddc-manager.vrack.vsphere.local NTP : GREEN +-----+-----------------------------------------+------------+-------+ | SL# | Area | Title | State | +-----+-----------------------------------------+------------+-------+ | 1 | ESXi : esxi-1.vrack.vsphere.local | ESX Time | GREEN | | 2 | ESXi : esxi-2.vrack.vsphere.local | ESX Time | GREEN | | 3 | ESXi : esxi-3.vrack.vsphere.local | ESX Time | GREEN | | 4 | ESXi : esxi-4.vrack.vsphere.local | ESX Time | GREEN | | 5 | vCenter : vcenter-1.vrack.vsphere.local | NTP Status | GREEN | +-----+-----------------------------------------+------------+-------+ Legend: GREEN - No attention required, health status is NORMAL YELLOW - May require attention, health status is WARNING RED - Requires immediate attention, health status is CRITICAL Health Check completed successfully for : [NTP-CHECK] The following text is sample output from a --vm-screenshots log collection operation. root@cloud-builder [ /opt/vmware/sddc-support ]#./sos --vm-screenshots --skip-known-host --force Welcome to Supportability and Serviceability(SoS) utility! User passed --force flag, Running SOS from Cloud Builder VM, although Bringup is completed and SDDC Manager is available. Please expect failures with SoS operations. Logs : /var/log/vmware/vcf/sddc-support/sos-2018-08-24-10-50-20-8013 VMware by Broadcom 39 VMware Cloud Foundation Deployment Guide Log file : /var/log/vmware/vcf/sddc-support/sos-2018-08-24-10-50-20-8013/sos.log Log Collection completed successfully for : [VMS_SCREENSHOT] VMware by Broadcom 40 VMware Cloud Foundation Glossary 4 In VMware Cloud Foundation, you perform specific operations and use unique constructs for automated SDDC deployment and maintenance. Term Description availability zone A collection of infrastructure components. Each availability zone is isolated from the other availability zones to prevent the propagation of failure or outage across the data center. In VMware Cloud Foundation, you implement availability of workloads across availability zones by using vSAN stretched clusters. Application virtual networks Virtual networks backed by overlay or VLAN NSX segments using the encapsulation (AVNs) protocol of VMware NSX. An AVN uses a single IP address space to span across data centers. bring-up Deployment and initial configuration of a VMware Cloud Foundation system. During the bring-up process, the management domain is created and the VMware Cloud Foundation software stack is deployed on the management domain. commission a host Adding a host to VMware Cloud Foundation inventory. The host becomes unassigned. composability The ability to dynamically configure servers to meet the needs of your workloads without physically moving any hardware components. You bind disaggregated hardware components (compute, network, storage, and offload components) together to create a logical system based on the needs of your applications. dirty host A host that has been removed from a cluster in a workload domain. A dirty host cannot be assigned to another workload domain until it is decommissioned, re-imaged, and commissioned again. decommission a host Removing an unassigned host from the VMware Cloud Foundation inventory. SDDC Manager does not manage decommissioned hosts. NSX Edge cluster A logical grouping of NSX Edge nodes. These nodes run on a vSphere cluster, and provide north-south and east-west routing and network services for the management or VI workload domain. free pool Hosts in the VMware Cloud Foundation inventory that are not assigned to a workload domain. host A server that is imaged with the ESXi software. install bundle Contains software for VI workload domains and VMware Aria Suite Lifecycle. You can use an install bundle to deploy later versions of the software components in a new VI workload domain than the versions in the Bill of Materials for VMware Cloud Foundation. inventory Logical and physical entities managed by VMware Cloud Foundation. VMware by Broadcom 41 VMware Cloud Foundation Deployment Guide Term Description Kubernetes - Workload With Kubernetes - Workload Management, you can deploy and operate the compute, Management networking, and storage infrastructure for vSphere with Tanzu workloads. A vSphere with Tanzu workload is an application with containers running inside vSphere pods, regular VMs, or Tanzu Kubernetes clusters. Lifecycle Manager (LCM) Automates patching and upgrading of the software stack. management domain One or more vSphere clusters of physical hosts that contain the management component VMs, such as vCenter Server, NSX Manager cluster, management NSX Edge cluster, SDDC Manager, and so on. The management domain supports only vSAN storage. network pool Automatically assigns static IP addresses to vSAN and vMotion VMkernel ports so that you don't need to enter IP addresses manually when creating a VI workload domain or adding a host or cluster to a workload domain. update bundle Contains software to update the VMware Cloud Foundation components in your management or VI workload domain. principal storage Required for each vSphere cluster, containing the data of the virtual machines in the cluster. For the management domain, only vSAN principal storage is supported. For a VI workload domain, you set the principal storage when creating the domain or when adding a cluster to the domain. You cannot change the principal storage later. See also supplemental storage. SDDC Manager A software component that provisions, manages, and monitors the logical and physical resources of a VMware Cloud Foundation system. SDDC Manager provides the user interface for managing VMware Cloud Foundation, CLI-based administrator tools, and an API for further automation. server A bare-metal server in a physical rack. After imaging, it is referred to as a host. supplemental storage Extends the capacity of the workload domain for hosting more virtual machines or storing supporting data, such as backups. You can add or remove supplemental storage to clusters in the management or VI workload domain at any time. unassigned host A host in the free pool that does not belong to a workload domain. vSphere Lifecycle Manager A vCenter Server service, which is integrated with VMware Cloud Foundation, that (vLCM) enables centralized and simplified life cycle management of ESXi hosts. virtual infrastructure (VI) One or more vSphere clusters that contain customer workloads. VMware Cloud workload domain Foundation scales and manages the life cycle of each VI workload domain ind

Deploy and Configure a VCF Management Domain Using VMware Cloud Builder PDF

Document Details

Tags

Related

Summary

Full Transcript

Upgrade to continue