C270 DevOps Fundamentals Module 7 PDF

7.1 Automating Infrastructure with Cisco © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 1  Academic Infrastructure and Automation use Introduction to Automating Infrastructure Automati...

7.1 Automating Infrastructure with Cisco © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 1  Academic Infrastructure and Automation use Introduction to Automating Infrastructure Automation is using code to configure, deploy, and manage applications together with the compute, storage, and network infrastructures and services on which they run. For automation with Cisco infrastructure, the platforms can integrate with the common tools such as Ansible, Puppet, Chef and so on, or provide direct API access to the programmable infrastructure. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 2  Academic Infrastructure and Automation use Cisco Automation Solutions There are several use cases for automation for the network. Depending on the operational model to be followed, there are choices to programmatically control the network configurations and infrastructure. Run: Activate policies and Fly: Deploy applications, Walk: Read only automation provide self-service across network configurations, and multiple domains more through CI/CD Using automation tools, With these Run stage For more complex automation information can be gathered automation scenarios, the and programmable examples, about the network users can safely provision their the Fly stage of the DevNet configuration. own network updates. Automation Exchange is used. One can use a read scenario On-boarding workflows can be Here the needs can be to audit configurations and do automated, day-to-day network managed by monitoring and the next natural step, which is configurations can be proactively managing the users to put the configuration back managed, and daily scenarios and devices, while also gaining into compliance. can be pervaded. insights with telemetry data. In the Automation Exchange, this shift is categorized as a walk-run-fly progression. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 3  Academic Infrastructure and Automation use Why Do We Need Automation? Speed and agility enable the business to explore, experiment with, and exploit opportunities ahead of their competition. Developers need to accelerate every phase of software building: coding and iterating, testing, and staging. DevOps practices require developers to deploy and manage apps in production, so developers should also automate those activities. Disadvantages of Manual Operations Add to costs, are time taking and are hard to scale. Are prone to human error, and documentation meant for humans is often incomplete and ambiguous, hard to test, and quickly outdated. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 4  Academic Infrastructure and Automation use Why Do We Need Automation? (Contd.) Dependency risks Today's software ecosystem is decentralized. Developers build individual components according to their needs and interests and mix and match components, infrastructure, and services needed to enable complete solutions and operate them efficiently at scale. This ecosystem introduces new requirements and new risks: Components need to be able to work alongside many other components in many different situations showing no more preference for specific companion components or architectures than absolutely necessary. Component developers may abandon support for obsolete features and rarely-encountered integrations. This disrupts processes that depend on those features. Dependency-ridden application setups tend to get locked into fragile and increasingly insecure deployment stacks. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 5  Academic Infrastructure and Automation use Why Do We Need Full - Stack Automation? Automation is a key component of functional software-defined infrastructure and distributed and dynamic applications. The benefits of full–stack automation are: Self-service: Automation provides self-service frameworks which enable users to requisition infrastructure on demand. Scale on demand: Apps and platforms need to be able to scale up and down in response to traffic and workload requirements and to use heterogeneous capacity. Observability: An observable system enables users to infer the internal state of a complex system from its outputs. Automated problem mitigation: The apps and platforms should be engineered to minimize the effects of issues, self-heal and monitor events. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 6  Academic Infrastructure and Automation use Software-Defined Infrastructure: A Case for Automation Software-defined infrastructure is also known as cloud computing. It lets the developers and operators to use the software to requisition, configure, deploy and manage bare-metal and virtualized compute, storage and network resources. Benefits of cloud paradigms Self-service (platforms on demand) Close specification, consistency, repeatability Platform abstraction Challenges faced in cloud paradigms Developers must pay close attention to platform design, architecture, and security. Access control is critical as cloud users with the wrong permissions can do a lot of damage to their organization's assets. When cloud resources can be self-served quickly via manual operations, consumption can be hard to manage, and costs are difficult to calculate. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 7 Infrastructure and Automation Academic use Distributed and Dynamic Applications: Another Case for Automation Modern application architectures are increasingly distributed. They are built up of small and relatively light components that are sometimes called microservices. These components may be isolated in containers, connected via discovery and messaging services (which abstract network connectivity) and backed by resilient, scalable databases (which maintain state). © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 8 Infrastructure and Automation Academic use Distributed and Dynamic Applications: Another Case for Automation (Contd.) Benefits of microservices: Challenges of microservices: Scalability - Microservices can be Increased complexity - Microservices scaled and load-balanced as needed mean that there are many moving parts across many networked servers or to configure and deploy. There are multiple geographically-separate public more demanding operations, including cloud regions. This eliminates single scaling-on-demand, self-healing and points of failure. other features. Infrastructure Automation tools - Automation is a requirement - Increasingly, the dynamism of Manual methods can not realistically microservice-based applications is cope with the complexity of deploying provided by infrastructure. These and managing dynamic applications. container automate on-demand scaling, self-healing, and more. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 9  Academic Infrastructure and Automation use Automating Infrastructure Summary These business and technical needs, trends, and dynamics, encourage developers and operators to use automation everywhere for the following tasks: Manage all phases of app building, configuration, deployment and lifecycle management. This includes coding, testing, staging, and production. Manage software-defined infrastructures on behalf of the applications you build. Alongside the applications, to preserve, update, and continually improve the automation code. This code helps to develop, test, stage, monitor, and operate the apps at production scales, and in various environments. Increasingly, all this code can be treated as one work-product. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 10 7.2 DevOps and SRE © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 11  Academic DevOps and SRE use Introduction to DevOps and SRE For full-stack automation to be truly effective, it requires changes to organizational culture, including breaking down the historical divides between Development (Dev) and Operations (Ops). Historically, creating applications was the job of software developers (Dev), and ensuring that apps work for users and the business has been the specialized province of IT operations (Ops). © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 12  Academic DevOps and SRE use DevOps Divide The following table describes the different characteristics of Dev and Ops: Characteristics Dev Ops Applications and how they run, plus Cares about Bespoke applications and how they work Infrastructure, OS, network, and so on Business treats Cost center: provides and accounts for Profit center: demands resources as resources Participates in Occasionally (only when issues are Regularly (point of spear) on-call rotation escalated to dev) Performance measured Abstractly (including bad metrics) Concretely (SLA compliance, issues resolved) More deep than broad: Languages, APIs, More broad than deep: Configuration, Skills required architecture, tools, process and so on administration, OS, automation, and so on Investments must be extensively justified, Agility required Move fast, innovate, break things, fix later expectations managed © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 13  Academic DevOps and SRE use DevOps Divide (Contd.) In the traditional, pre-virtualization, enterprise IT ecosystem, separating Dev from Ops seemed sensible. In the early 2000s, there began a movement to treat Dev and Ops as a single entity: Make coders responsible for deployment and maintenance. Treat virtualized infrastructure as code. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 14  Academic DevOps and SRE use Evolution of DevOps DevOps evolved and continues to evolve in many places in parallel. Some key events have shaped the discipline as we know it today. Defining Moments 1: Site Reliability Engineering (SRE): Institutionalization of SRE by Google in 2003. Defining Moments 2: Debois and Agile Infrastructure: Patrick Debois’ presentation in 2009 on automating virtual and physical infrastructure using version control and applying Agile methods. Defining Moments 3: Allspaw and Hammond: Presentation by John Allspaw and Paul Hammond in 2009 outlining a simple set of DevOps best practices founded on the idea that both Dev and Ops cooperatively enable the business. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 15  Academic DevOps and SRE use Core Principles of DevOps DevOps/SRE have many core principles and best practices: A focus on automation The idea that "failure is normal" A reframing of "availability" in terms of what a business can tolerate © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 16  Academic DevOps and SRE use Core Principles of DevOps (Contd.) SLOs, SLIs, and error budgets The two linked ideas to DevOps/SRE culture are DevOps must deliver measurable, agreed- upon business value and the statistical reality of doing so perfectly is impossible. These ideas are codified in a Service Level Objective (SLO) that is defined in terms of real metrics called Service Level Indicators (SLIs). SLIs map to the practical reality of delivering a service to customers. SLO/SLI methodology permits cheaper, more rapid delivery of business value by removing the obligation to seek perfection. It can also influence the pace, scope, and other aspects of development to ensure and improve adequacy. One way of modeling SLO/SLI results requires establishing an error budget for a service for a given period of time and then subtracting failures to achieve SLO from this value. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 17  Academic DevOps and SRE use DevOps and SRE Summary DevOps/SRE is co-evolving with technologies like virtualization and containerization, enabling a unified approach and unified tool set to support coordinated application and infrastructure engineering. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 18 7.3 Basic Automation Scripting © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 19  Academic Basic Automation Scripting use Introduction to Basic Automation Scripting Powerful automation tools like Ansible, Puppet, and Chef bring ease of use, predictability, discipline and the ability to work at scale to DevOps work. Although automation tooling partly works by wrapping shell functionality, operating system utilities, API functions and other control plane elements for simplicity, uniformity, feature enrichment, and compatibility in DevOps scenarios, these tools still do not solve every problem of deployment and configuration. Every automation tool has one or more functions that execute basic commands and scripts on targets and return results. It is therefore important to have basic automation scripting skills. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 20  Academic Basic Automation Scripting use Basic Tools for Automation Scripting Shells are ubiquitous, so shell scripting is historically the bedrock of automation. Bash The Bash is a Unix shell and is a default on most Linux distributions and on macOS. Using commands in a Bash script is much the same as using them directly from the command line. Bash can be used to script access to the AWS CLI, you can use the built-in features and libraries of more sophisticated languages to parse complex returned datasets (such as JSON), manage many parallel operations, process errors, handle asynchronous responses to commands, and more. Programming languages beyond Bash Sophisticated languages improve on Bash when complexity and scale requirements increase. They are useful when building and operating virtualized infrastructure in cloud environments, using SDKs. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 21  Academic Basic Automation Scripting use Procedural Automation An imperative procedure is an ordered sequence of commands aimed at achieving a goal. The sequence may include flow-control, conditions, functional structure, classes, and more. To ensure efficiency and reusability of scripts one can: Standardize the ordering and presentation of parameters, flags, and errors. Create a code hierarchy that divides tasks logically and efficiently. Create high-level scripts for entire deployments and lower-level scripts for deployment phases. Separate deployment-specific data from the code, making the code as generic and reusable as possible. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 22  Academic Basic Automation Scripting use Procedural Automation (Contd.) Carefully-written procedural scripts and declarative configuration tools examine targets before performing tasks on them, and only perform the tasks needed to achieve the desired state. This quality of software is called idempotency. Basic principles of idempotency are: Ensure the change you want to make has not already been made Get to a known-good state, if possible, before making changes Test for idempotency All components of a procedure must be idempotent © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 23  Academic Basic Automation Scripting use Executing Scripts Locally and Remotely To configure remote systems, the user need to access and execute scripts on them. There are many ways to do this: Store scripts locally, transmit them to target machines with a shell utility like scp, then log into the remote machine using ssh and execute them. Pipe scripts to a remote machine using cat | ssh and execute them in sequence with other commands, capturing and returning results to the terminal, all in one command. Install a general-purpose secure file-transfer client like SFTP, then use that utility to connect to the remote machine, transfer, set appropriate permissions, then execute the script file. Store scripts on a webserver, log into the remote machine and retrieve them with wget, curl, or other utilities, or store the scripts in a Git repository. Install a full remote-operations solution like VNC or NoMachine locally, install its server on the target, transmit/copy and then execute scripts. If the target devices are provisioned on a cloud framework, there is usually a way to inject a configuration script via the same CLI command or WebUI action that manifests the platform. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 24  Academic Basic Automation Scripting use Cloud Automation Cloud automation enables the user to provision virtualized hosts, configure virtual networks and other connectivity, requisition services, and then deploy applications on this infrastructure. Cloud providers and open source communities often provide specialized subsystems for popular deployment tools, which extract a complete inventory of resources from a cloud framework and keep it updated in real time while automation makes changes. The user can also manage cloud resources using scripts written in Bash, Python, or other languages. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 25  Academic Basic Automation Scripting use Cloud CLIs and SDKs IaaS and other types of infrastructure cloud also provide CLIs and SDKs that enable easy connection to their underlying interfaces, which are usually REST-based. Cisco UCS - a bare Cisco Intersight RESTful API metal cloud Range of SDKs for the Intersight RESTful API, including ones for Python and Microsoft PowerShell Range of Ansible modules VMWare Datacenter CLI vSphere CLI for Linux and Windows PowerCLI for Windows PowerShell Host of SDKs for popular languages, aimed at vSphere Automation, vCloud Suite, and other products OpenStack OpenStack Client (OSC) OpenStack Compute, Identity, Image, Object Storage, and Block Storage APIs OpenStack Python SDK OpenStack Toolkits © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 26  Academic Basic Automation Scripting use Summary of Basic Automation Scripting Basic automation scripting techniques are great to have in the toolbox and understanding them will improve the facility as an operator and user of mature automation platforms. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 27 7.4 Automation Tools © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 28  Academic Automation Tools use Introduction to Automation Tools In this topic, the three most popular automation tools, Ansible, Puppet, and Chef, are being discussed. There will also be an option to install one or all of them on the local workstation. To try this, one must have access to a Linux-based workstation, such as Ubuntu or macOS and must refer to the tool's own installation documentation for the operating system. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 29  Academic Automation Tools use What Do Automation Tools Do For Us? What do automation tools do for us? Automation tools offer powerful capabilities compared to ad-hoc automation strategies using BASH, Python, or other programming languages. These tools enable developers to: Simplify and standardize Accelerate development with out-of-the-box features Facilitate reusability, segregate concerns, promote security Perform discovery and manage inventory Handle scale Engage community © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 30  Academic Automation Tools use Critical Concepts Idempotency: a review An Idempotent software produces the same desirable result each time that it is run. In a deployment software, Idempotency enables convergence and composability and allows to: More easily gather components in collections that build new kinds of infrastructure and perform new operations tasks. Execute whole build/deploy collections to safely repair small problems with infrastructure, perform incremental upgrades, modify configuration, or manage scaling. Procedure vs. Declarative Procedural code can achieve idempotency, but many infrastructure management, deployment, and orchestration tools have adopted another method, which is creating a declarative. A declarative is a static model and is used by middleware that incorporates deployment-specific details, examines present circumstances, and brings real infrastructure into alignment with the model, and usually least time-consuming path. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 31  Academic Automation Tools use Critical Concepts (Contd.) Provisioning vs. configuration vs. deployment vs. orchestration Provisioning Configuration Deployment Orchestration This refers to obtaining This means installing This involves building, This may refer to several things: compute, storage, and base applications and arranging, integrating, User-built or platform-inherent network infrastructure services, and performing and preparing multi- automation aimed at (real or virtual), enabling the operations, tasks, component applications managing workload lifecycles communications, putting and tests required to or higher-level platforms, and reacting dynamically to it into service, and prepare a low-level often across multiple changing conditions. making it ready for use platform to deploy nodes. Processes or workflows that by operators and applications or a higher- link automation tasks to developers. level platform. deliver business benefits, like self-service. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 32  Academic Automation Tools use Critical Concepts (Contd.) Statelessness Automation works best when applications can be made stateless. This means that redeploying them in place does not destroy or lose track of the data that users or operators need. The two states of an application are: Not Stateless – An app that saves important information in files or in a database on the local file. Stateless – An app that persists its state to a separate database or that provides service that requires no memory of state between invocations. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 33  Academic Automation Tools use Popular Automation Tools The first modern automation tool was Puppet which was introduced in 2005 as open source, and then commercialized as Puppet Enterprise by Puppet Labs in 2011. The most popular automation tools are Ansible, Puppet, Chef. They share the following characteristics: Relatively easy to learn Available in open source versions Plugins and adapters enable them to directly or indirectly control many types of resources Many other solutions also exist. Private and public cloud providers often endorse their own tools for use on their platforms such as OpenStack's HEAT project, AWS' CloudFormation, SaltStack and Terraform. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 34  Academic Automation Tools use Ansible Ansible is available as open source, and in a version with added features, from IBM/Red Hat, called Ansible Tower. Ansible is substantially managed from the Bash command line, with automation code developed and maintained using any standard text editor. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 35  Academic Automation Tools use Ansible (Contd.) Ansible Architecture: Simple and Lightweight Control node runs on any Linux machine running Python 2 or 3. All system updates are performed on control node. Control node connects to managed resources over SSH and enable Ansible to: Run shell commands on a remote server, or transact with a remote router, or other network entity, via its REST interface. Inject Python scripts into targets and remove them after they run. Install Python on target machines if required. Plugins enable Ansible to gather facts from and perform operations on infrastructure that can't run Python locally. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 36  Academic Automation Tools use Ansible (Contd.) Installing Ansible The Ansible control node application is installed on a Linux machine from its public package repository. To install Ansible on a workstation, refer to the installation documentation appropriate to the device. Ansible code structure In the Ansible code structure, work is separated into YAML (.yml) files that contain a sequence of tasks, executed in top-down order. Ansible has hundreds of pre-built Python modules that wrap operating- system-level functions and meta-functions. Playbooks and roles An Ansible playbook can be written as a monolithic document with a series of modular, named tasks. Developers build a model of a complex DevOps task out of low-level playbook task sequences called roles and then reference these in higher-level playbooks, sometimes adding additional tasks at the playbook level. This segregation of concerns ensures clarity, reusability and shareability of roles. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 37  Academic Automation Tools use Ansible (Contd.) Ansible project organization Ansible projects are organized in a nested directory structure. The hierarchy is easily placed under version control and used for GitOps-style infrastructure as code. Ansible folder hierarchy elements include Inventory files, Variable files, Library and utility files, Main playbook files. Ansible at scale There are scaling challenges for large organizations, such as managing and controlling access to many Ansible nodes flexibly and securely. This also includes putting remote controllers seamlessly and safely under control of centralized enterprise automation. For this, there are two control-plane solutions: Red Hat Ansible Tower and AWX project. Larger-scale Ansible implementations also benefit from Ansible Vault, a built-in feature that enables encryption of passwords and other sensitive information. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 38  Academic Automation Tools use Ansible (Contd.) Cisco Ansible resources Cisco and the Ansible community maintain extensive libraries of Ansible modules for automating Cisco compute and network hardware including: A very large set of built-in modules for configuring Cisco Application-Centric Infrastructure fabrics via the Application Policy Infrastructure Controller (APIC). Remote control of Cisco network devices running IOS-XR, plus modules for sending commands and retrieving results from these devices via CLI, or via the standard NETCONF REST interface. Ansible modules for configuring Cisco UCS infrastructure via the Intersight REST interface. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 39  Academic Automation Tools use Ansible Example Ansible normally uses ssh to connect with remote hosts and execute commands. Let's see how to create a simple website on a remote host. Prerequisites A target host running a compatible operating system (such as Ubuntu 18.04 server) SSH and keywise authentication configured on that host Ansible installed on your local workstation © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 40  Academic Automation Tools use Ansible Example (Contd.) Building an Ansible project file tree For the purposes of this exercise, the target machine's (DNS- resolvable) hostname is target. With your target machine SSH-accessible, begin building a base folder structure for the Ansible project. At the top level in your project folder, you need: An inventory file, containing information about the machine(s) on which you want to deploy. A top level site.yml file, containing the most abstract level of instructions. A role folder structure to contain your webserver role. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 41  Academic Automation Tools use Ansible Example (Contd.) Creating your inventory file Your inventory file for this project can be very simple. Make it the DNS-resolvable hostname of your target machine: You are defining a group called webservers and putting your target machine's hostname (or IP) in it. You could add new hostnames/IPs to this group block, or add additional group blocks, to assign hosts for more complex deployments. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 42  Academic Automation Tools use Ansible Example (Contd.) Creating your top level playbook file A top-level playbook typically describes the order, permissions, and other details under which lower-level configuration acts, defined in roles, are applied. In this example, site.yml file identifies which hosts you want to perform an operation on, and which roles you want to apply to these hosts. The line become: true tells Ansible that you want to perform the roles as root, via sudo. Creating your webservers role Next step is to create the role that installs and configures your web server. You've already created the folder structure for the role using ansible-galaxy. Code for the role is contained in a file called main.yml in the role's /tasks directory. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 43  Academic Automation Tools use Ansible Example (Contd.) You can edit roles/webserver/tasks/main.yml file directly, as shown here. The role has two tasks: Deploy Apache2. Copy a new index.html file into the Apache2 HTML root, replacing the default index.html page. In the apt: stanza, you name the package, its required state, and instruct the apt module to update its cache. In the second stanza, Ansible's copy routine moves a file from your local system to a directory on the target and also changes its owner and permissions. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 44  Academic Automation Tools use Ansible Example (Contd.) Creating your index.html file Of course, you will need to create a new index.html file as well. The Ansible copy command assumes that such files will be stored in the /files directory of the role calling them, unless otherwise specified. Navigate to that directory and create the index.html file, saving your changes afterward. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 45  Academic Automation Tools use Ansible Example (Contd.) Running your deployment Now you're ready to run your deployment. From the top level directory of your project, you can do this with the statement: -i names your inventory file. -u argument names your sudo user. -K tells Ansible to ask us for your sudo password, as it begins execution. site.yml is the file that governs your deployment. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 46  Academic Automation Tools use Ansible Example (Contd.) If all is well, Ansible should ask us for your BECOME password (sudo password), then return results similar to the following: And now, if you visit the IP address of your target machine in a browser, you should see your new homepage. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 47  Academic Automation Tools use Ansible Example (Contd.) Ansible CI/CD walkthrough Let's walk through the example as if they were part of a CI/CD pipeline. Developer collaborating with you on GitHub commits a change to the website such as in index.html file. Next, tests in the repository execute syntax and sanity checks as well as code review rules against each pull request. Next, the CI/CD system prepares an environment and runs predefined tests for any Ansible playbook. It should indicate the version expected each time and install it. Here's an example pipeline: After Jenkins is done running the job, you can get a notification that all is ready for staging and you can push these changes to production with another pipeline, this time for pushing to production. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 48  Academic Automation Tools use Lab – Use Ansible to BackUp and Configure a Device In this lab, you will complete the following objectives: Part 1: Launch the DEVASC VM and CSR1000v VM Part 2: Configure Ansible Part 3: Use Ansible to Back Up a Configuration Part 4: Use Ansible to Configure a Device © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 49  Academic Automation Tools use Lab – Use Ansible to Automate Installing a Web Server In this lab, you will complete the following objectives: Part 1: Launch the DEVASC VM Part 2: Perform a Backup with Ansible Part 3: Configure IPv6 Addressing with Ansible Part 4: Use Ansible to install Apache on Web Servers Part 5: Add Options to Your Ansible Playbook for Apache Web Servers © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 50  Academic Automation Tools use Puppet Founded as an open source in 2005 and commercialized as Puppet Enterprise by Puppet Labs in 2011. Architecture components A designated server to host main application components. A secure client, also known as a Puppet Agent. Modules to enable connections for cloud APIs and hardware that cannot run an agent. In scaled-out implementations, a proxy agent to offload the work of directly connecting to device CLIs and exchanging information. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 51  Academic Automation Tools use Puppet (Contd.) Installing the Puppet Puppet Server requires powerful hardware (or a big VM), and a Network Time Protocol client to be installed, configured, and tested. The agents will need the puppet.conf file configured to communicate with the Puppet Server. After client service started, it will have certificate signed by the server. The Server will now be able to gather facts from the client and update the client state with any configuration changes. Puppet Code Structure Puppet stores components of a project or discrete configuration in a directory tree (/etc/puppetlabs/code/environments). Subsidiary folders are created according to the configuration in puppet.conf or by the operator. Puppet comes with a set of basic resources built in. Many additional resources for performing all sorts of operations can be downloaded and installed from Puppet Forge using the puppet module command. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 52  Academic Automation Tools use Puppet (Contd.) Puppet at Scale Puppet Server is somewhat monolithic, and a monolithic installation is recommended by the (open source) implementors. The step to accommodate more hosts is to create additional "compile masters", which compile catalogs for client agents and place these behind a load balancer to distribute work. Puppet Enterprise customers can further expand capacity by replacing PuppetDB with a stand-alone, customized database called PE-PostgreSQL. Cisco Puppet resources Cisco and the Puppet community maintain extensive libraries of modules for automating Cisco compute and network hardware. These include: Cisco IOS modules enabling management of IOS infrastructure Cisco UCS modules enabling control of UCS via UCS Manager © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 53  Academic Automation Tools use Puppet Example This exercise describes how to install Puppet and then use Puppet to install Apache 2 on a device. This approximates the normal workflow for Puppet operations in an automated client/server environment. Note that modules can be completely generic and free of site-specific information, then separately and re-usably invoked to configure any number of hosts or infrastructure components. Installing Puppet Server Puppet Server requires powerful hardware (or a big VM), and a Network Time Protocol client to be installed, configured, and tested. Instructions for installing the server can be found in Puppet's documentation. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 54  Academic Automation Tools use Puppet Example (Contd.) Installing Puppet Client When you have the Puppet Server running, you can install Puppet Agents on a host. For example, on a Debian-type Linux system, you can install Puppet Agent using a single command: Modify When installed, the Puppet Agent needs to be configured to seek a Puppet Server. Add the lines to the file /etc/puppet/puppet.conf which tells the client, the hostname of your server and name of the authentication certificate that you will generate in the next step. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 55  Academic Automation Tools use Puppet Example (Contd.) Start the puppet service on the Client: You should get a response similar to the following: © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 56  Academic Automation Tools use Puppet Example (Contd.) Certificate signing Puppet Agents use certificates to authenticate with the server before retrieving their configurations. When the Client service starts for the first time, it sends a request to its assigned server to have its certificate signed, enabling communication. On the Server, issue the ca list command returns a list of pending certificates. The response should be similar to the following: © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 57  Academic Automation Tools use Puppet Example (Contd.) You can then sign the certificate, enabling management of the remote node: The response: The Server and Client are now securely bound and able to communicate. This will enable the Server to gather facts from the Client, and let you create configurations on the Server that are obtained by the client and used to converge its state (every 15 minutes). © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 58  Academic Automation Tools use Puppet Example (Contd.) Creating a configuration Puppet lets you store components of a project or discrete configuration in a directory tree. Subsidiary folders are created according to the configuration in puppet.conf or by the operator. In this example, having declared environment = production, Puppet has already created a directory for this default site, containing a modules subdirectory in which we can store subsidiary projects and manifests for things we need to build and configure. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 59  Academic Automation Tools use Puppet Example (Contd.) You will now install Apache2 on your managed client. Puppet operations are typically performed as root, so become root on the Server temporarily by entering sudo su -. Navigate to the /modules folder in the /production environment. Create a folder structure to contain the install apache module. Inside the manifests folder, create a file called init.pp, which is a reserved filename for the initialization step in a module. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 60  Academic Automation Tools use Puppet Example (Contd.) The class definition orders the steps we want to perform: Step 1. Invoke the package resource to install the named package by ensure =>installed. Step 2. Invoke the service resource to run if its requirement is met. Instruct it to ensure that the service is available, and then enable it to restart automatically when the server reboots. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 61  Academic Automation Tools use Puppet Example (Contd.) Navigate to the associated manifests folder. Create a site.pp file that invokes the module and applies it to the target machine. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 62  Academic Automation Tools use Puppet Example (Contd.) Deploying the configuration You have two options to deploy the completed configuration: Restarting the Puppet Server will now cause the manifests to be compiled and made available to the Puppet Agent on the named device. The agent will retrieve and apply them, installing Apache2 with the next update cycle: For development and debugging, you can invoke Puppet Agent on a target machine. The agent will immediately interrogate the server, download its catalog and apply it. The results will be similar to the following: © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 63  Academic Automation Tools use Puppet Example (Contd.) The agent will immediately interrogate the server, download its catalog and apply it. The results will be similar to the following: After the application has been successfully deployed, enter the target machine's IP address in your browser. This should bring up the Apache 2 default homepage. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 64  Academic Automation Tools use Chef Chef provides a complete system for treating infrastructure as code. Chef products are partly licensed, but free for personal use (in Chef Infra Server's case, for fewer than 25 managed nodes). Chef's products and solutions enable infra-as-code creation, testing, organization, repository storage, and execution on remote targets, either from a stand-alone Chef Workstation, or indirectly from a central Chef Infra Server. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 65  Academic Automation Tools use Chef (Contd.) Chef Architecture – Components Chef Workstation Chef Infra Client (the host agent) Chef Infra Server Most configuration tasks can also be carried out directly between Chef Workstation and managed nodes and devices. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 66  Academic Automation Tools use Chef (Contd.) Components of Chef Workstation include command-line tools, interaction with Chef Infra Servers, Test Kitchen, ChefSpec and InSpec. Installing Chef Workstation To begin using Chef, the first step is to install Chef Workstation, which provides a complete operations environment. Refer to the Chef Workstation downloads page for more information. The Workstation is available for Linux and Windows. Running Chef at scale Chef Infra Server can be configured for high availability by deploying its front-end services into an array of load-balanced proxies. Chef also provides an array of products that together solve most of the problems enterprises face in dealing with increasingly-complex, large-scale, hybrid infrastructures. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 67  Academic Automation Tools use Chef (Contd.) Cisco Chef Resources Cisco has developed modified Chef Infra Agents that run in the guest shell of NX-OS switch equipment, enabling this hardware to work with Chef as if it were a managed host. It has also developed and maintains a Cisco Chef Cookbook for NX-OS infrastructure, available on Chef Supermarket. A GitHub public repo of cookbook and recipe code is also maintained to enable control of a wide range of Cisco products. Cisco UCS infrastructure is easily managed with Chef through a cookbook enabling integration with Integrated Management Controllers. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 68  Academic Automation Tools use Chef Example – Install and Use Chef This example describes how to install Chef and use it to install Apache 2 on a device. Installing Chef Workstation Chef Workstation provides a complete operations environment. The following example assume that you are installing on an Ubuntu 18.04 LTS virtual machine. If your machine is set up with a standard desktop, browse to the Chef Workstation downloads page, find the download for Ubuntu 18.04, and install it automatically with the Debian package manager. Alternatively, you can install from the command line by copying the URL of the.deb package. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 69  Academic Automation Tools use Chef Example – Install and Use Chef (Contd.) Basic Configuration Management After Workstation is installed, start making configuration changes on accessible hosts. You will use the chef-run command for this. The first time you use chef-run, you may be asked to accept licensing terms for the utility. For the first configuration exercise, you will provide the information Chef needs to install the ntp package. In the process, you will provide the remote username, their sudo password, the name of the remote host target and the name of the resource verb. When the client is installed, the task is handed to it, and the process completes. You get back as follows: © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 70  Academic Automation Tools use Chef Example – Install and Use Chef (Contd.) Install Chef Infra Client Chef Infra Client runs locally on conventional compute nodes. Chef Workstation can bootstrap Infra Client onto target nodes. You can also preinstall Infra Client on nodes, for example, while creating new nodes on a public cloud. Below is an example script you might run on a target host to do this. The script uses a Chef-provided installer called Omnitruck to do this. A Windows version of this script is also available that runs on PowerShell: Note that the parameters shown above will install the latest version of the Chef client, and do not pin the version. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 71  Academic Automation Tools use Chef Example – Install and Use Chef (Contd.) Chef Infra Server prerequisites Before installing Chef Infra Server, install openssh-server and enable keywise access. You would also need to install NTP for time synchronization. You can do this with Chef, or manually: On an Ubuntu system, turn off the default timedatectl synchronization service to prevent it from interfering with NTP synchronization: After NTP is installed, ensure that it is synchronizing with a timeserver in its default pool. This may take a few minutes, so repeat the command until you see the following: When this shows up, you can install Chef Infra Server. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 72  Academic Automation Tools use Chef Example – Install and Use Chef (Contd.) Install Chef Infra Server Chef Infra Server stores configuration and provides it to Clients automatically, when polled, enabling Clients to converge themselves to a desired state. To install Chef Infra Server on Ubuntu 18.04, you can perform steps similar to the manual Workstation install after obtaining the URL of the.deb package. At time of writing, the current stable version was 13.1.13-1. After Chef Infra Server is installed, issue the following command to tell it to read its default configuration, initialize, and start all services. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 73  Academic Automation Tools use Chef Example – Install and Use Chef (Contd.) Install Chef-Manage You can also install the web interface for Chef server. This can be done by entering: When the process completes, restart the server and manage components. These are Chef operations, and may take a while, as before. The argument --accept-license prevents chef-manage-ctl from stopping to ask you about the unique licenses for this product. When this process is complete, you can visit the console in a browser via https://. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 74  Academic Automation Tools use Chef Example – Install and Use Chef (Contd.) Finish configuring Workstation Before Chef Workstation can talk to your Infra Server, you need to do a little configuration. To begin, retrieve the keys generated during Server configuration, and store them in the folder /home/myname/.chef created during Workstation installation: /path/ is the path from your home directory on the Server to the directory in which the Server stored keys. If you are not using keywise authentication to your Server, scp will ask for your password. The. after user@host: refers to your original user's home directory, from which the path is figured. The wildcard expression finds files ending in.pem at that path. The closing dot means copy to the current working directory. Run the ls command from within the.chef folder to see if your keys made it. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 75  Academic Automation Tools use Chef Example – Prepare to Use Knife Prepare to use Knife Knife is a tool for managing cookbooks, recipes, nodes, and other assets, and for interacting with the Chef Infra Server. Within the.chef folder, folder, edit the (initially empty) file named config.rb and include the following lines of code, adapting them to your environment: © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 76  Academic Automation Tools use Chef Example – Prepare to Use Knife (Contd.) Save the config.rb file and then create the directory /home/myname/cookbooks. Finally, issue the command knife ssl fetch. If you have correctly set up the config.rb, Knife will consult with your server, retrieve its certificate, and store it in the directory. Chef will find this automatically when it is time to connect with the server, providing assurance that the server is authentic. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 77  Academic Automation Tools use Chef Example – Prepare to Use Knife (Contd.) Bootstrap a target node with knife After Knife is configured, you can bootstrap your target node. To bootstrap, issue the following command, replacing variable fields with your information. The command is set up to use keywise authentication to the target machine. The redundant --sudo and --use-sudo-password commands tell Knife to use sudo to complete its work. The -P option provides your sudo password on the target machine. is an arbitrary name. The --ssh-verify-host-key never flag and argument cause the command not to pause and ask your permission interactively if it finds that you've never logged into this server before. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 78  Academic Automation Tools use Chef Example – Prepare to Use Knife (Contd.) If the command works correctly, you would get back the given output. Note that Chef has detected the earlier installation and has not overwritten it. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 79  Academic Automation Tools use Chef Example – Prepare to Use Knife (Contd.) Chef Manage Displays Your Target Node: Now, if you check back in your browser and refresh Chef Manage, you should see that your target machine is now being managed by the server. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 80  Academic Automation Tools use Chef Example – Putting it all Together Now you will use everything together to create an actual recipe, push it to the server, and tell the target machine's client to requisition and converge on the new configuration. To start, create a cookbook to build a simple website. Navigate to your cookbooks directory, create a new cookbook called apache2 and navigate in it. Check the cookbook folder structure. There are folders already prepared for recipes and attributes. Add an optional directory and subdirectory for holding files your recipe needs. Files in the /default subdirectory of a /files directory within a cookbook can be found by recipe name alone, no paths are required. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 81  Academic Automation Tools use Chef Example – Putting it all Together (Contd.) Now create a homepage for your website. Save the file and exit. Navigate to the recipes directory , where Chef has already created a default.rb file for us. The default.rb file will be executed by default when the recipe is run with this command. Add some data to the default.rb file and again edit the file. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 82  Academic Automation Tools use Chef Example – Putting it all Together (Contd.) The header at the top is created for you. Underneath, the recipe performs three actions. The first resource you are invoking apt_update handles the apt package manager on Debian. The package function is used to install the apache2 package from public repositories. Finally, you use the cookbook_file resource to copy the index.html file from /files/default into a directory on the target server. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 83  Academic Automation Tools use Chef Example – Putting it all Together (Contd.) Save the default.rb file and then upload the cookbook to the server. You can then confirm that the server is managing your target node. The Knife application can interoperate with your favorite editor. To enable this, perform the following export with your editor's name: This lets the next command execute interactively, putting the node definition into vi to let you alter it manually. As you can see, the expression "recipe[apache2]" has been added to the run_list array which contains an ordered list of the recipes you want to apply to this node. Save the file in the usual manner. Knife immediately pushes the change to the Infra Server. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 84  Academic Automation Tools use Chef Example – Putting it all Together (Contd.) Finally, you can use the knife ssh command to identify the node, log into it non-interactively using SSH, and execute the chef-client application. If all goes well, Knife gives you back a very long log that shows you exactly the content of the file that was overwritten, and confirms each step of the recipe as it executes. At this point, you should be able to point a browser at the target machine's IP address, and see your new index page. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 85  Academic Automation Tools use Summary of Automation Tools Summary This has been a high-level introduction to three modern DevOps toolkits. You should now be ready to: Deploy and integrate free versions of the major components of Ansible, Puppet, and/or Chef on a range of substrates, from desktop virtual machines to cloud-based VMs on Azure, AWS or other IaaS platforms. Experience each platform's declarative language, style of infra-as-code building and organizing, and get a sense of the scope of its library of resources, plugins, and integrations. Get practice automating some of the common IT tasks you may do at work, or solve deployment and lifecycle management challenges you set yourself, in your home lab. Hands-on exercises and work will give you a complete sense of how each platform addresses configuration themes, and help you overcome everyday IT challenges. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 86 7.5 Infrastructure as Code © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 87  Academic Infrastructure as Code use Why Store Infrastructure as Code? The term immutability refers to maintaining systems entirely as code, performing no manual operations on them at all. GitOps: modern infrastructure-as-code GitOps is also referred to as "operations by pull request." In a typical GitOps setup, you might maintain a repository, such as a private repo on GitHub, with several branches called Development, Testing/UAT and Production. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 88  Academic Infrastructure as Code use Why Store Infrastructure as Code? (Contd.) Where can GitOps take you? When all the GitOps procedures, workflow and other components are in place, developers can look at implementing elite deployment strategies. Blue/Green deployment Blue/green deployment is a method for reducing or eliminating downtime in production environments. It is required to maintain two identical production environments (Not necessarily, blue and green. Any two colors such as Red and Black will do). It is also required to develop the capability of quickly redirecting application traffic to one or the other. Note: Some DevOps practitioners differentiate between blue/green and red/black strategies. They say that in blue/green, the traffic is gradually migrated from one environment to the other, so it hits both systems for some period; whereas in red/black, traffic is cut over all at once. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 89  Academic Infrastructure as Code use Why Store Infrastructure as Code? (Contd.) A release is deployed to the environment not currently in use (Green). After acceptance testing, redirect traffic to this environment. If problems are encountered, switch traffic back to the original environment (Blue). If the Green deployment is judged adequate, resources owned by the Blue deployment can be relinquished, and roles swapped for the next release. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 90  Academic Infrastructure as Code use Why Store Infrastructure as Code? (Contd.) Canary Testing Canary testing is similar to rolling blue/green deployment, but somewhat more delicate. The migration between old and new deployments is performed on a customer-by-customer (or even user-by-user) basis. Migration are made to reduce the risk and improve the quality of feedback. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 91 7.6 Automating Testing © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 92  Academic Automating Testing use Automated Test and Validation By using unit testing tools like pytest developers can build an environment where the code can be tested automatically when changes are made. Unit-testing frameworks make tests a part of the codebase, following the code through developer commits, pull requests, and code-review gates to QA/test and Production. This unit test framework is useful in test- driven development (TDD) environments. The challenges of testing a network The behavior and performance of a real-world network is collective, maintained by the configurations of many discrete pieces of equipment and software. In traditional environments, connectivity and functionality are manually maintained across numerous individual pieces of equipment via diverse interfaces. This is difficult, time-consuming, extremely error- prone, and risky. As networks become more complex and carry more diverse and performance-sensitive traffic, risks to security and performance degradations are higher. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 93  Academic Automating Testing use Automated Test and Validation (Contd.) Testing Software Defined Networks (SDN) Cisco has made huge progress in developing Software Defined Networks (SDN) and middleware that let engineers to address a physical network as a single programmable entity. In Cisco's case, this includes: Application Centric Infrastructure (ACI) Digital Network Architecture Center (Cisco DNA Center) REST API and SDKs enable integration with automation tools like Ansible, Puppet, and Chef Python Automated Test System (pyATS) Python Automated Test System (pyATS) is a Python-based network device test and validation solution. pyATS originated as the low-level Python underpinning for the test system as a whole. Its higher-level library system, Genie, provides the necessary APIs and libraries that drive and interact with network devices, and perform the actual testing. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 94  Academic Automating Testing use Automated Test and Validation (Contd.) pyATS has several key features: pyATS framework and libraries can be leveraged within any Python code. It is modular and includes components such as AEtest and Easypy. A CLI enables rapid interrogation of live networks, extraction of facts, and helps automate running of test scripts and other forensics. pyATS provides an enormous interface library to Cisco and other infrastructure via a range of interfaces. pyATS can consume, parse, and implement topologies described in JSON, as YANG models, and from other sources. pyATS can also be integrated with automation tools for building, provisioning, and teardown. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 95  Academic Automating Testing use pyATS Example The following content shows how to use pyATS to create and apply tests. Virtual environments The pyATS tool is best installed for personal work inside a Python virtual environment (venv). A venv is an environment copied from your base environment, but kept separate from it. This enables you to avoid installing software that might permanently change the state of your system. Virtual environments exist in folders in your file system. When they are created, they can be activated, configured at will, and components installed in them can be updated or modified without changes being reflected in your host's configuration. The ability to create virtual environments is native to Python 3, but Ubuntu 18.04 may require you to install a python3-venv package separately. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 96  Academic Automating Testing use pyATS Example (Contd.) To create a venv on Ubuntu 18.04: Ensure that python3-pip, the Python3 package manager, is in place and install git. Create a new virtual environment in the directory of your choice. Venv creates the specified working directory and folder structure containing functions and artifacts describing this environment's configuration. At this point, you can cd to the myproject and activate the venv. Installing pyATS Install the pyATS from the public Pip package repository (PyPI). Verify that it was installed by listing the help, using pyats --help. Clone the pyATS sample scripts repo, maintained by Cisco DevNet, which contains sample files. Note: You may see "Failed building wheel for…" errors while installing pyATS. You can safely ignore those errors as pip has a backup plan for those failures and the dependencies are installed despite errors reported. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 97  Academic Automating Testing use pyATS Example (Contd.) pyATS test case syntax The test declaration syntax for pyATS is inspired by Python unit-testing frameworks like pytest. It supports basic testing statements, such as an assertion that a variable has a given value, and adds to that the ability to explicitly provide results. pyATS scripts and jobs A pyATS script is a Python file where pyATS tests are declared. pyATS testbed file A testbed can be a single YAML file or can be programmatically assembled from YAML and Python. The testbed file is an essential input to the rest of pyATS library and ecosystem as it provides information to the framework for loading the right set of library APIs for each device, and how to effectively communicate to them. Real testbed files for large topologies can be long, deeply-nested, and complex. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 98  Academic Automating Testing use pyATS Example (Contd.) pyATS Library: Genie Genie is the pyATS higher-level library system that provides APIs for interacting with devices, and a powerful CLI for topology and device management and interrogation. When installed, it adds its features and functionalities into the pyATS framework. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 99  Academic Automating Testing use Lab – Automated Testing Using pyATS and Genie In this lab, you will complete the following objectives: Part 1: Launch the DEVASC VM and CSR1000v VM Part 2: Create a Python Virtual Environment (venv) Part 3: Using the pyATS testing library Part 4: Introducing Genie and creating a testbed file Part 5: Using Genie to compare configurations © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 100 7.7 Network Simulation © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 101  Academic Network simulation use Network Simulation and VIRL Network simulation provides a means to test network configurations, debug configuration code, and to work with and learn Cisco infrastructure and APIs in a safe, convenient, and non-cost-prohibitive way. Cisco Virtual Internet Routing Laboratory (VIRL) is a commercial product originally developed for internal use at Cisco, with broad and active community support. Now in version 2, VIRL can run on bare metal, or on large virtual machines on several hypervisor platforms. The virtual equipment that runs inside VIRL uses the same code that runs inside actual Cisco products. VIRL components and workflow VIRL provides a local CLI for system management, a REST interface for integration with automation, and a powerful UI that offers a complete graphical environment for building and configuring simulation topologies. The UI comes with several topologies to get started. Among these is a two-router IOS network simulation that can quickly be made active and explored. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 102  Academic Network simulation use Network Simulation and VIRL (Contd.) VIRL's Design Perspective view allows to modify existing simulations or compose new simulations by dragging, dropping, and connecting network entities, configuring them. The visualization has clickable elements that explore configuration of entities and make changes via the WebUI, or by connecting to network elements via console. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 103  Academic Network simulation use Network Simulation and VIRL (Contd.) VIRL Files The individual device configurations, or entire simulated network configs can be extracted as.virl files. VIRL enables you to define simulations as code, enabling both-ways integration with other software platforms for network management and testing. VIRL's native configuration format is called a.virl file which is a human-readable YAML file. The.virl file contains complete descriptions of the IOS routers, their interface configurations and connection, credentials for accessing them, and other details. These files can be used to launch simulations via the VIRL REST API and the.virl files can be converted to and from testbed files to use with PyATS and Genie. The.virl file provides a method for determining if configuration drift has occurred on the simulation. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 104 7.8 Infrastructure and Automation Summary © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 105  Academic Infrastructure and Automation Summary use What Did I Learn in this Module? Automation is using code to configure, deploy, and manage applications together with the compute, storage, and network infrastructures and services on which they run. Cloud computing, lets developers and operators use software to requisition, configure, deploy, and manage bare-metal and virtualized compute, storage, and network resources. For full-stack automation to be truly effective, it requires changes to organizational culture, including breaking down the historical divides between Development (Dev) and Operations (Ops). DevOps/SRE have many core principles and best practices: A focus on automation, the idea that failure is normal and a reframing of availability in terms of what a business can tolerate. Cloud automation enables you to provision virtualized hosts, configure virtual networks and other connectivity, requisition services, and then deploy applications on this infrastructure. Three of the most popular automation tools are Ansible, Puppet, and Chef. Immutability refers to maintaining systems entirely as code, performing no manual operations on them at all. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 106  Academic Infrastructure and Automation Summary use What Did I Learn in this Module? (Contd.) The unit test framework is useful in test-driven development (TDD) environments. Network simulation provides a means to test network configurations, debug configuration code, and to work with and learn Cisco infrastructure and APIs in a safe, convenient, and non-cost-prohibitive way. Cisco Virtual Internet Routing Laboratory (VIRL) can run on bare metal or on large virtual machines on several hypervisor platforms. © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 107