Defining Infrastructure Declaratively with Crossplane PDF

# Defining Infrastructure Declaratively with Crossplane From *Continuous Delivery with Kubernetes* by Mauricio Salatino This article discusses using Crossplane to provision real infrastructure in a declarative way. Take 35% off *Continuous Delivery for Kubernetes* by entering **fccsalatino** into...

# Defining Infrastructure Declaratively with Crossplane From *Continuous Delivery with Kubernetes* by Mauricio Salatino This article discusses using Crossplane to provision real infrastructure in a declarative way. Take 35% off *Continuous Delivery for Kubernetes* by entering **fccsalatino** into the discount code box at checkout at [manning.com](https://manning.com/). ## Defining Infrastructure in a declarative way using Crossplane Using Helm to install application infrastructure components inside Kubernetes is far from ideal for large applications and user-facing environments, as the complexity of maintaining these components and their requirements, such as advanced storage configurations, might become too complex to handle for your teams. Cloud Providers do a fantastic job at allowing us to provision infrastructure, but they rely on cloud provider-specific tools which are outside of the realms of Kubernetes. In this article, we look at one such tool; a CNCF project called Crossplane (https://crossplane.io), which uses the Kubernetes APIs and extension points to enable users to provision real infrastructure in a declarative way, using the Kubernetes APIs. Crossplane relies on the Kubernetes APIs to support multiple Cloud Providers; this also means that it integrates nicely with all the existing Kubernetes tooling. ## Crossplane Providers Crossplane extends Kubernetes by installing a set of components called "Providers” which are in charge of understanding and interacting with cloud provider-specific services to provision these components for us. By installing Crossplane providers we extend the Kubernetes APIs functionality to provision external resources such as Databases, Message Brokers, Buckets and other Cloud Resources. Several Crossplane providers cover the major cloud providers such as GCP, AWS and Azure. You can find these Crossplane providers in the Crossplane Github's organization: [https://github.com/crossplane/](https://github.com/crossplane/). Once a Crossplane Provider is installed you can create provider-specific resources in a declarative way, which means that you can create a Kubernetes Resource, apply it with *kubectl apply -f*, or include the yaml file inside a Helm Chart or use a GitOps approach, where you store the yaml file in a repository and use an environment pipeline to provision these resources in an environment. Provisioning cloud-specific resources relying on the Kubernetes APIs is a big step forward but Crossplane doesn't stop there. If you go to look at the details of what it takes to provision a database in any major cloud provider you realize that provisioning the component is only one of the tasks involved in getting the component ready to be used. For connecting to these provisioned components, you need network and security configurations, as well as user credentials. ## Crossplane Compositions Crossplane aims to serve two different Personas: Infra Ops teams and App Ops teams. Although Infra Ops teams are Cloud Providers experts that understand how to provision cloud provider-specific components, App Ops teams know the application requirements and understand what is required from the Application Infrastructure perspective. The interesting thing about this approach is that when using Crossplane Infra Ops teams can define these complex configurations in the Cloud Provider and expose simplified interfaces for App Ops teams ( closer to developers) who are interested in simple ways to provision Application Infrastructure components. For achieving these abstractions and simplified interfaces Crossplane introduced the concept of "CompositeManagedResources”. ## Crossplane Components and Requirements To work with Crossplane Providers and `CompositeManagedResources` we need to understand how Crossplane components work together to provision and manage these components inside different Cloud Providers. This section covers what Crossplane needs in order to work and how Crossplane components manage our `CompositeManagedResources`. First of all, it's important to understand that you need to install Crossplane in a Kubernetes cluster. This can be the Cluster where your applications are running or a separate cluster where Crossplane runs. This cluster has some Crossplane components that understand our `CompositeManagedResources` and have enough permissions on the cloud platform to provision resources on our behalf. You might be wondering if you can install Crossplane into a KIND Cluster, and the answer is yes, but Crossplane shines if you have access to a Cloud Provider. For that reason, we use Google Cloud Platform in the next sections. Each of the Crossplane Providers available require a specific security configuration to work and an account inside the Cloud Provider where we can create resources. Once a Crossplane Provider is installed and configured, in this case the GCP provider we can start creating resources which are managed by this provider. You can find the resources offered by each provider in the following documentation site: https://doc.crds.dev/github.com/crossplane/provider-gcp ## Crossplane Behaviours In contrast with installing components with Helm in our Kubernetes Clusters, we use Crossplane to interact with the cloud provider specific APIs to provision resources inside the Cloud infrastructure. This should simplify the maintenance tasks and costs related with these resources. Another important thing that the Crossplane provider (GCP provider in this case) monitors the created `Managed Resources` for us. These `Managed Resources` offer some advantages compared with the installed resources using Helm. `Managed Resources` have well-defined behaviors; here's a summary of what to expect from a Crossplane `Managed Resource`: * **Visible as any other Kubernetes Resource**: Crossplane `Managed Resources` are only Kubernetes resources, which means that we can use any Kubernetes tool to monitor and query the state of these resources. * **Continuous Reconciliation**: when a managed resource is created the Provider continuously monitors the resource to make sure that it exists and that it's working, and report back the status to the Kubernetes resource. The parameters defined inside the managed resource are considered the desired state (source of truth) and providers work to apply these configurations into the Cloud Provider resources. Once again, we can use standard Kubernetes tools to monitor change in state and trigger remediation flows. * **Immutable Properties**: providers are in charge of reporting back if a user manually changed properties in the cloud provider. The idea here's to avoid configuration drifts from what was defined to what is running in the cloud provider. If true, the state is reported back to the managed resource. Crossplane won't delete the cloud provider resource, but it notifies back to allow actions to be taken. Other tools like terraform ([https://www.terraform.io](https://www.terraform.io)) automatically delete the remote resources in order to recreate them. * **Late initialization**: some properties in the `Managed Resources` can be optional, meaning that each provider selects the default values for these properties. When this happens Crossplane creates the resource with the default values and then sets the selected values into the `Managed Resource`. This simplifies the amount of configuration needed to create resources and reuses the sensible defaults defined by cloud providers usually in their user interfaces. * **Deletion**: when deleting a `Managed Resource`, the action is immediately triggered in the cloud provider, but the `Managed Resource` is kept until the resource is fully removed from the Cloud Provider. Errors that might happen during deletion on the cloud provider are added to the `Managed Resource` status field. * **Importing existing resources**: Crossplane doesn't necessarily needs to create the resources to manage them. You can create `Managed Resources` that start monitoring components which were created previously before Crossplane was installed. You can achieve this by using a specific Crossplane annotation on the `Managed Resource`: `crossplane.io/external-name` To summarize the interactions between Crossplane, the Crossplane GCP Provider and our `Managed Resources`, let's look at the following diagram: The following points indicates the sequence observed in Figure 5: 1. First, we need to create a resource, we can use any tool to create Kubernetes resources, `kubectl` here's an example. 2. If the resource that we created is a Crossplane `Managed Resource`, let's imagine a `CloudSQLInstance` resource, the specific Crossplane Provider picks it up and manages it. 3. The first step to execute when managing a resource is checking if it exists in the infrastructure (which is in the configured GCP account). If it doesn't exist the Provider sends a request for the resource to be created in the infrastructure. Depending on the properties set on the resource, for example which kind of SQL database is required, the appropriate SQL database is provisioned; imagine for the sake of the example that we chose a PostgreSQL database. 4. The Cloud Provider after receiving the request, if the resources are enabled, proceeds to create a new PostgreSQL instance with the configured parameters in the `Managed Resource`. 5. The status of the PostgreSQL is reported back to the `Managed Resource`, which means that we can use `kubectl`or any other tool to monitor the status of the provisioned resources. Crossplane providers keep these in sync. 6. When the database is up and running the Crossplane Provider creates a secret to store the credentials and properties that our applications need to connect to the newly created instance 7. Crossplane regularly checks the status of the PostgreSQL instance and updates the managed resource. ## Crossplane Configuration Packages If we create a `CloudSQLInstances` resource from the GCP provider we still make a strong reference to the provider. These GCP resources are detailed, as for example, it allows us to set up properties such as `DiskEncryptionConfiguration`, `GceZone`, `OnPremisesConfiguration` among others ([https://doc.crds.dev/github.com/crossplane/provider-gcp/database.gcp.crossplane.io/CloudSQLInstance/[email protected]](https://doc.crds.dev/github.com/crossplane/provider-gcp/database.gcp.crossplane.io/CloudSQLInstance/[email protected])) that we might want to standardize across teams or have a policy defined on how to set them. In order to abstract away these providers specific and low-level details on the resources, Crossplane allows us to define `Composite Managed Resources`. These compositions allow us to compose several provider specific resources, configure them together and expose a simple interface for the App Ops teams. You can create your packages with your own `Managed Resources` based on these compositions. These packages are easy to create as soon as you know who will be consuming them and what kind of resources they want to create. These packages also allow you to create the `Managed Resources` that can have different cloud providers implementations but expose the same simplified type for the users. At the end of the day a package is a set of configuration files that can be packaged as an OCI container and published to Docker Hub or any other container registry. That's all for this article. If you want to learn more about the book, check it out on Manning's liveBook platform here. # FAUN - Developer Community Get your weekly dose of the best hand-curated stories delivered to your inbox, for free! - [Website](https://www.faun.dev/) - [Podcast](https://www.faun.dev/podcast/) - [Twitter](https://twitter.com/faun_community) - [Facebook](https://www.facebook.com/fauncommunity) - [Instagram](https://www.instagram.com/faun_community/) - [Facebook Group](https://www.facebook.com/groups/faun.dev) - [LinkedIn Group](https://www.linkedin.com/groups/3271182/) - [Slack](https://www.faun.dev/slack/) - [iCloud Native News](https://www.faun.dev/icloud_native_news/) - [More](https://www.faun.dev/more/) If this post was helpful, please click the clap button below a few times to show your support for the author. # Tags Crossplane Continuous Delivery Kubernetes AWS Gcp # Author [Manning Publications](https://www.manning.com/) # Comments One user (Hutger H.) commented on the article: > Other tools like terraform ([https://www.terraform.io](https://www.terraform.io)) automatically delete the remote resources in order to recreate them. The author responded: > That's not true. Terraform will report the state and return a plan for reconciliation. It's at operator's discretion deciding for applying or not. For some resources, you have the option also to ignore_changes if applicable. # More articles from Manning Publications and FAUN - Developer Community * **In The Startup by Manning Publications** * The Startup by Manning Publications - [Repeated I ests I est](https://medium.com/the-startup/repeated-i-ests-i-est-9d70a66f0e40) * **In FAUN-Developer Community** * FAUN-Developer Community - [Top-5 Highly Recommended Cloud Certifications ](https://medium.com/faun-dev/top-5-highly-recommended-cloud-certifications-11dd92117bb6) by Ali Hamza * **In FAUN-Developer Community** * FAUN-Developer Community - [Top 10 Enterprise Technology Trends in 2025: Platform, Infrastructure, Data, Cybersecurity](https://medium.com/faun-dev/top-10-enterprise-technology-trends-in-2025-platform-infrastructure-data-cybersecurity-60a5d47e4be1) by Torsten V. * **In CodeX by Manning Publications** * CodeX by Manning Publications - [Learning Go via Pocket-Sized Projects](https://medium.com/codex/learning-go-via-pocket-sized-projects-7297ae553e86) by Aliénor Latour, Pascal Bertrand, and Donia. * **In FAUN-Developer Community** * FAUN-Developer Community - [Dependency Injection and Testing in JUnit](https://medium.com/faun-dev/dependency-injection-and-testing-in-junit-4aed731e686b) by Catalin Tudose * **In Cloud Platform Engineering** * Cloud Platform Engineering - [The Top 10 Internal Developer Platforms for 2024 (based on G2)](https://medium.com/cloud-platform-engineering/the-top-10-internal-developer-platforms-for-2024-based-on-g2-6f53827ad910) by Angelo * **In ITNEXT** * ITNEXT - [The 5 Key Mental Models Every Kubernetes Beginner Must Know ](https://medium.com/itnext/the-5-key-mental-models-every-kubernetes-beginner-must-know-7d1a5ae6551d) by Mayank Kumar * **In Peaksys Engineering** * Peaksys Engineering - [Two years of Backstage-behind the scenes of our developer portal - the Design System, a strategic advantage](https://medium.com/peaksys-engineering/two-years-of-backstage-behind-the-scenes-of-our-developer-portal-the-design-system-a-strategic-advantage-35f012ca15a2) by Nicolas Thibaut * **In AWS Tip** * AWS Tip - [EKS Auto Mode](https://medium.com/aws-tips/eks-auto-mode-5427164df34f) by Ravi Yasakeerthi * **In Better Programming** * Better Programming - [Running Dual-homed Teams Across the World](https://medium.com/better-programming/running-dual-homed-teams-across-the-world-7bbbf9664b48) by Javier Turegano * **In DevOps** * DevOps - [Enhance Terraform/Tofu Automation with GitHub Action](https://medium.com/devops/enhance-terraformtofu-automation-with-github-action-8789f1ed88bf) by Rishav Dhar