By Nelson Araujo, Software Engineer
Managing cloud resources is a critical part of the application lifecycle. That’s why today, we released and open sourced a set of comprehensive cookbooks for Chef users to manage Google Cloud Platform (GCP) resources.
Chef is a continuous automation platform powered by an awesome community. Together, Chef and GCP enable you to drive continuous automation across infrastructure, compliance and applications.
The new cookbooks allow you to define an entire GCP infrastructure using Chef recipes. The Chef server then creates the infrastructure, enforces it, and ensures it stays in compliance. The cookbooks are idempotent, meaning you can reapply them when changes are required and still achieve the same result.
The new cookbooks support the following products:
We also released a unified authentication cookbook that provides a single authentication mechanism for all the cookbooks.
We tested the cookbooks on CentOS, Debian, Ubuntu, Windows and other operating systems. Refer to the operating system support matrix for compatibility details. The cookbooks work with Chef Client, Chef Server, Chef Solo, Chef Zero, and Chef Automate.
To learn more about these Chef cookbooks, register for the webinar with myself and Chef’s JJ Asghar on 15 October 2017.
Using these new cookbooks is as easy as following these four steps:
Now, let’s discuss these steps in more detail.
You can find all the GCP cookbooks for Chef on Chef Supermarket. We also provide a “bundle” cookbook that installs every GCP cookbook at once. That way you can choose the granularity of the code you pull into your infrastructure.
Note: These Google cookbooks require neither administrator privileges nor special privileges/scopes on the machines that Chef runs on. You can install the cookbooks either as a regular user on the machine that will execute the recipe, or on your Chef server; the latter option distributes the cookbooks to all clients.
The authentication cookbook requires a few of our gems. You can install them using various methods, including using Chef itself:
chef_gem 'googleauth' chef_gem 'google-api-client'
For more details on how to install the gems, please visit the authentication cookbook documentation.
Now, you can go ahead and install the Chef cookbooks. Here’s how to install them all with a single command:
knife cookbook site install google-cloud
Or, you can install only the cookbooks for select products:
knife cookbook site install google-gcompute # Google Compute Engine knife cookbook site install google-gcontainer # Google Container Engine knife cookbook site install google-gdns # Google Cloud DNS knife cookbook site install google-gsql # Google Cloud SQL knife cookbook site install google-gstorage # Google Cloud Storage
To ensure maximum flexibility and portability, you must authenticate and authorize GCP resources using service account credentials. Using service accounts allows you to restrict the privileges to the minimum necessary to perform the job.
Note: Because service accounts are portable, you don’t need to run Chef inside GCP. Our cookbooks run on any computer with internet access, including other cloud providers. You might, for example, execute deployments from within a CI/CD system pipeline such as Travis or Jenkins, or from your own development machine.
Also make sure to enable the the APIs for each of the GCP services you intend to use.
Once you have your service account, add the following resource block to your recipe to begin authenticating with it. The resource name, here 'mycred' is referenced in the objects in the credential parameter.
gauth_credential 'mycred' do action :serviceaccount path '/home/nelsonjr/my_account.json' scopes ['https://www.proxy.ustclug.org/auth/compute'] end
For further details on how to setup or customize authentication visit the Google Authentication cookbook documentation.
You can manage any resource for which we provide a type. The example below creates an SQL instance and database in Cloud SQL. For the full list of resources that you can manage, please refer to the respective cookbook documentation link or to this aggregate summary view.
gsql_instance ‘my-app-sql-server’ do action :create project 'google.com:graphite-playground' credential 'mycred' end gsql_database 'webstore' do action :create charset 'utf8' instance ‘my-app-sql-server’ project 'google.com:graphite-playground' credential 'mycred' end
Note that the above code has to be described in a recipe within a cookbook. We recommend you have a “profile” wrapper cookbook that describes your infrastructure, and reference the Google cookbooks as a dependency.
Next, we direct Chef to enforce the recipe in the “profile” cookbook. For example:
$ chef-client -z --runlist ‘recipe[mycloud::myapp]’
In this example,
mycloud is the “profile” cookbook, and
myapp is the recipe that contains the GCP resource declarations.
Please note that you can apply the recipe from anywhere that Chef can execute recipes (client, server, automation), once or multiple times, or periodically in the background using an agent.
Now you're ready to start managing GCP resources with Chef, and start reaping the benefits of cross-cloud configuration management. Our plan is to continue to improve the cookbooks and add support for more Google products. We're also preparing to release the technology used to create these cookbooks as open source. If you have questions about this effort please visit Chef on GCP discussions forum, or reach out to us on firstname.lastname@example.org.