A love story of Template Variables, Notifications and Controlling
Idempotency.
The Problem: Manager now wants a
website for multiple Gap Brands
§ The Problem: We need to deploy multiple Gap brand websites on our
webserver
§ Solution: Have Gap and Old Navy sites running on different ports on
the same webserver
Time for a minor refactor of a cookbook.
(Git work)
When it is time to refactor your cookbook, the first thing you should do
is do a little clean up in your git directory.
§ Remember to git pull to get the latest version of the code from
master
§ Create a new branch for your work
Next if we are going to functionally change the way our cookbook
works, we should probably bump the version number.
Time for a minor refactor of a cookbook.
(Bump version in metadata.rb)
name 'mysite'!
maintainer 'The Authors'!
maintainer_email 'you@example.com'!
license 'all_rights'description'!
Installs/Configures mysite'!
long_description 'Installs/Configures mysite'!
version '0.2.0'!
!§
Version numbers follow Semantic Versioning Policy
§ Major, Minor, Patch
Create the attributes that define our new
websites
(mysite/attributes/default.rb)
default['mysite']['indexfile'] = 'index1.html' !
default['mysite']['sites']['gap'] = { 'port' => 80 }!
default['mysite’]['sites']['oldnavy'] = { 'port' => 81 }!
!
We can define all of our new websites under a new attribute called
sites, which contains data for the website name and port it should
listen on.
§ One for Gap running on port 80
§ One for Old Navy running on port 81
Task out the work for the refactor
package 'httpd’!
!
service 'httpd' do!
supports :reload => true!
action [:start, :enable]!
end!
!
cookbook_file '/var/www/html/index.html' do!
source 'index.html’!
mode '0644’!
owner 'nobody’!
group 'nobody’!
notifies :reload, 'service[httpd]’!
end!
Tasks:
§ Disable the default virtual host
§ Iterate over apache sites
− Set the document root
− Add template for new Virtual Host
config
− Create new directory for docroot
− Add template for site index.html
Task: Disable the default virtual host
§ To disable the default virtual host in apache, we need to delete or
rename the file /etc/httpd/conf.d/welcome.conf
§ Its an odd requirement, since we should only have to do this once.
§ Can someone recommend a resource for this action?
§ Check out docs.chef.io/resources
The resource of last resort - execute
Test/Repair? What if there is no Test?
First sentence for execute resource from docs.chef.io:
§ Use the execute resource to execute a single command. (bingo)
So we could solve our task with the following code:
execute 'mv /etc/httpd/conf.d/welcome.conf /etc/httpd/conf.d/welcome.conf.disabled' do!
notifies :restart, 'service[httpd]' !
end!
Does anyone see an issue with this execute resource?
Execute resources are generally
idempotent
The second sentence for execute resource from docs.chef.io:
Commands that are executed with this resource are (by their nature)
not idempotent
§ Most Unix/Windows command line utilities are not designed to be
idempotent. They expect to be run by a Human or wrapped in a
script with tests.
§ In our example, the mv command will be successful the first time, but
will fail every subsequent chef client run
§ It is up to you to make your execute resources idempotent.
§ Use not_if and only_if to guard this resource for idempotence.
Enter the not_if / only_if guard
metaparameters
From docs.chef.io/resources.html#guards
A guard property is useful for ensuring that a resource is idempotent by
allowing that resource to test for the desired state as it is being
executed, and then if the desired state is present, for the chef-client
to do nothing.
§ Use not_if and only_if to guard this resource for idempotence.
only_if do!
File.exist?('/etc/httpd/conf.d/welcome.conf')!
end!
!§
The only_if executes if the test is true
§ The not_if executes if the test is false
Notifications
A different metaparameter
notifies :restart, 'service[httpd]'!
!§
Resource Notifications in Chef are used to trigger action on a resource
when the current resources are successful
§ "If we change the configuration file, restart apache"
§ The first argument is an action, the second is the string
representation of the target resource in the resource_collection
§ Like not_if and only_if, notifies is a metaparameter and can be
applied to any resource.
§ Notifies - Which resource takes action when this resource’s state
changes
§ Subscribes - Specify that this resource is to listen to another
resource, and then take action when that resource’s state changes
Task: Disable the default virtual host
Add our new execute resource
(mysite/recipes/default.rb)
package 'httpd'!
!
service 'httpd' do!
action [:start, :enable]!
end!
!
execute 'mv /etc/httpd/conf.d/welcome.conf /etc/httpd/conf.d/welcome.conf.disabled' do!
only_if do!
File.exist?('/etc/httpd/conf.d/welcome.conf')!
end!
notifies :restart, 'service[httpd]'!
end!
!
node.default['mysite']['indexfile'] = 'index2.html'!
cookbook_file '/var/www/html/index.html' do!
source node['mysite']['indexfile']!
mode '0644'!
owner 'nobody'!
group 'nobody'!
Task: Iterate over apache sites
Delete the cookbook_file resource
(mysite/recipes/default.rb)
!
execute 'mv /etc/httpd/conf.d/welcome.conf /etc/httpd/conf.d/welcome.conf.disabled' do!
only_if do!
File.exist?('/etc/httpd/conf.d/welcome.conf')!
end!
notifies :restart, 'service[httpd]'!
end!
!
node.default['mysite']['indexfile'] = 'index2.html'!
cookbook_file '/var/www/html/index.html' do!
source node['mysite']['indexfile']!
mode '0644'!
owner 'nobody'!
group 'nobody'!
notifies :reload, 'service[httpd]
end
Task: Iterate over apache sites
(mysite/recipes/default.rb)
!
execute 'mv /etc/httpd/conf.d/welcome.conf /etc/httpd/conf.d/welcome.conf.disabled' do!
only_if do!
File.exist?('/etc/httpd/conf.d/welcome.conf')!
end!
notifies :restart, 'service[httpd]'!
end!
!
# Iterate over the apache sites!
node['mysite']['sites'].each do |site_name, site_data|!
# set the docroot!
document_root = "/srv/apache/#{site_name}"!
!!
§ node['apache']['sites'] is a ruby hash, with keys and values
Task: Iterate over apache sites
What is happening here?
(mysite/recipes/default.rb)
node['mysite']['sites'].each do |site_name, site_data|!
document_root = "/srv/apache/#{site_name}"!
!
§ Calling .each causes ruby to look over each site
!
default['mysite']['sites']['gap'] = { 'port' => 80 }!
default['mysite’]['sites']['oldnavy'] = { 'port' => 81 }!
!
§ First Pass
− site_name = 'gap'
− site_data = { 'port' => 80 }
!
§ Second Pass
− site_name = 'oldnavy'
− site_data = { 'port' => 81 }
Task: Iterate over apache sites
Set the document_root
(mysite/recipes/default.rb)
node['mysite']['sites'].each do |site_name, site_data|!
document_root = "/srv/apache/#{site_name}"!
!!
§ Create a variable called document_root!
§ #{site_name} means "insert the value of variable site_name here"
§ First Pass
− document_root = '/src/apache/gap'
§ Second Pass
− document_root = '/src/apache/oldnavy'
Task: Iterate over apache sites
Add template for new Virtual Host Config
(mysite/recipes/default.rb)
# Iterate over the apache sites!
node['mysite']['sites'].each do |site_name, site_data|!
# set the docroot!
document_root = "/srv/apache/#{site_name}"!
!
template '/etc/httpd/conf.d/#{site_name}.conf' do!
source 'custom.erb'!
mode '0644'!
variables(!
:document_root => document_root,!
:port => site_data['port']!
)!
notifies :restart, 'service[httpd]'
end
Template Variables
variables(!
:document_root => document_root,!
:port => site_data['port']!
)!
§ Not all data you might need in a template is necessarily a node
attribute
§ The variables parameter lets you pass in custom data for use in the
template ERB
Task: Iterate over apache sites
Create new directory for document_root
§ Use a directory resource
§ The name is document_root!
§ The resource should have at least 2 parameters:
− mode is '0755'
− recursive is set to true
§ Can you guess what the recursive parameter does?
(Check out docs.chef.io/resources.html#directory for more info
Task: Iterate over apache sites
Create new directory for document_root
(mysite/recipes/default.rb)
!
template '/etc/httpd/conf.d/#{site_name}.conf' do!
source 'custom.erb'!
mode '0644'!
variables(!
:document_root => document_root,!
:port => site_data['port']!
)!
notifies :restart, 'service[httpd]'!
end!
!
directory document_root do!
mode '0755'!
recursive true!
end!
Task: Iterate over apache sites
Add template for site index.html
(mysite/recipes/default.rb)
directory document_root do!
mode '0755'!
recursive true!
end!
!
template "#{document_root}/index.html" do!
source 'index.html.erb'!
mode '0644'!
variables(!
:site_name => site_name,!
:port => site_data['port']!
)!
end!
end!
Task: Iterate over apache sites
Don't forget to close the iterator loop
(mysite/recipes/default.rb)
!
directory document_root do!
mode '0755'!
recursive true!
end!
!
template "#{document_root}/index.html" do!
source 'index.html.erb'!
mode '0644'!
variables(!
:site_name => site_name,!
:port => site_data['port']!
)!
end!
end!
Create the custom.erb template
(mysite/templates/default/custom.erb)
§ Note the two template variables are
prefixed by an @ symbol
§ We use a conditional if in the <% -%>
block
§ Don't type this ERB, its mostly just
apache configuration with some ERB
thrown in.
§ Download it:
http://bit.ly/1TMUkI5
<% if @port != 80 -%>!
Listen <%= @port %>!
<% end -%>!
!<
VirtualHost *:<%= @port %>>!
ServerAdmin webmaster@localhost!
DocumentRoot <%= @document_root %>!
<Directory />!
Options FollowSymLinks!
AllowOverride None!
</Directory>!
<Directory <%= @document_root %>!
Options Indexes FollowSymLinks MultiViews!
AllowOverride None!
Order allow,deny!
allow from all!
</Directory> !
</VirtualHost>!
Create the index.html.erb
(mysite/templates/default/index.html.erb)
<html>!
<body>!
<h1>Welcome to Gap, Inc.</h1>!
<h2>We love <%= @site_name %></h2>!
<%= node['ipaddress'] %>:<%= @port %>!
</body>!
</html>!
!
§ This ERB uses both node attributes and template variables
§ Note the two template variables are prefixed by an @ sign again
§ Gist: http://bit.ly/1hhExzW!
Update default .kitchen.yml to use our
vagrant boxes
(mysite/.kitchen.yml)
---!
driver:!
name: vagrant!
!
provisioner:!
name: chef_zero!
!
platforms:!
- name: rhel66!
driver_config:!
box: packer-rhel66!
box_url: http://repo1.phx.gapinc.dev/gapSoftware/vagrant-boxes/packer-rhel66.box!
!
suites:!
- name: mysite!
run_list:!
- recipe[mysite::default]!
attributes:!
Run kitchen converge to test the cookbook
kitchen converge
-----> Starting Kitchen (v1.4.0)!
-----> Creating <default-rhel66>...!
Bringing machine 'default' up with 'virtualbox' provider...!
==> default: Importing base box 'packer-rhel66'...!
==> default: Matching MAC address for NAT networking...!
==> default: Setting the name of the VM: kitchen-mysite-defaultrhel66_
default_1439686485320_90515!
Skipping Berkshelf with --no-provision!
==> default: Clearing any previously set network interfaces...!
==> default: Preparing network interfaces based on configuration...!
default: Adapter 1: nat!
==> default: Forwarding ports...!
default: 22 => 2222 (adapter 1)!
==> default: Booting VM...!
==> default: Waiting for machine to boot. This may take a few minutes...!
default: SSH address: 127.0.0.1:2222!
default: SSH username: vagrant!
default: SSH auth method: private key!
What failed?
Running handlers:!
[2015-08-15T18:47:32-07:00] ERROR: Running exception handlers!
Running handlers complete!
[2015-08-15T18:47:32-07:00] ERROR: Exception handlers complete!
Chef Client failed. 10 resources updated in 126.409526806 seconds!
[2015-08-15T18:47:32-07:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chefstacktrace.
out!
[2015-08-15T18:47:32-07:00] ERROR: service[httpd] (mysite::default line 9) had an error:
Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'!
---- Begin output of /sbin/service httpd start ----!
STDOUT: Starting httpd: [FAILED]!
STDERR: Syntax error on line 11 of /etc/httpd/conf.d/#{site_name}.conf:!
<Directory> directive missing closing '>'!
---- End output of /sbin/service httpd start ----!
Ran /sbin/service httpd start returned 1!
[2015-08-15T18:47:33-07:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process
exited unsuccessfully (exit code 1)!
Update the custom.erb template
(mysite/templates/default/custom.erb)
<% if @port != 80 -%>!
Listen <%= @port %>!
<% end -%>!
!<
VirtualHost *:<%= @port %>>!
ServerAdmin webmaster@localhost!
DocumentRoot <%= @document_root %>!
<Directory />!
Options FollowSymLinks!
AllowOverride None!
</Directory>!
<Directory <%= @document_root %> >!
Options Indexes FollowSymLinks MultiViews!
AllowOverride None!
Order allow,deny!
allow from all!
</Directory> !
</VirtualHost>!
Fix string interpolation error in default.rb
(mysite/recipes/default.rb)
# Iterate over the apache sites!
node['mysite']['sites'].each do |site_name, site_data|!
# set the docroot!
document_root = "/srv/apache/#{site_name}"!
!
template "/etc/httpd/conf.d/#{site_name}.conf" do!
source 'custom.erb'!
mode '0644'!
variables(!
:document_root => document_root,!
:port => site_data['port']!
)!
notifies :restart, 'service[httpd]'!
end!
!
§ Only " " does string interpolation, ' ' is just a plain old string with odd characters in it
§ Update the template resource to use " "
Run kitchen converge to test the cookbook
(Lather, Rinse, Repeat)
$ kitchen converge
-----> Starting Kitchen (v1.4.0)!
-----> Creating <default-rhel66>...!
Bringing machine 'default' up with 'virtualbox' provider...!
==> default: Importing base box 'packer-rhel66'...!
==> default: Matching MAC address for NAT networking...!
==> default: Setting the name of the VM: kitchen-mysite-defaultrhel66_
default_1439686485320_90515!
Skipping Berkshelf with --no-provision!
==> default: Clearing any previously set network interfaces...!
==> default: Preparing network interfaces based on configuration...!
default: Adapter 1: nat!
==> default: Forwarding ports...!
default: 22 => 2222 (adapter 1)!
==> default: Booting VM...!
==> default: Waiting for machine to boot. This may take a few minutes...!
default: SSH address: 127.0.0.1:2222!
default: SSH username: vagrant!
default: SSH auth method: private key!
This again? I thought we fixed that...
$ kitchen converge
Running handlers:!
[2015-08-15T18:47:32-07:00] ERROR: Running exception handlers!
Running handlers complete!
[2015-08-15T18:47:32-07:00] ERROR: Exception handlers complete!
Chef Client failed. 10 resources updated in 126.409526806 seconds!
[2015-08-15T18:47:32-07:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chefstacktrace.
out!
[2015-08-15T18:47:32-07:00] ERROR: service[httpd] (mysite::default line 9) had an error:
Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'!
---- Begin output of /sbin/service httpd start ----!
STDOUT: Starting httpd: [FAILED]!
STDERR: Syntax error on line 11 of /etc/httpd/conf.d/#{site_name}.conf:!
<Directory> directive missing closing '>'!
---- End output of /sbin/service httpd start ----!
Ran /sbin/service httpd start returned 1!
[2015-08-15T18:47:33-07:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process
exited unsuccessfully (exit code 1)!
Chef will only apply policy to files that are
under its management
§ Even though we fixed our recipe and our ERB tempate, we still have a
problem.
§ The file /etc/httpd/conf.d/#{site_name}.conf still exists on the
server from the last run.
§ If this was a live server, we'd have to write a contra-resource to get
Chef to delete the file.
§ Luckly, this is test-kitchen, kill the box and re-converge:
− kitchen destroy
− kitchen converge
Run kitchen converge to test the cookbook
(Lather, Rinse, Repeat)
kitchen converge
+<h1>Welcome to Gap, Inc.</h1>!
+<h2>We love oldnavy</h2>!
+10.0.2.15:81!
+</body>!
+</html>!
!
- restart service service[httpd]!
Running handlers:!
Running handlers complete!
Chef Client finished, 11/11 resources updated in 136.216494692 seconds!
Finished converging <default-rhel66> (2m18.22s).!
-----> Kitchen is finished. (3m5.93s)!
Success!
§ Run kitchen converge again to prove it is idempotent.
Upload our cookbook to the Chef Server
knife cookbook upload mysite!
Uploading mysite [0.2.0]!
Uploaded 1 cookbook.
Check your work
Welcome to Gap, Inc.
we love gap
172.xxxx.xxx:80
Best Practice: Recipes contain the pattern,
attributes provide the data
§ Recipes contain the pattern: How do automatically deploy multiple
Apache Virtual Hosts?
§ Attributes provide the data: What Virtual Hosts should we deploy?
The goal is to write a single tested cookbook to manage an application.
§ It should do something sane if no attributes are overridden
§ But be flexible enough to change its default behavior if different data
is passed to it.
No comments:
Post a Comment