Problem and Solution
§ Problem: We don’t want to have to update
and re-upload a cookbook every time we
want to add a new repo
§ Solution: Abstract the addition of a yum
repo into a data structure outside of the
cookbook, leaving the cookbook static
Where could we store repo data?
§ Could use Node Attibutes.
− Messy
− You end up saving a lot of extra copies of data
on every node object
− Does not scale well
§ Hard to integrate with an external source
of truth
− A reminder that some Chef interfaces have easily
accessible APIs
What about Data Bags?
§ A data bag is a container for items
that represent information about your
infrastructure that is not tied to a
single node
§ Examples
− Users
− Groups
− Application Release Information
Interesting facts about Data Bags!
§ The whole Organization can see them
− maaaaaaaaabye that’s good?
− maaaaaaaaabye that’s bad?
§ Not versioned
§ Can be encrypted with chef-vault and git-crypt
§ NOT MEANT TO BE A TRANSACTIONAL DATABASE
Creating a collection of data bag entries
mkdir –p data_bags/repos!
$ tree data_bags/!
data_bags/!
└── repos!
!
1 directory, 0 files
Create a local copy of JSON formatted data
$ mate data_bags/repos/epel-6-latest.json
TextMate File Edit View Text Navigation Bundles Window help
{
"baseurl" : "http://ks64.phx.gapinc.dev/RHN/epel-6-latest",
"enable": 0,
"id": "epel-6-latest"
}
Did someone mumble JSON?
Does every aspect of chef use a different language? Are you just
messing with us?
JSON - JavaScript Object Notation
http://json.org
§ JSON is built on two structures:
§ A collection of name/value pairs. In various languages, this is realized
as an object, record, struct, dictionary, hash table, keyed list, or
associative array.
object
^------------------------------------------------------^
|----------- { ------^----------string------- : ---------value------------^---------- } ------|
v------------------------- , ---------------------------v
http://json.org
§ JSON is built on two structures:
§ An ordered list of values. In most languages, this is realized as an
array, vector, list, or sequence.
array
^------------------------------------------------------^
|----------- [ ------^----------------------value------------------------------ ] ---------|
v------------------------- , ---------------------------v
Upload some data to a collection named repos
$ knife data bag from file repos epel-6-latest.json!
Updated data_bag_item[repos::epel-6-latest]!
Create a local copy of JSON formatted data
$ mine data_bags/repos/gapCorp.json!
{
"id" : "gapcorp",
"baseurl": "http://ks64.phx.gapinc.dev/gapcorp",
"enabled":0
}
Upload some data to a collection named repos
$$ knife data bag from file repos gapCorp.json
Updated data_bag_item[repos::gapCorp]
Search the data bag named repos for anything
$ knife search repos "*:*"
2 items found!
!
baseurl: http://ks64.phx.gapinc.dev/RHN/epel-6-latest!
chef_type: data_bag_item!
data_bag: repos!
enabled: 0!
id: epel-6-latest!
!
baseurl: http://ks64.phx.gapinc.dev/gapCorp!
chef_type: data_bag_item!
data_bag: repos!
enabled: 0!
id: gapCorp!
Search for just some fields/attributes
$ knife search repos "id:gapCorp" -a baseurl
1 items found!
:
baseurl: http://ks64.phx.gapinc.dev/gapCorp
chef generate a repos cookbook
$ cd cookbooks/!
$ chef generate cookbook repos!
Compiling Cookbooks...!
Recipe: code_generator::cookbook!
* directory[/Users/lamont/chef-repo/cookbooks/repos] action create!
- create new directory /Users/lamont/chef-repo/cookbooks/repos!
* template[/Users/lamont/chef-repo/cookbooks/repos/metadata.rb] action create_if_missing!
- create new file /Users/lamont/chef-repo/cookbooks/repos/metadata.rb!
- update content in file /Users/lamont/chef-repo/cookbooks/repos/metadata.rb from none to 65281f!
(diff output suppressed by config)!
* template[/Users/lamont/chef-repo/cookbooks/repos/README.md] action create_if_missing!
- create new file /Users/lamont/chef-repo/cookbooks/repos/README.md!
- update content in file /Users/lamont/chef-repo/cookbooks/repos/README.md from none to dfdded!
(diff output suppressed by config)!
...!
* template[/Users/lamont/chef-repo/cookbooks/repos/recipes/default.rb] action create_if_missing!
- create new file /Users/lamont/chef-repo/cookbooks/repos/recipes/default.rb!
- update content in file /Users/lamont/chef-repo/cookbooks/repos/recipes/default.rb from none to
d14ea9!
(diff output suppressed by config)!
* cookbook_file[/Users/lamont/chef-repo/cookbooks/repos/.gitignore] action create!
- create new file /Users/lamont/chef-repo/cookbooks/repos/.gitignore!
- update content in file /Users/lamont/chef-repo/cookbooks/repos/.gitignore from none to dd37b2!
(diff output suppressed by config)!
Copy a gist of cookbooks/repos/recipes/default.rb
$ wget -O default.rb http://bit.ly/1LgnY3M!
--2015-08-17 01:06:32-- http://bit.ly/1LgnY3M!
Resolving bit.ly... 69.58.188.40, 69.58.188.39!
Connecting to bit.ly|69.58.188.40|:80... connected.!
HTTP request sent, awaiting response... 301 Moved Permanently!
Location: http://github.gapinc.dev/gist/La2o5e5/c3a6b3a48c62a1d4fd48/raw/
8335dbf755785df924d9e07fa590322578d86db2/gistfile1.txt [following]!
--2015-08-17 01:06:32-- http://github.gapinc.dev/gist/La2o5e5/
c3a6b3a48c62a1d4fd48/raw/8335dbf755785df924d9e07fa590322578d86db2/
gistfile1.txt!
Resolving github.gapinc.dev... 10.105.65.29!
Connecting to github.gapinc.dev|10.105.65.29|:80... connected.!
HTTP request sent, awaiting response... 200 OK!
Length: unspecified [text/plain]!
Saving to: 'default.rb'!
!
default.rb [ <=> ] 550 --.-KB/s in 0s!
Should look like
#!
# Cookbook Name:: repos!
# Recipe:: default!
#!
# Copyright (c) 2015 The Authors, All Rights Reserved.!
execute "yum_clean" do!
command "/usr/bin/yum clean all && /usr/bin/yum makecache"!
action :nothing!
end!
!
repos = search('repos', '*:*')!
repos.each do |repo|!
template "/etc/yum.repos.d/#{repo['id']}.repo" do!
source 'repo.erb'!
owner 'root'!
group 'root'!
mode 0644!
variables(!
:id => repo['id'],!
:baseurl => repo['baseurl'],!
:enabled => repo['enabled']!
)!
notifies :run, 'execute[yum_clean]'!
end
end
RubyMine view
#!
# Cookbook Name:: repos!
# Recipe:: default!
#!
# Copyright (c) 2015 The Authors, All Rights Reserved.!
execute "yum_clean" do!
command "/usr/bin/yum clean all && /usr/bin/yum makecache"!
action :nothing!
end!
!
repos = search('repos', '*:*')!
repos.each do |repo|!
template "/etc/yum.repos.d/#{repo['id']}.repo" do!
source 'repo.erb'!
owner 'root'!
group 'root'!
mode 0644!
variables(!
:id => repo['id'],!
:baseurl => repo['baseurl'],!
:enabled => repo['enabled']!
)!
notifies :run, 'execute[yum_clean]'!
end
end
The Search
repos = search('repos', '*:*')
repos.each do |repo|
template "/etc/yum.repos.d/#{repo['id']}.repo" do
source 'repo.erb'
owner 'root'
group 'root'
mode 0644
variables(
:id => repo['id'],
:baseurl => repo['baseurl'],
:enabled => repo['enabled']
)
notifies :run, 'execute[yum_clean]'
end
end
§ We use the same search query we
tested with the knife command
§ Search returns an array of data bag
items
§ We loop over each item binding it
to the variable repos
§ Inside the loop, create a template
per repo
§ Does the search happen during
compile or execute phase?
Edit a new repo.erb template
$ mkdir -p repos/templates/default
$ vi repos/templates/default/repo.erb
[<%= @id %>]
name=<%= @id %>
baseurl=<%= @baseurl %>
gpgcheck=0
enabled=<%= @enabled %>
Upload the repos cookbook
$ knife cookbook upload repos
Uploading repos [0.1.0]
Uploaded 1 cookbook.
Add the recipe to the run_list
knife node run_list add node1 'recipe[repos]'
node1:
run_list:
recipe[mysite]
recipe[motd]
recipe[repos]
Converge chef-client
sudo chef-client
Starting Chef Client, version 12.4.1
resolving cookbooks for run list: ["mysite", “motd”, "repos"]
Synchronizing Cookbooks:
- mysite
- repos
Compiling Cookbooks...
Converging 12 resources
Client converges with repos
Recipe: repos::default!
* execute[yum_clean] action nothing (skipped due to action :nothing)!
...!
* template[/etc/yum.repos.d/gapCorp.repo] action create!
- create new file /etc/yum.repos.d/gapCorp.repo!
- update content in file /etc/yum.repos.d/gapCorp.repo from none to f339d5!
--- /etc/yum.repos.d/gapCorp.repo !2015-08-17 04:22:43.970999979 -0400!
+++ /tmp/chef-rendered-template20150817-12289-utdyg7 !2015-08-17 04:22:43.970999979 -0400!
@@ -1 +1,6 @@!
+[gapCorp]!
+name=gapCorp!
+baseurl=http://ks64.phx.gapinc.dev/gapCorp!
+gpgcheck=0!
+enabled=0!
- change mode from '' to '0644'!
- change owner from '' to 'root'!
- change group from '' to 'root'!
* execute[yum_clean] action run!
- execute /usr/bin/yum clean all && /usr/bin/yum makecache!
!
Running handlers:!
Running handlers complete!
Chef Client finished, 3/12 resources updated in 4.643865038 seconds!
Examine repos
ls -l /etc/yum.repos.d/
[gaptech@node1 ~]$ ls -l /etc/yum.repos.d/!
total 16!
-rw-r--r-- 1 root root 109 Aug 17 04:22 epel-6-latest.repo
-rw-r--r-- 1 root root 87 Aug 17 04:22 gapCorp.repo
-rw-r--r-- 1 root root 144 Aug 13 01:12 os65.repo
-rw-r--r-- 1 root root 238 Aug 13 01:12 rhn_rhn_tools_rhel_x86_64_server_6.repo
Examine repos
ccat /etc/yum.repos.d/gapCorp.repo
[gapCorp]
name=gapCorp
baseurl=http://ks64.phx.gapinc.dev/gapCorp
gpgcheck=0
enabled=0
Review
§ We created a centralized organization-wide key/
value store
§ We wrote a generic cookbook that needs no
further updating no matter how many additional
repos we wish to define
§ We used knife commands from our workstation to
establish a (trivial) search
§ We implemented that search in our repos
cookbook
No comments:
Post a Comment