Infrastructure as Code -- chef overview

programmatically provision, configure and deploy

------->  treat like any other code base:source control, unit test

--------> build / rebuild whole environments using source     control,databackups  and compute resources.

But Why Chef?

 Are there better tools than Chef?

− Puppet

− Ansible

− Salt

− CFEngine

− FSS

− Bash Scripts

Maybe?

Are there better tools than Chef?

− Maybe

 Is Chef the right tool for every job?

− No!

Okay…Then Why Chef?

Pure Ruby

 Its Opensource Software

 Huge Active Community

 supermarket.chef.io

What is Chef?

Chef is a Policy Based Automation

Framework

− Policy is defined in the Chef code.

− Policy defines the “Desired State” of

our resources, but not how to get

there

− The nodes will pull their policy from

the Chef Server and chef-client will

enforce the policy

Chef has a Vocabulary

§ Resources

§ Recipes

§ Cookbooks

§ Roles

§ Environments

§ Organizations

§ Data Bags

§ Node

§ Run_list

Resources

Resources

− The basic building blocks of Chef

− Define a part of your infrastructure

and its state

− Can be:

> A package to be installed

> A service that should be running

> A file that should be created

> Much much more

Recipes

Recipes

− Are a collection of Resources that

define a specific task

− Can:

> Provision new nodes

> Install and configure an application

> Deploy application code

> Define monitoring checks

> Run other recipes

> Much much more

Cookbooks

Cookbooks

− Are a collection of Recipes that

defines all desired states for one

application

− Also contain:

> Default values for Chef Variables

(Attributes)

> Static Files / Templates

> Custom Resources (libraries, lwrps, etc…)

> Testing code

− Are VERSIONED

− Can also depend on / include other

cookbooks

Cookbooks

− Are the basic unit of configuration and

policy distribution

− Set sane defaults for expected

behavior

Roles

Roles

− Represent types of servers in your

infrastructure

− Define a horizontal slice of your

architecture

> Load Balancer

> Web Servers

> Application Servers

> Databases

Roles can also define policy

− Roles can have a run_list

− Roles can also define or override

attributes to change the default

behavior of a cookbook to tailor it to

its needs.

Environments

Environments

− Used to define the workflow our your

application

− Generally used to segment the

different life stages of your application

> Dev / Integration

> VDEV

> Staging

> Production

 GID is configured this way

 Corp is defined by Data Center, but

is the process of changing to use

best-practices

Environments also define policy

− Environments can also define or

override attributes

> Define different API locations for Dev /

Stage / Prod

> Dev sends alerts to team, Prod sends

alerts to L1

− Ideal place to pin to a specific version

of a cookbook.

Organizations

Organizations are

− Virtual independent Chef Servers

− Share nothing between other

Organizations

− Define different business units

> Gap

> GID

> Stores

Data Bags

Data Bags

− Free form data that is stored as JSON

data and is accessible from a Chef

server

− Indexed for searching

− Can be accessed directly via a recipe

or searched for

− Can be encrypted!

Nodes

Nodes

− Are virtual or physical nodes in our

infrastructure

− Will belong to only one Organization

− Will belong to one Environment

− May have zero to multiple roles

Nodes define policy

− Nodes can have a run_list

− Nodes can also define or override

attributes

Run Lists

A Run List defines all of the

information necessary for Chef to

configure a node into the desired

state

§ A Run List is:

− An ordered list of roles and/or recipes

that are run in the exact order defined

in the run-list

− Always specific to the node on which it

runs

− Stored as part of the node object on

the Chef server

Chef has a Tool Chain

§ Chef-Client

§ Ohai

§ Knife

§ Test-kitchen

§ Chefspec

§ ServerSpec

§ FoodCritic

§ Rubocop

Chef Tools

Chef-Client

− an agent that runs locally on every

node

− will perform all of the steps that are

required to bring the node into the

desired state

− Enforces policy on the node

Ohai

− is a tool that is used to detect

attributes on a node, and then provide

these attributes to the chef-client at

the start of every chef-client run

− Attributes that are collected by Ohai

are automatic attributes.

− These attributes are used to define the

state of the node.

> Number of CPUS

> How much Memory

> IP Address

Knife

− Command-line tool that provides an

interface between a local chef-repo

and the Chef server

− Can be used to verify the state of chef

objects on the server

Due to the number of active developers

using Chef, the xxxx restricts the use of

knife to our various pipelines and a

select group of admins

Chef Testing Tools

§ Test-Kitchen

− A tool to allow local testing of

cookbooks across multiple OS

platforms

§ Chefspec

− A framework that tests resources and

recipes as part of a simulated chefclient

run.

− Very Quick, runs in memory

§ ServerSpec

− A framework that tests actual state of

a target server

Chef Linting Tools

Foodcritic

− helps you detect problems that will

cause a runtime failure

− Helps you enforce desirable behavior

− Detects common anti-patterns

§ Rubocop

− Evaluate your Ruby coding style

against community standards

− Improve your code by iterating through

style warnings

(I hate Rubocop)

Automation / Chef Concepts

§ Desired State Configuration

§ Convergence

§ Idempotence

§ Desired State Configuration

− The cumulative definition of Policy as

described by cookbooks, roles,

environment, data bags applied to a

specific node.

§ Idempotence

− in math, means an operation can be

applied multiple times without

changing the result. 1 x 1 = 1 is an

idempotent operation of

multiplication.

In Chef, Resources are designed to be

idempotent. This means we can run the

same resource repeatedly, and if its

already in its desired state then nothing

happens.

§ Convergence

− is when the configuration management

system brings the node into

compliance with policy.

Once a node has achieved convergence,

we should be able to re-run chef-client

and nothing will change.

Chef is a Policy-Based Automation

Framework

§ Chef provides a domain-specific language (DSL) that allows you to

specify policy for your infrastructure

§ Policy is defined via:

− Cookbooks

− Roles

− Environment

− Nodes

§ Policy describes the desired state, but not how to get there

Nodes adhere to Policy

§ When the chef-client application is run on the node, it:

− Gathers the current system configuration of the node (Ohai)

− Downloads the desired system configuration policies from the Chef server

for that node (cookbooks, roles, data bags, environment)

− Configures the node such that it adheres to those policies

Run List specifies Policy

§ The Run List is an ordered collection of policies that the Node should

follow

§ Run List can be defined in only 2 Chef objects

− Roles

− Nodes

§ Chef-client obtains the Run List from the Chef Server

§ Chef-client ensures the Node complies with the policy in the Run List

A Quick-n-Dirty example using Chef-Apply

§ What is Chef-Apply?

− chef-apply is an executable program that allows you to work with resources

− It is included as part of the ChefDK

− A great way to explore resources

− NOT how you’ll eventually use Chef in production

PROBLEM: Our manager wants a webserver installed on our node

§ Let’s solve this using Chef-Apply to see Chef in action

DIGCA!

§ Determine the desired state of your infrastructure

§ Identify the Resources required to meet that state

§ Gather the Resources into Recipes

§ Compose a Run List from Recipes and Roles

§ Apply a Run List to each Node in your Environment

§ Your infrastructure adheres to the policy modeled in Chef!

DIGCA! Okay that is a terrible acronym.

Configuration Drift

§ Configuration Drift happens when:

− Your infrastructure requirements change

− The configuration of a server falls out of policy

§ Chef makes it easy to manage

− Model the new requirements in your Chef configuration files

− Run the chef-client to enforce your policies

§ Chef Configuration files + Source Control == Happiness

Design Tenants of Chef

§ Reasonability

− Default actions of a cookbook should do what is expected

§ Sane Defaults

− Default Attributes should address major of cases of default install

§ Flexibility

− Data Driven Cookbooks are better than static ones

Tenants borrowed from Larry Wall (aka creator of Perl)

§ Whipuptitude

− aptitude for whipping things up

§ Manipulexity

− manipulation of complex things