OSU DevOps Bootcamp Documentation Release - Read the Docs

OSU DevOps Bootcamp Documentation
Release 0.0.1
OSU OSL
OSU LUG
October 11, 2014
Contents
1
2
Contents:
1.1 Get Involved . . . . . . . . . . . . . . . . . . . . . . .
1.2 Policies . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3 Curriculum . . . . . . . . . . . . . . . . . . . . . . . .
1.4 BootCamp Guide . . . . . . . . . . . . . . . . . . . . .
1.5 Recommended Resources . . . . . . . . . . . . . . . .
1.6 FAQ . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.7 What is DevOps? . . . . . . . . . . . . . . . . . . . . .
1.8 Purpose of DevOps BootCamp . . . . . . . . . . . . . .
1.9 Vagrant . . . . . . . . . . . . . . . . . . . . . . . . . .
1.10 DevOps Bootcamp Introduction . . . . . . . . . . . . .
1.11 Lesson 1: The Very Basics . . . . . . . . . . . . . . . .
1.12 Lesson 2: Single System Fundamentals . . . . . . . . .
1.13 Lesson 2: Homework . . . . . . . . . . . . . . . . . . .
1.14 Lesson 3: Editors & Version Control . . . . . . . . . .
1.15 Lesson 3: Intro to Git . . . . . . . . . . . . . . . . . .
1.16 Lesson 3: GitHub . . . . . . . . . . . . . . . . . . . .
1.17 Lesson 4: Scripting, Troubleshooting, & Workflow . . .
1.18 Lesson 5: Web Applications . . . . . . . . . . . . . . .
1.19 Lesson 6: Boot and the Filesystem Hierarchy . . . . . .
1.20 Lesson 7: Databases . . . . . . . . . . . . . . . . . . .
1.21 Lesson 8: Security & Authentication . . . . . . . . . .
1.22 Lesson 8: Web application security . . . . . . . . . . .
1.23 Lesson 9: Networking . . . . . . . . . . . . . . . . . .
1.24 Lesson 10: Networking part 2 . . . . . . . . . . . . . .
1.25 Lesson 11: Devops & Configuration Management Intro
1.26 Lesson 12: Configuration Management . . . . . . . . .
1.27 Lesson 13: Contributing to Open Source . . . . . . . .
1.28 Lesson 14: Review . . . . . . . . . . . . . . . . . . . .
1.29 Lesson 16: Email . . . . . . . . . . . . . . . . . . . . .
Indices and tables
Bibliography
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
3
3
3
5
7
7
8
9
9
10
12
15
28
39
39
46
50
54
63
68
81
88
100
104
111
125
134
142
147
149
155
157
i
ii
OSU DevOps Bootcamp Documentation, Release 0.0.1
The OSL is kicking off DevOps BootCamp this year with the brand new DevOps DayCamp! DevOps DayCamp is
a dual-track day with one track to help inexperienced attendees get started with DevOps, as well as a second track
comprised of a hands-on hackathon with educational sessions throughout the day for the more advanced DevOps
crowd. Advanced sessions will be given by industry professionals and will include Ansible, Travis CI and Docker. For
more information, checkout http://devopsbootcamp.osuosl.org/daycamp/.
DevOps BootCamp is an OSU Open Source Lab and OSU Linux Users Group program that aims to boost our impact
and outreach while shrinking the skills gap for OSU students interested in DevOps.
The OSU Open Source Lab provides quality hosting for more than 160 open source projects. Along with delivering over 460 terabytes of open source data a month, the OSL employs and mentors students to be expert systems
administrators and software developers ready to enter the industry. The OSL encounters more demand for qualified
students than the employees graduating each year can supply. WeвЂ™d like to give back to OSU and the wider open
source community by training interested students from all disciplines in the real-world skills necessary for success as
software developers and systems engineers. To this end, weвЂ™ll be offering an extracurricular training program with 2 to
3 sessions per month through the school year, in which current OSL students and software professionals will teach all
topics that are essential for success. Although hands-on labs will be offered later in the program, no initial familiarity
with software development or Linux will be assumed.
This program is based on the PSU Braindump, where for 21 years they have been turning PSU students into sysadmins
who have rapidly been flowing into the growing number of startups in Portland.
The DevOps BootCamp program will be dynamically shaping itself based on the feedback we get from interested
students.
Contents
1
OSU DevOps Bootcamp Documentation, Release 0.0.1
2
Contents
CHAPTER 1
Contents:
1.1 Get Involved
1.1.1 Mailing list
Join the mailing list for updates.
1.1.2 IRC
Join us on irc.freenode.net in #devopsbootcamp (students will be setting up an IRC network for the
program in a later lesson).
1.1.3 Website & Curriculum
If youвЂ™d like to help edit this site, email devopsbootcamp or ping anyone in #devopsbootcamp on Freenode with
your github username to get access to the web site repo. YouвЂ™ll also want to learn the ReStructured Text markup
language to edit the site, if you donвЂ™t already know it.
1.1.4 Meetings
We typically meet weekly in KEC 1007 or 1005 depending on the room schedules. Please check back later for when
our first meeting in the Fall will be. Recurring meeting times following the first meeting will be decided based on the
availability of those who show up for the first meeting. Last year we met on Thursdays from 6-8pm.
1.2 Policies
1.2.1 вЂњThe DealвЂќ
You get:
вЂў Mentorship from students and professionals with advanced skills in software development and systems administration
вЂў Professional connections in the software industry
3
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂў A welcoming environment to start learning in if youвЂ™ve always wanted to learn about software development and
systems administration but have never been sure where to start
вЂў An opportunity to fill in any gaps in your knowledge if youвЂ™re a self-taught coder or sysadmin
вЂў The skills to build and deploy open source software or to contribute to existing projects
Why weвЂ™re offering this:
вЂў The OSL gets a larger pool of candidates to recommend to companies interested in recruiting OSL students
вЂў The OSL gets to work with a wider variety of students, which fits better with its new status as part of the school
of EECS
вЂў The open source community gets more contributors to its projects
1.2.2 Attendance
We will schedule BootCamp meetings for a time that works for a majority of attendees. Each meetingвЂ™s curriculum
will build on what you learned the previous session, so if you miss a meeting youвЂ™re responsible for studying the basics
of its content on your own. To facilitate this, we will provide resources for independent study about each topic that we
cover.
BootCamp leaders will be available at times outside of the regular meetings to help answer any questions about the
training programвЂ™s content. We will not spend class time reviewing material for those who skip a lecture. If you attend
a lesson and donвЂ™t understand something from it, youвЂ™re welcome to ask in class, because others very likely have the
same concern. But if you want to make up a class you skipped, itвЂ™s disrespectful to those who attended to spend their
class time on questions which you could resolve on your own time.
1.2.3 Laptops
As the course progresses, you will need a laptop. We hope and recommend that you will decide to set up your laptop
to dual-boot to Linux as the course progresses, but but itвЂ™s not required. If you donвЂ™t own a laptop and are an OSU
student you can check out a laptop from the OSU Library for at least 24 hours at a time.
Realistically, as long as itвЂ™s new enough to boot from USB and connect to wireless networks, your laptopвЂ™s specifications donвЂ™t matter much. If itвЂ™s so new that its UEFI configuration prevents you from dual-booting with Linux, it
will be powerful enough to run virtual machines. If itвЂ™s old enough to be unable to run VMs but still has wireless
connectivity, weвЂ™ll teach you how to ssh to a remote server to perform more computationally intensive tasks.
If you are not an OSU student and do not have access to a working laptop, contact the DevOps BootCamp organizers
and weвЂ™ll see whether we can arrange to loan you something for meetings.
1.2.4 Target Audience
Our goal is to make the DevOps BootCamp program accessible to students and community members from all different
backgrounds. You have to want to learn, be willing to ask questions whenever you donвЂ™t understand something, and
be open to making time to play with the cool tools/toys which weвЂ™ll be teaching you about.
Fluency in English is strongly recommended, because itвЂ™s the language in which the course will be taught as well as
the language of almost all open source technical documentation and code comments.
4
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.3 Curriculum
The curriculum will be constantly evolving as we develop this course. The first unit will be an overview of what
students will need to know in order to understand the deeper technical concepts which will be covered later.
HereвЂ™s what weвЂ™re hoping to mentor and teach students about:
1. Linux Basics
2. Basic System Administration
3. Basic FOSS Development Methodologies
4. Base infrastructure services for any organization (DNS, Email, etc)
5. Building a mock infrastructure for a mock company from top to bottom
Note: Note that the curriculum for future weeks is constantly under development. Lesson titles are included to
provide an overview of what weвЂ™ll cover, but specific content is likely to get moved around between lessons as we
refine the curriculum. Slide content will be linked here as soon as itвЂ™s drafted, but only finalized 1-2 weeks before the
lessons when itвЂ™s presented.
1.3.1 Lesson 1: The Very Basics
Shell, virtualbox+vagrant, IRC.
вЂў 6pm 11/7/2013, KEC1007. 34 people attended.
вЂў Lesson 1 Video
вЂў Lesson 1 Slides
1.3.2 Lesson 2: Single System Fundamentals
File permissions, users, groups, and package management
вЂў 6pm 11/14/2013, KEC1001. 27 people showed up.
вЂў Lesson 2 Video
вЂў Lesson 2 Slides
1.3.3 Lesson 3: Editors & Git
Vim, Emacs, and Git for version control
вЂў 6pm 11/21/2014, BEXL. 21 people showed up.
вЂў Part 1, text editors video
вЂў Part 2, version control video
вЂў Lesson 3 Slides
1.3. Curriculum
5
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.3.4 Lesson 4: Scripting & Troubleshooting
Python and Bash. Overview of troubleshooting/debugging skills.
вЂў 6pm 01/9/2014, KEC1007.
вЂў Scripting and Troubleshooting Video
вЂў Lesson 4 Slides
1.3.5 Lesson 5: Services & Deploying a Web App
вЂў 6pm 1/16/2014, KEC1007
вЂў Services & Web App video 1/2
вЂў Services & Web App video 2/2
вЂў Lesson 5 Slides
1.3.6 Lesson 6: Boot Process & Filesystem Hierarchy
вЂў 6pm 1/30/3014 вЂ“ 14 attendees; time conflicts with Intel and Reddit founder
вЂў Boot & Filesystem Video
вЂў Lesson 6 slides
1.3.7 Lesson 7: Databases
вЂў 6pm 2/13/2014 (moved from 2/6 due to snow)
вЂў Lesson 7 slides
1.3.8 Review of VM and App Setup
вЂў 6pm 2/20/2014
вЂў No video, hands-on... 14 people showed up
вЂў Review Slides
1.3.9 Lesson 8: Security & Authentication
вЂў 6pm 2/27/2014 вЂ“ 22 people present
вЂў Lesson 8 video
вЂў Lesson 8 slides
1.3.10 Lesson 9: Networking overview
вЂў 6pm 4/10/2014
вЂў Lesson 9 video
вЂў Lesson 9 slides
6
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.3.11 Lesson 10: DNS
вЂў 6pm 5/1/2014
вЂў Lesson 10 video
вЂў Lesson 10 slides
1.3.12 Lesson 11: Automation and DevOps
вЂў 6pm 5/8/2014
вЂў Lesson 11 video
вЂў Lesson 11 slides
1.3.13 Lesson 12: Configuration Management
вЂў 6pm 5/15/2014 вЂ“ 8 students, 10 people total
вЂў Lesson 12 video
вЂў Lesson 12 slides
1.3.14 Lesson 13: Open Source & Hands-On
вЂў 6pm 5/22/2014
вЂў Lesson 13 Video
вЂў Lesson 13 slides
1.4 BootCamp Guide
This is the DevOps BootCamp Guide. It will cover the various topics during the sessions in more detail.
вЂў Vagrant
1.4.1 Administrative Stuff
вЂў link for finding rooms
1.5 Recommended Resources
1.5.1 Scripting
вЂў The Advanced Bash Scripting Guide
вЂў Learn Python The Hard Way
1.4. BootCamp Guide
7
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.5.2 Systems Administration Tutorials
вЂў LinuxTraining
вЂ“ This is the resource which the OSL uses to quickly train systems administrators. The whole book is
here.
вЂў OpsSchool is working on a tutorial somewhat like DevOps BootCamp, but itвЂ™s currently incomplete. Consider
contributing to it!
вЂў Linux Command Line Tips: Become a Master is a quick start guide to get you going with bash
1.5.3 Open Source
The following resources are used as textbooks in Carlos JensenвЂ™s CS419 Open Source class:
вЂў The Cathedral and the Bazaar (Eric S. Raymond)
вЂў Producing Open Source Software (Karl Fogel)
вЂў Open Advice (Lydia Pintscher)
1.5.4 Culture
вЂў The Jargon File
вЂў ESRвЂ™s guide to asking good questions
вЂў Geek Feminism and Systers are worth looking into if you enjoy discussing diversity or ITвЂ™s lack thereof.
1.6 FAQ
Email questions to devopsbootcamp@osuosl.org if they arenвЂ™t answered here!
1.6.1 Is this a for-credit class?
No, although the curriculum is based on CS312, Linux System Administration, which hasnвЂ™t been offered for several
years.
1.6.2 What does it cost?
Financially, itвЂ™s free. However, to benefit from the hands-on learning opportunities that weвЂ™ll be offering, you should
plan to commit 5 or more hours per week to attending lectures and playing with the tools and technologies that the
lectures will discuss.
1.6.3 I donвЂ™t know about computers! Can I still join?
YES! If you already knew all this stuff, youвЂ™d be teaching rather than studying it. If you think you donвЂ™t know enough,
try anyway вЂ“ in the worst case youвЂ™ll discover that your self-assessment was correct, and in the best case youвЂ™ll become
a successful open source contributor.
8
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.6.4 When will meetings be?
Weekday evenings, with a day of the week being chosen during the first meeting. We try to choose a day that works
best for most students but itвЂ™s always difficult to find something that works for everyone. For example, last year we
had meetings on Thursdays.
1.6.5 Is this only for students?
No, anyone from the community can participate. Parts of the earlier lessons will assume access to student resources,
but we can work with non-students on an individual basis to get them access to equivalent tools. Community members
can use the Visitor wifi network when on campus.
1.6.6 WhatвЂ™s the difference between BootCamp and regular LUG meetings?
Regular LUG meetings are usually targeted toward an intermediate to expert audience, and make no attempt to comprehensively teach a set of skills across multiple meetings. We might teach about version control one week, privacy
tools the next, and a new open source project by a local company the next.
DevOps BootCamp will provide continuity and comprehensive open source education thatвЂ™s outside the scope of
regular LUG meetings. As you progress through Bootcamp, youвЂ™ll understand more and more of the technical content
of LUG talks.
1.6.7 I wasnвЂ™t able to participate during Fall term. Can I still join?
Yes! Please read through lessons 1-3 (linked from the curriculum page) and ask in IRC if you have trouble following
the instructions for them. Catch up on that content and youвЂ™ll be ready to join in Winter term.
1.6.8 I have a question?
Ask it on IRC if you have that set up, or email devopsbootcamp@osuosl.org or the mailing list.
1.7 What is DevOps?
DevOps (a portmanteau of software development and server operations) represents the integration between software
development and operations engineering that has become increasingly necessary with the emergence of cloud computing. In essense, the jobs are no longer mutually exclusive- developers need more operations-based knowledge in order
to understand how their application will be deployed, test locally and preemptively address potential security risks,
while systems administrators need to know developer-based materials in order to design infrastructure optimally to
fit an applicationвЂ™s needs without wasting capacity and to troubleshoot issues with the increasingly complex software
cycle. Additionally, site reliability engineering and many modern security roles require a background with a balance
between development and operations.
1.8 Purpose of DevOps BootCamp
1.8.1 From front page:
DevOps BootCamp is an OSU Open Source Lab and OSU Linux Users Group program aimed at boosting our impact
and outreach while shrinking the skills gap for OSU students interested in DevOps.
1.7. What is DevOps?
9
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.8.2 From FAQ:
DevOps BootCamp will provide continuity and comprehensive open source education thatвЂ™s outside the scope of
regular LUG meetings.
1.8.3 вЂњThe DealвЂќ
Students get:
вЂў Mentorship from students and professionals with advanced skills in software development and systems administration
вЂў Professional connections in the software industry
вЂў A welcoming environment to start learning in, if youвЂ™ve always wanted to learn about software development and
systems administration but never been sure where to start
вЂў An opportunity to fill in any gaps in your knowledge, if youвЂ™re a self-taught coder or sysadmin
вЂў The skills to build and deploy open source software, or contribute to existing projects
Why weвЂ™re offering this:
вЂў The OSL gets a larger pool of candidates to recommend to companies interested in recruiting OSL students
вЂў The OSL gets to work with a wider variety of students, which fits better with its new status as part of the school
of EECS
вЂў The open source community gets more contributors to its projects
1.8.4 Target Audience
Our goal is to make the DevOps BootCamp program accessible to students and community members from all different
backgrounds. You have to want to learn, and be willing to ask questions whenever you donвЂ™t understand something,
and be open to making time to play with the cool tools/toys which weвЂ™ll be teaching you about.
1.9 Vagrant
1.9.1 Trying Linux as a virtual machine with Vagrant
In order to create a stable learning environment, weвЂ™re going to have everyone use a Devop tool called Vagrant.
Vagrant is typically used to help assist both developers and system engineers in ensuring that their application and
system deployments work predictably. For our purposes weвЂ™re going to use it as an easy way for new people to get to
a Linux prompt quickly with no fear of breaking their system.
Vagrant basically interfaces with hypervisors such as VirtualBox with a unified command line interface. This makes
it portable between Mac, Windows and even Linux! Vagrant itself is just a collection of ruby scripts while Virtualbox
does all the virtual machine magic.
We will go in more depth with how to install Linux manually, but for now weвЂ™ve done all the hard work and have
create pre-made Linux images.
10
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.9.2 Installing Virtualbox
VirtualBox is a free and open source virtual machine manager that you can install on a variety of operating system
platforms such as Mac and Windows.
1. Go to the VirtualBox download page and download the latest copy of VirtualBox.
2. Go through the installer and use the default settings when prompted.
3. Once completed, see if you can open up the VirtualBox Manager
4. If it asks you to download the latest extensions, go ahead and do that.
1.9.3 Installing Vagrant
Vagrant is a free and open source tool used to build complete devops environments.
1. Go to the Vagrant download page and choose the newest version available.
2. Choose all the defaults during the install if it asks you any questions
1.9.4 Testing Vagrant
1. If youвЂ™re on a Mac, open up the Terminal. If youвЂ™re on windows, open up a DOS prompt (cmd.exe).
2. Type: vagrant status
3. If there are no errors then Vagrant was installed correctly!
1.9.5 Cloning the Vagrant Repo
We keep all of our vagrant configuration in a git version controlled repository. If you donвЂ™t have git installed, please
go download git and install it.
Once you have downloaded and installed git, go to a folder where you want to keep the repository and type the
following:
git clone https://github.com/DevOpsBootcamp/devopsbootcamp-vagrant.git
If installing git is too difficult, you can also download a zip file containing the repository. We will teach you more
about git later in the year, so donвЂ™t worry!
1.9.6 Using Vagrant
Now that we have the entire environment working, lets get to playing with Linux! Open the terminal and get to the
directory where the devopsbootcamp-vagrant repo is at. You can run the following commands:
# Start the VM
vagrant up
# SSH into the VM
vagrant ssh
# Stop and halt the VM
vagrant halt
1.9. Vagrant
11
OSU DevOps Bootcamp Documentation, Release 0.0.1
# Destroy and remove the VM
vagrant destroy
Also check out the Vagrant Documentation for more information. You can also always type -h to find out more
information about a command.
1.9.7 Troubleshooting
Depending on how your laptop or computer is confused in the BIOS, you may or may not run into issues getting
Vagrant and VirtualBox to work properly.
Booting the VM takes forever and never completes
The most common problem is when you type vagrant up the system just waits and waits and never finishes the
boot process. The most likely cause is because your system doesnвЂ™t have hardware virtualization enabled in the BIOS.
You can either enable the feature or you can disable the requirement in the Vagrantfile.
1. Enabling Hardware Virtualization
Reboot your machine and press the appropriate buttons to get into the BIOS or Setup screen. This is usually done by
hitting a combination of the escape or function keys.
Once you get into the BIOS, find a screen that has some options for the CPU. Each BIOS is different but you are
basically trying to find a feature called Hardware Virtualization. Once you find it, go ahead and enable it and reboot
your system.
2. Disabling HW Virtualization in Vagrant
Open the Vagrantfile with your favorite editor of choice. Find the line that says hwvirtex and remove the #
from the front of the line. Now run vagrant destroy -f and then vagrant up again. This should boot the
VM properly now.
1.10 DevOps Bootcamp Introduction
OSU OSL & OSU LUG
http://devopsbootcamp.osuosl.org
1.10.1 WhatвЂ™s DevOps?
Software Development + Operations Engineering
1.10.2 WhatвЂ™s BootCamp?
вЂў New this year
вЂў New this year
вЂў Free education program
вЂ“ Mentorship
вЂ“ Apprenticeship
12
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂ“ Hands-on training
вЂў NOT:
вЂ“ OSU class for credit
вЂ“ student job
1.10.3 Why are we doing this?
вЂў OSLвЂ™s move to EECS
вЂў Bridge the вЂњSkills GapвЂќ
вЂў Demand from companies for students
вЂў Build community
1.10.4 What youвЂ™ll do:
вЂў Learn:
вЂ“ Linux systems
вЂ“ Networking
вЂ“ Software development
вЂ“ ...and the other skills youвЂ™ll need
вЂў Build:
1.10. DevOps Bootcamp Introduction
13
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂ“ Duplicate internet infrastructure
вЂ“ Graduate to helping with OSL tasks
* work with real open source projects
* solve real problems
1.10.5 Can you do it?
вЂў Probably!
вЂў No background knowledge needed
вЂў Time commitment
вЂ“ Attend lectures
вЂ“ Play with what we teach you
вЂ“ You get out what you put in
вЂў If in doubt. . . try anyway!
1.10.6 Career Paths:
вЂў Hands-on learning & OSL reputation:
вЂ“ Internships
вЂ“ Industry connections
вЂ“ Start career with mid-level/high-level experience
вЂў Justin
вЂ“ community experience builds skills, professional relationships
вЂ“ Linux skills helped in grad school
* skills learned there helped land first sysadmin job
вЂ“ having your own вЂ�playgroundвЂ™ to experiment with
1.10.7 OSL Hiring:
вЂў Students need basic skills to join
вЂ“ Systems engineers
вЂ“ Developers
вЂ“ Media
вЂў Typically 2-3 years employment
вЂў Alumni
вЂ“ Google, Rackspace, Intel, Microsoft, Mozilla, Facebook
14
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.10.8 What next?
вЂў Find our site (http://devopsbootcamp.osuosl.org)
вЂў Fill out registration with times available
вЂў Join mailing list
вЂў Tell us about yourself
вЂ“ Who are you
вЂ“ What do you want to learn here
вЂ“ Any background in devops?
1.11 Lesson 1: The Very Basics
1.11.1 TodayвЂ™s Agenda
вЂў Getting (to) Linux
вЂў The Terminal & Shell
вЂ“ Scripts, file paths, special characters
вЂў Productivity tricks
вЂ“ Getting help
1.11. Lesson 1: The Very Basics
15
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂў IRC
вЂ“ Vocabulary
вЂ“ Get connected
вЂ“ Etiquette
1.11.2 A note about notation
вЂў Variables
вЂ“ $varname
вЂ“ <varname>
вЂў Shell prompt
вЂ“ $
вЂ“ вЂ�literal stuff in backticksвЂ�
вЂў foo, bar, baz, username, etc.
1.11.3 How to get (to) Linux
вЂў How many have it already installed?
вЂў Install VM or dual-boot
вЂў When stuck on Windows, use PuTTy:
вЂў Students:
ssh <onidusername>@shell.onid.oregonstate.edu
вЂў flip{1-3} are Engineering servers; less reliable
1.11.4 Trying Linux on a Virtual Machine
Virtual machines act as a full system on a physical machine
вЂў Hypervisors:
вЂ“ VirtualBox (free)
16
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.11. Lesson 1: The Very Basics
17
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂ“ VMWare (mostly free)
вЂ“ KVM (Linux only hosts)
вЂ“ Parallels
вЂў Public Cloud Virtual Machines
вЂ“ Amazon EC2, Rackspace Cloud, Google Compute Engine, etc
вЂў Easy way to test without breaking your machine!
1.11.5 Installing Linux on Virtualbox
Note: Try other distributions if you like to see whatвЂ™s different. Debian is a great next step to try out.
1. Download and install: https://www.virtualbox.org/wiki/Downloads
2. Grab the latest minimal ISO: http://centos.osuosl.org/6/isos/x86_64/
3. Create VM
(a) New -> Name вЂњCentOSвЂќ -> Default Ram -> Default Disk settings
(b) Settings -> Storage -> Empty -> CD/DVD Drive -> Select ISO
(c) Start -> press enter -> Skip media check
4. \o/
1.11.6 Vagrant & VirtualBox
Note: WeвЂ™re using CentOS as our base image for now but will use Debian later. You can see the gui by uncommenting
the line in the Vagrantfile.
вЂў Vagrant is a tool used with Virtualbox (and other) platforms
вЂў Make a reproducible pre-installed Linux environment
вЂў Download and install: http://www.vagrantup.com/
вЂў Clone our repo, start and access the vm:
# clone
git clone https://github.com/DevOpsBootcamp/devopsbootcamp-vagrant.git
# start up
cd devopsbootcamp-vagrant
vagrant up
# access vm
vagrant ssh
1.11.7 Vagrant cheat sheet
Note: WeвЂ™ll get into more detail later in how you can access ports on your VMs and other use cases.
18
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
# start
vagrant up
# stop
vagrant halt
# destroy (remove vm)
vagrant destroy
# ssh to the vm
vagrant ssh
Also check out the Vagrant Documentation for more information.
1.11.8 The Terminal
вЂў Used to mean the keyboard+monitor
вЂ“ Now thatвЂ™s a crash cart
вЂў Terminal emulator
вЂў Shell: Use bash; others include csh, zsh, tsch
вЂ“ ~/.bashrc
1.11.9 Basic Shell Commands
Note:
Explain architecture built in commands vs. external binaries
Demo commands Directory movement and file manipulation: Cd, pwd, ls, rm, mv, touch
User info id, whoami, w
1.11. Lesson 1: The Very Basics
19
OSU DevOps Bootcamp Documentation, Release 0.0.1
20
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
Pipes redirection (pipe.txt, redirect.txt)
Special variables $?, $$ (pid.sh), !!, !*, !$
вЂў ls, cd, cat, echo
вЂў invoke/call an installed program
вЂў get help: man <program>
test@x230 ~ $ tree
.
-- Documents
|
-- Code
|
|
-- scripts
|
|
-- test.sh
|
-- School
|
-- Work
-- Pictures
-- manatee.gif
-- turtle.png
6 directories, 5 files
1.11.10 Invoking a script
Note: Permissions discussed later.
$ ls -l
$ chmod +x $filename
Arguments are extra information that you pass to a script or program when you call it. They tell it in more detail what
you want to do.
$ ls -a -l
$ ls -al
1.11. Lesson 1: The Very Basics
21
OSU DevOps Bootcamp Documentation, Release 0.0.1
Why pass arguments on the command line rather than having an interactive mode?
1.11.11 File Paths
вЂў . means current directory
вЂў .. means parent directory
вЂў Tilde (~) means your homedir (/home/$username)
вЂў / separates directories (not \)
вЂў / is root directory, so ~ expands to /home/$username/
вЂў current path appears in your prompt: IвЂ™m logged in as the user test on the machine named x230
test@x230 ~ $ ls
Documents Pictures
test@x230 ~ $ cd Documents/
test@x230 ~/Documents $ ls
Code School Work
test@x230 ~/Documents $
Note: root directory is not to be confused with a home directory for the root account
1.11.12 Special Characters
вЂў escape with \ to use them literally
вЂў # means a comment
вЂў ; allows multiple commands per line
вЂў !, ?, *, &&, &
вЂў Regular expressions (weвЂ™ll learn more later)
22
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.11.13 Type less
вЂў Reverse-i-search
вЂ“ ctrl+r then type command
вЂў aliases
вЂ“ ~/.bashrc
вЂў Tab completion
Automation > Typing > Mouse
1.11.14 Help, get me out of here!
вЂў ctrl+c kills/quits
вЂў ctrl+d sends EOF (end-of-file)
вЂ“ also means logout
вЂў :q gets you out of Vi derivatives and man pages
вЂ“ esc - esc - :q if you changed modes
вЂў read whatвЂ™s on your screen; itвЂ™ll help you
1.11.15 Knowledge Check
1.11. Lesson 1: The Very Basics
23
OSU DevOps Bootcamp Documentation, Release 0.0.1
test@x230 ~ $ tree
.
-- Documents
|
-- Code
|
|
-- scripts
|
|
-- test.sh
|
-- School
|
-- Work
-- Pictures
-- manatee.gif
-- turtle.png
6 directories, 5 files
вЂў What user am I logged in as?
вЂў What command did I just run?
вЂў What is my current directory when I run that command?
1.11.16 More about Man Pages
вЂў the manual (rtfm):
$ man <program>
$ man man
вЂў use /phrase to search for phrase in the document; n for next match
вЂў else:
$ <program> --help
1.11.17 Documentation
Man pages, blogs you find by Googling, StackOverflow
вЂў Contribute to community
вЂ“ Correct it if itвЂ™s wrong
вЂ“ Remind them what newbies donвЂ™t know
вЂ“ Write your own
вЂў For your future self as well
вЂў Start now
1.11.18 Asking for help
ItвЂ™s okay to ask.
1. What should be happening?
2. WhatвЂ™s actually happening?
3. Google it
24
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
4. Skim the manuals of each component
5. Identify a friend, mentor, or IRC channel who could help
6. When theyвЂ™re not busy, give them a quick synopsis of points 1 and 2, mentioning what possibilities youвЂ™ve ruled
out by searching.
Contributions = expertise + time
DonвЂ™t waste expertsвЂ™ time, but do build your expertise.
1.11.19 IRC
вЂў Internet Relay Chat
вЂў Very old (RFC 1459 May 1993)
вЂў Works on everything (no GUI needed)
вЂў The people you want to listen to are there
1.11.20 A Client
Note: Switche to a terminal and show example
Use irssi in screen
# This step is optional, but persistent IRC is cool
$ ssh <username>@<preferred shell host>
# start Screen
$ screen -S irc
# start your client
$ irssi
# after ending ssh session, to get back:
$ ssh <username>@<preferred shell host>
$ screen -dr IRC
1.11.21 Networks
/connect irc.freenode.net
/nick <myawesomenickname>
/msg nickserv register <password> <email>
/nick <myawesomenickname>
/msg nickserv identify <password>
1.11. Lesson 1: The Very Basics
25
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.11.22 Channels
/join #osu-lug
/join #devopsbootcamp
/list
вЂў tells all channels on network
вЂў DonвЂ™t do this on Freenode!
/topic tells you the current channelвЂ™s topic
/names tells you whoвЂ™s here
1.11.23 Commands
вЂў take action with /me does thingвЂ�
вЂў everything else starting with / is a command
/say $thing
/join, /part, /whois <nick>, /msg, /help <command>
Note that nothing shows up in the channel when you run a /whois command; it shows up either in your status buffer
or your conversation with the person.
12:04 -!- _test_ [~test@c-50-137-46-63.hsd1.or.comcast.net]
12:04 -!- ircname : Example User
12:04 -!- channels : #ExampleChannel
12:04 -!- server
: moorcock.freenode.net [TX, USA]
12:04 -!- hostname : c-50-137-46-63.hsd1.or.comcast.net 50.137.46.63
12:04 -!- idle
: 0 days 0 hours 2 mins 38 secs [signon: Wed Nov 6
12:00:30
2013]
12:04 -!- End of WHOIS
1.11.24 Useful tricks
вЂў Tab-complete works on nicknames. use it.
вЂў Highlight when people say your name
вЂў Symbols are not part of names; they mark status in channel
вЂў Logging (expect it); вЂ�/set autolog onвЂ�
вЂў chanserv and nickserv are good bots to know
вЂ“ hamper is also a bot
1.11.25 Screen & Irssi Hints
вЂў Paste with ctrl+shift+v
вЂ“ PuTTY defaults to right-click to paste
вЂў to get back, screen -dr IRC
вЂў Can you use man screen to find out what the d and r flags mean?
26
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
SCREEN(1)
SCREEN(1)
NAME
screen - screen manager with VT100/ANSI terminal emulation
SYNOPSIS
screen [ -options ] [ cmd [ args ] ]
screen -r [[pid.]tty[.host]]
screen -r sessionowner/[[pid.]tty[.host]]
Manual page screen(1) line 1 (press h for help or q to quit)
1.11.26 Etiquette
вЂў Lurk more
вЂў DonвЂ™t ask to ask
вЂ“ Lure help out of hiding with tasty details of problem
вЂў Show that youвЂ™re worth helping
вЂў Read the topic
вЂ“ /topic
вЂ“ Output only shows up in your channel, not to everyone else
вЂў Pastebin code
вЂў Choose your nick carefully
1.11.27 Terminology
вЂў ping/pong
вЂў flapping
вЂў tail
вЂў hat
вЂў nick
вЂў netsplit
вЂў kick/ban/k-line
вЂў common emotes
1.11. Lesson 1: The Very Basics
27
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂ“ o/ AND \o high fives
вЂ“ /me & means afk
1.11.28 Review
вЂў WhatвЂ™s Linux?
вЂў How do you open a terminal emulator?
вЂ“ this varies between window managers
вЂў I have the script test.py. How do I run it??
вЂў How do you list all the files in the current directory?
вЂў Give 2 ways to change directory to your home directory.
вЂў How do you start an irc client?
вЂ“ How often should you need to start your IRC client?
вЂў How do you reconnect to a screen session?
вЂў Give an example of something which you should not do in IRC
1.12 Lesson 2: Single System Fundamentals
Or, how to be a power user.
1.12.1 Housekeeping
10 minutes to make sure everyoneвЂ™s Virtualbox instances are up and running
Your opinions:
вЂў Want to meet during dead week vs. have last meeting of term next week?
вЂў Start wk1 of winter term?
28
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.12.2 TodayвЂ™s Topics
вЂў What are files?
вЂ“ Permissions/users
вЂў What are user accounts?
вЂ“ User management
вЂ“ Root vs. normal
вЂ“ Groups
вЂў What are packages?
вЂ“ How to use package managers
вЂ“ Differences between Linux Distributions
вЂ“ Other package managers
1.12.3 What are users?
вЂў You, right now
$
$
$
$
whoami
who
w
id
#
#
#
#
your username
who is logged in?
who is here and what are they doing?
user ID, group ID, and groups youвЂ™re in
вЂў Not just people: Apache, Mailman, ntp
1.12.4 Users have
вЂў Username
вЂў UID
вЂў Group
вЂў Usually (but not always) password
вЂў Usually (but not always) home directory
1.12.5 Managing users
$
#
$
$
$
cat /etc/passwd
username:x:UID:GID:GECOS:homedir:shell
useradd $USER # vs adduser, the friendly Ubuntu version
userdel $USER
passwd
#
#
$
$
GECOS: full name, office number and building, office phone extension,
home phone number (General Electric Comprehensive Operating System)
chfn # change GECOS information; only works sometimes
finger # tells you someoneвЂ™s GECOS info
1.12. Lesson 2: Single System Fundamentals
29
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.12.6 Passwords
вЂў /etc/shadow, not /etc/passwd
test@x230 ~ $ ls -l /etc/ | grep shadow
-rw-r----- 1 root shadow
1503 Nov 12 17:37 shadow
$ sudo su $ cat /etc/shadow
daemon:*:15630:0:99999:7:::
bin:*:15630:0:99999:7:::
sys:*:15630:0:99999:7:::
mail:*:15630:0:99999:7:::
# name:hash:time last changed: min days between changes: max days
#
between changes:days to wait before expiry or disabling:day of
#
account expiry
$ chage # change when a userвЂ™s password expires
1.12.7 Root/Superuser
вЂў UID 0
вЂў sudo
1.12.8 Acting as another user
$
$
$
$
su $USER
su
sudo su sudo su $USER
#
#
#
#
become user, with
become root, with
use user password
become $USER with
THEIR password
rootвЂ™s password
instead of rootвЂ™s
your password
If someone has permissions errors:
вЂў Check that they or their group owns the files
вЂў Check that they have the flag +x to execute
1.12.9 What are groups?
вЂў Manage permissions for groups of users
30
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.12. Lesson 2: Single System Fundamentals
31
OSU DevOps Bootcamp Documentation, Release 0.0.1
$
$
$
$
groupadd
usermod
groupmod
cat /etc/group
root:x:0:
daemon:x:1:
bin:x:2:
sys:x:3:
adm:x:4:
tty:x:5:
# group name:password or placeholder:GID:member,member,member
1.12.10 Hands-On: Users and Groups
Note: To give yourself sudo powers do the following:
1. Add your user to the wheel group using gpasswd.
2. As the root user, use visudo and uncomment this line:
%wheel
ALL=(ALL)
ALL
3. Save the file and now you should have sudo!
WeвЂ™ll cover sudo in more depth at a later time.
вЂў Create a user on your system for yourself, with your preferred username
вЂў Give your user sudo powers
вЂў Use use su to get into your user account
вЂў Change your password
вЂў Create a directory called bootcamp in your home directory
вЂў Create a group called devops
1.12.11 What are files?
вЂў Nearly everything
вЂў Files have:
вЂ“ Owner
вЂ“ Permissions
вЂ“ inode
вЂ“ Size
вЂ“ Filename
test@x230 ~ $ ls -il
total 8
2884381 drwxrwxr-x 5 test test 4096 Nov 6 11:46 Documents
2629156 -rw-rw-r-- 1 test test
0 Nov 13 14:09 file.txt
2884382 drwxrwxr-x 2 test test 4096 Nov 6 13:22 Pictures
32
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.12.12 File extensions
вЂў .jpg, .txt, .doc
вЂў Really more of a recommendation
вЂ“ File contains information about its encoding
$ file $FILENAME # tells you about the filetype
test@x230 ~ $ file file.txt
file.txt: ASCII text
test@x230 ~ $ file squirrel.jpg
squirrel.jpg: JPEG image data, JFIF standard 1.01
1.12.13 ls -l
вЂў First bit: type
вЂў Next 3: user
вЂў Next 3: group
вЂў Next 3: world
вЂў user & group
$ ls -l
drwxrwxr-x 5 test test 4096 Nov 6 11:46 Documents
-rw-rw-r-- 1 test test
0 Nov 13 14:09 file.txt
drwxrwxr-x 2 test test 4096 Nov 6 13:22 Pictures
chmod and octal permissions
+=====+========+=======+
| rwx | Binary | Octal |
+=====+========+=======+
| --- | 000
| 0
|
| --x | 001
| 1
|
| -w- | 010
| 2
|
| -wx | 011
| 3
|
| r-- | 100
| 4
|
| r-x | 101
| 5
|
| rw- | 110
| 6
|
| rwx | 111
| 7
|
+=====+========+=======+
вЂў u, g, o for user, group, other
вЂў -, +, = for remove, add, set
вЂў r, w, x for read, write, execute
chown, chgrp
user & group
1.12. Lesson 2: Single System Fundamentals
33
OSU DevOps Bootcamp Documentation, Release 0.0.1
# Change the owner of myfile to "root".
$ chown root myfile
# Likewise, but also change its group to "staff".
$ chown root:staff myfile
# Change the owner of /mydir and subfiles to "root".
$ chown -hR root /mydir
# Make the group devops own the bootcamp dir
$ chgrp -R devops /home/$yourusername/bootcamp
1.12.14 Types of files
drwxrwxr-x
5 test
test
4096
Nov 6 11:46 Documents
-rw-rw-r-1 test
test
0
Nov 13 14:09 file.txt
drwxrwxr-x
2 test
test
4096
Nov 6 13:22 Pictures
---------------- ------- -------- ------------ ------------|
|
|
|
|
|
|
|
|
|
|
File Name
|
|
|
|
+--- Modification Time
|
|
|
+------------Size (in bytes)
|
|
+----------------------Group
|
+-------------------------------Owner
+---------------------------------------------File Permissions
- is a normal file
d is a directory
b is a block device
1.12.15 ACLs
вЂў Access control lists
вЂў Not recommended; hard to maintain
вЂў Typically how other OSes manage permissions
вЂў Support depends on OS and filesystem
1.12.16 Hands-On: Files and Permissions
$ touch foo # create empty file called foo
вЂў As root, create a file in /home/$yourusername/bootcamp
вЂў Who can do what to the file?
вЂў Make the devops group own the file
вЂў Make a file called allperms and give user, group, and world +rwx
вЂў Make more files and practice changing their permissions
34
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.12.17 Package Management
Take care of installation and removal of software
Core Functionality:
вЂў Install, Upgrade & uninstall packages easily
вЂў Resolve package dependencies
вЂў Install packages from a central repository
вЂў Search for information on installed packages and files
вЂў Pre-built binaries (usually)
Popular Linux Package Managers
вЂў .deb / APT (used by Debian, Ubuntu)
вЂў .rpm / YUM (used by RedHat, CentOS, Fedora)
1.12.18 RPM & yum (RedHat, CentOS, Fedora)
RPM
Binary file format which includes metadata about the package and the application binaries as well.
Yum
RPM package manager used to query a central repository and resolve RPM package dependencies.
Yum Commands (Redhat, CentOS, Fedora)
WeвЂ™ll use the tree package as an example below.
# Searching for a package
$ yum search tree
# Information about a package
$ yum info tree
1.12. Lesson 2: Single System Fundamentals
35
OSU DevOps Bootcamp Documentation, Release 0.0.1
# Installing a package
$ yum install tree
# Upgrade all packages to a newer version
$ yum upgrade
# Uninstalling a package
$ yum remove tree
# Cleaning the RPM database
$ yum clean all
RPM Commands
Low level package management. No dependency checking or central repository.
# Install an RPM file
$ rpm -i tree-1.5.3-2.el6.x86_64.rpm
# Upgrade an RPM file
$ rpm -Uvh tree-1.5.3-3.el6.x86_64.rpm
# Uninstall an RPM package
$ rpm -e tree
# Querying the RPM database
$ rpm -qa tree
# Listing all files in an RPM package
$ rpm -ql tree
1.12.19 DPKG & Apt (Debian, Ubuntu)
Deb
Binary file format which includes metadata about the package and the application binaries as well.
DPKG
Low level package installer for the .deb file format. Does no package dependency resolution.
Apt
DPKG package manager used to query a central repository and resolve Deb package dependencies. Considered mostly a front-end to dpkg.
36
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
Apt Commands (Debian, Ubuntu)
Note: You can also use aptitude as a front-end to dpkg instead of apt-get.
# Update package cache database
$ apt-get update
# Searching for a package
$ apt-cache search tree
# Information about a package
$ apt-cache show tree
# Installing a package
$ apt-get install tree
# Upgrade all packages to a newer version
$ apt-get upgrade
$ apt-get dist-upgrade
# Uninstalling a package
$ apt-get remove tree
$ apt-get purge tree
Dpkg Commands
Low level package management. No dependency checking or central repository.
# Install or upgrade a DEB file
$ dpkg -i tree_1.6.0-1_amd64.deb
# Removing a DEB package
$ dpkg -r tree
# Purging a DEB package
$ dpkg -P tree
# Querying the DPKG database
$ dpkg-query -l tree
# Listing all files in a DEB package
$ dpkg-query -L tree
1.12.20 Language-specific Package Managers
вЂў Languages sometimes have their own package management suite
вЂў Can be useful for using newer versions of packages
вЂў Examples
вЂ“ pip (Python)
вЂ“ rubygems (Ruby)
вЂ“ CPAN (Perl)
1.12. Lesson 2: Single System Fundamentals
37
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂ“ cabal (Haskell)
вЂ“ npm (NodeJS)
вЂ“ ... and so on forever ...
1.12.21 Other Package Managers
They each fill a specific niche and have their own pros and cons.
вЂў Portage (Gentoo) вЂ“ Source based package installer
вЂў pacman (Arch Linux)
вЂў ZYpp / Zypper (SUSE) вЂ“ Yet another RPM package manager
1.12.22 Installing from source
вЂў Download source tarball, run build scripts and install in a local directory.
вЂў RPM/DEB packages do this for you
вЂў Not for the faint of heart ... Not recommended!
вЂў Using grep as an example
$
$
$
$
$
$
wget http://mirrors.kernel.org/gnu/grep/grep-2.15.tar.xz
tar -Jxvf grep-2.15.tar.xz
cd grep-2.15
./configure
make
make install
1.12.23 Hands-on: Package Management
вЂў Install the git package
вЂў Query the RPM/APT database for installed packages
вЂў List the files in an installed package
вЂў Remove the git package
1.12.24 Questions:
вЂў read example output of ls -al
вЂў read output of yum or aptitude search
вЂў install a package on their VM/partition (Vim, Git)
вЂ“ explain what dependencies it also installed
38
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.13 Lesson 2: Homework
1. Why is a salt used when storing encrypted passwords in /etc/shadow?
2. What portion of the MD5 hash вЂ™$1$xxUwcovy$JfV9i7j9H/NFA3RBCrVHN.вЂ™ is the salt?
3. What UID is used for the root user?
4. Where is a userвЂ™s primary (вЂњdefaultвЂќ) group defined? Specifically which file and which вЂњfieldвЂќ.
5. Add a user foobar to your system. Use useradd to add the user and to create their home directory containing
files from /etc/skel. Show the userвЂ™s entry in /etc/passwd as well as the full useradd command
needed.
6. Create a group bootcamp on your system. Show the command used (not editing files by hand).
7. Assume the user foobar belongs to multiple groups. Add foobar to the bootcamp group by using a system
command (not editing files by hand) without changing any of their other groups. Show the command used.
8. What chmod command (using octal mode) would you use to allow owner read and write access and group read
access (and no other permissions!) to a file foo? Using chmod without octal mode, how would you do the
same?
9. What does it mean for a binary to be setuid? is setuid a potential security risk? Why is this important for tools
such as passwd?
10. You have the following foo directory
drwxr-xr-x 7 lance bootcamp 4096 Mar 31 09:15 foo
What chmod command can you run to ensure files created inside that directory will default to having
bootcamp group ownership? Assume the user creating the files is in the bootcamp group.
1.14 Lesson 3: Editors & Version Control
1.14.1 Check in from last week
Is anyone having problems with package management?
Install the packages named git, vim, and emacs
1.14.2 What weвЂ™ll discuss
вЂў WhatвЂ™s a text editor?
вЂў Features of an editor
вЂў Version control: Git!
1.14.3 How do you edit files?
1.13. Lesson 2: Homework
39
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.14.4 No. Never, Ever edit code in Word/LibreOffice
вЂў Autocorrect is your worst enemy
вЂў Syntax highlighting is nice
вЂў You are editing plain text, not вЂ�documentsвЂ™, i.e. .doc, .rtf, etc
1.14.5 Then what should you use?
вЂў Text editors
вЂ“ Command-line
вЂ“ available in your package manager
вЂў For coding, an IDE can be helpful
1.14.6 Integrated Development Environments
вЂў Entire toolchain in one place
вЂў Usually graphical
1.14.7 Sysadmin tools
вЂў Frequently work with remote machines over a ssh connection
вЂ“ xforwarding can sort of give you a GUI
вЂ“ Not all remote machines have a desktop environment
вЂў Faster to edit file in place than transfer it down then back up
вЂ“ Frequent small changes when testing or debugging
вЂў Broken machines often have only terminal via crash cart
Editor Requirements
вЂў Small program, runs in terminal
вЂў Preferably installed by default on most systems
1.14.8 Features of an editor
вЂў Needs to be installed on your system
вЂў Create files
вЂў Move around within the file
вЂў Add or delete text
вЂў Easily automate tedious tasks
40
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.14.9 Text Editors
Note: Quick intro/history: ed editor Pros: low-bandwidth, installed pretty much everywhere, very fast and powerful
for complicated and repetitive tasks Cons: Steep learning curve, different вЂњmodesвЂќ can be confusing at first Sublime
and other desktop editors: nice for serious programming, but learn the basics of simple text editors even if you want
to be a developer, you wonвЂ™t always be able to edit your code on your own desktop
вЂў ed -> Vi -> Vim
вЂў Stallman -> lisp -> emacs
Avoid Pico/Nano, Notepad++, SublimeText
1.14.10 Emacs
1.14. Lesson 3: Editors & Version Control
41
OSU DevOps Bootcamp Documentation, Release 0.0.1
Note: Originally released 1976 name from Editor MACros for TECO editor, originally Tape Editor and COrrector at
MIT in the вЂ�60s
But, along the way, I wrote a text editor, Emacs. The interesting idea about Emacs was that it had a programming
language, and the userвЂ™s editing commands would be written in that interpreted programming language, so that you
could load new commands into your editor while you were editing. You could edit the programs you were using and
then go on editing with them.
вЂ“ Richard Stallman, http://www.gnu.org/gnu/rms-lisp.html
1.14.11 Vim
Note: originally written for Amiga systems (Commodore PCs), 1988 vim released 1991 vimscript, Lua (as of Vim
7.3), Perl, Python, Racket, Ruby, Tcl (tool command language). vi written by Bill Joy in 1976, visual mode for line
editor called ex line editors are from age of teleprinters, no cursors
вЂў Available almost everywhere
вЂў Lightweight
вЂў Design decisions explained in http://docs.freebsd.org/44doc/usd/12.vi/paper.html
вЂў Modal editor (command, insert, visual)
1.14.12 How to choose
вЂў What can the people around you help with?
вЂў Try both; choose one and get good at it
вЂў Have a good answer when people ask why you made that choice
вЂ“ вЂњBecause itвЂ™s familiarвЂќ is tolerated
вЂ“ вЂњBecause I was initially taught itвЂќ is common but accepted (honesty)
42
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂ“ вЂњBecause $usecaseвЂќ provokes argument but more respected
вЂ“ вЂњBecause I tried both and picked this oneвЂќ is rare but good
вЂў Your use case as a sysadmin or developer
1.14.13 Modes
How to tell?
-- INSERT --- VISUAL --
144,1
144,77
36%
36%
1.14.14 Commands
Note: Moving around in a file Search / replace Text manipulation, ie: cw, dw, c$, yy / p, x, .
1.14.15 Moving Around
h
j
k
l
0
move
move
move
move
move
one character to the left.
down one line.
up one line.
one character to the right.
to the beginning of the line.
1.14. Lesson 3: Editors & Version Control
43
OSU DevOps Bootcamp Documentation, Release 0.0.1
$ move to the end of the line.
w move forward one word.
b move backward one word.
G move to the end of the file.
gg move to the beginning of the file.
. move to the last edit.
1.14.16 Configuration / customization
Note: there are many many options and pre-existing packages to make editing nice for sysadmins and developers
вЂў .vimrc
вЂў :set
Some sets of Vim plugins and configurations are available
вЂў https://github.com/astrails/dotvim
вЂў https://github.com/carlhuda/janus
Use them for research on whatвЂ™s available to improve dev productivity
1.14.17 Learning Resources
вЂў $ vimtutor
вЂў http://vim-adventures.com/
1.14.18 Regular expressions
You should know basic substitution:
:%s/foo/bar/g
On IRC, Hamper does rudimentary regex in the form s/foo/bar/ applying only to the most recent comment.
This is not shell globbing
Resources for learning
вЂў RegExr - an interactive Regular Expression editor and debugger
вЂў Regular-Expressions.info - Tutorials and general information
1.14.19 Editor questions?
вЂў Open an editor, find a cheat sheet, try to add some text
вЂў Modify the text: вЂњdisemvowelвЂќ it
$ vim testvim.txt
<i>
Hello world!
<esc>
:%s/[aeiou]//g
44
$ emacs testemacs.txt
Hello world!
<esc>
<
<alt + x>
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.14. Lesson 3: Editors & Version Control
45
OSU DevOps Bootcamp Documentation, Release 0.0.1
:wq
replace-regexp
[aeiou]
<enter>
<ctrl + x> <ctrl + s>
<ctrl + x> <ctrl + c>
1.15 Lesson 3: Intro to Git
1.15.1 Version Control is Hard
1.15.2 Why Bother?
1.15.3 Better Options: Version Control
Note: Collaboration with multiple developers is important to mention
вЂў Commit = Snapshot of part of your projectвЂ™s state
вЂў Centralized (SVN, CVS) vs. Decentralized (Git, hg)
вЂў WeвЂ™ll look at Git today
вЂ“ Easier to learn other VCS from Git
вЂ“ Widely used in the open source world
1.15.4 Git
git noun. Brit.informal. 1. an unpleasant or contemptible person.
1.15.5 Using Git Locally
# This initializes a git repo. Use вЂ�man git-initвЂ� for more info.
$ git init
# This puts <filename> into the staging area. It isnвЂ™t committed yet.
# вЂ�git diffвЂ� to see what changes arenвЂ™t yet in staging.
$ git add <filename>
Use
# This actually makes the commit. Use вЂ�git statusвЂ� to see whatвЂ™s in staging
# but not yet committed. Use вЂ�git showвЂ� or вЂ�git logвЂ� to see recent commits.
$ git commit -m "I did a thing!"
вЂў Undo things? the git book explains well
вЂў Did I remember to commit that? $ git status
вЂў What commits have I made lately? $ git log
46
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
Figure 1.1: Image from XKCD
Figure 1.2: Image from PhD Comics
1.15. Lesson 3: Intro to Git
47
OSU DevOps Bootcamp Documentation, Release 0.0.1
48
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.15.6 More on commits
Your work goes from unstaged to staging area with git add.
$ git config --global user.name вЂ™Your NameвЂ™
$ git config --global user.email you@somedomain.com
вЂў Everything in staging gets wrapped up into an object that contains
вЂ“ changes
вЂ“ timestamp
вЂ“ author info
вЂ“ parent commit hash
вЂў These live in .git/ in your project directory
вЂў Commits go to other locations with git push
1.15.7 What Not To Do
Figure 1.3:
github/
image from http://arstechnica.com/security/2013/01/psa-dont-upload-your-important-passwords-to-
вЂў DonвЂ™t delete the .git files
Note: If you kill them, git loses its memory :(
вЂў Redundant copies of same work
вЂў вЂњoops, undoing thatвЂќ commits.
вЂ“ Use git commit --amend
Note: Amending is fine as long as you havenвЂ™t pushed yet. ItвЂ™s generally a bad idea to amend or rebase work that
youвЂ™ve already shared with others, unless you really know what youвЂ™re doing.
вЂў DonвЂ™t wait too long between commits
вЂ“ Squashing them together later is easy
Note: Commit every time you think you might want to return to the current state. You can revert back to any previous
commit, but there is no way to magically add a commit in where you forgot to make one.
1.15. Lesson 3: Intro to Git
49
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂў DonвЂ™t commit compiled/generated items
Note: Mostly relevant to writing code, .gitignore allows you to avoid dealing with compiled binaries, generated
output, log files, etc
вЂў DonвЂ™t commit secrets...
Note: Yes, there are ways to sort of take them down off of GitHub, but somebody might have cloned your repo while
it had the secrets in. Once someone has a piece of information, you canвЂ™t just take it away.
1.15.8 Daily workflow
вЂў Pull
вЂў Work
вЂў Add changes
вЂў Commit
вЂў Push
Larger projects have more complex workflows
Note: The picture is of the Git Flow branching model, and youвЂ™ll probably see it every single time anyone explains Git
branching and merging to you. If you are working on a larger project or writing code, youвЂ™ll likely be using branches,
this allows a project to keep many simultaneous code changes organized.
1.16 Lesson 3: GitHub
вЂў Manage permissions on repos
вЂў Back up your work
вЂў Social/gamification
вЂў Amazing documentation: http://help.github.com
Note: GitHub serves a threefold purpose: It also has amazing documentation which you should all go read right now
and consult whenever youвЂ™re the least bit confused. ItвЂ™s like the Ubuntu forums in that itвЂ™s explained in a way the
newbies can understand, but unlike them in that itвЂ™s all written by people who know what theyвЂ™re doing.
1.16.1 LetвЂ™s Walk Through
вЂў Creating an account
вЂ“ Gravatar
вЂ“ How to read a profile
50
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.16. Lesson 3: GitHub
51
OSU DevOps Bootcamp Documentation, Release 0.0.1
52
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
Note: you just go to github.com and click the account creation links. To make a custom icon, go to gravatar.com and
set up an account using the same email address as you used for github. The picture you upload on Gravatar will then
show up for your github account.
The most important thing about reading profiles is that not all of a personвЂ™s repos will display on the front page of their
profile вЂ“ to see them, got to the вЂ�repositoriesвЂ™ tab instead of вЂ�contributionsвЂ™.
вЂў Creating SSH keys
вЂ“ ssh-keygen -t rsa
вЂў Uploading your SSH key
вЂў Creating a new repository
вЂў Fork somebody elseвЂ™s repo
вЂў Edit files online
вЂў Submit a pull request
1.16.2 Help, EverythingsвЂ™s Broken!
Permission denied (publickey).
fatal: The remote end hung up unexpectedly
Solution ssh-add ~/.ssh/id-rsa or whatever key you have added on github
To git@github.com:edunham/slides.git
! [rejected]
master -> master (non-fast-forward)
error: failed to push some refs to вЂ™git@github.com:edunham/slides.gitвЂ™
hint: Updates were rejected because the tip of your current branch is behind
hint: its remote counterpart. Merge the remote changes (e.g. вЂ™git pullвЂ™)
hint: before pushing again.
hint: See the вЂ™Note about fast-forwardsвЂ™ in вЂ™git push --helpвЂ™ for details.
Solution To avoid a messy merge commit, git pull --rebase.
1.16.3 Learn More
вЂў http://git-scm.com/book
вЂў http://try.github.io/levels/1/challenges/1
1.16.4 Hands-On
вЂў Fork the devopsbootcamp dotfiles repo
вЂў Clone a copy of the repo to your VM and make a branch
вЂў Make a commit with a helpful commit message and push to your fork
$
$
$
$
$
ssh-keygen -t rsa # make an SSH key and add it to your account
git clone <url from sidebar of your fork> # clone the repo
cd dotfiles # git commands only work in project directlry
git checkout -b <yourname> # -b creates branch
vim <filename>
1.16. Lesson 3: GitHub
53
OSU DevOps Bootcamp Documentation, Release 0.0.1
#
#
#
$ git
$ git
$ git
вЂ™iвЂ™ to enter insert mode
<esc> to get back to command mode
:wq to save and quit
add <filename>
commit -m "please use a helpful commit message, not like this one"
push
1.17 Lesson 4: Scripting, Troubleshooting, & Workflow
Note: week 1, 1/9/2014 - log files - git bisect, git log, git blame - scientific method (small changes)
1.17.1 Agenda
вЂў Scripting
вЂ“ WhatвЂ™s a script?
вЂ“ WhatвЂ™s a scripting language?
вЂ“ Common scripting languages
вЂ“ Bash & Python examples
вЂў Troubleshooting
54
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂ“ Techniques
вЂ“ Examples
вЂў Workflow
вЂ“ Life cycle of a bug
1.17.2 Scripting
1.17.3 WhatвЂ™s a script?
вЂў Piece of code to automate a boring task
вЂў Explanation of how to do what you do
1.17.4 WhatвЂ™s a scripting language?
Note: This is more programming language than scripting specific.
вЂў Compiled vs interpreted
вЂў Expresses logic and actions
вЂ“ Logic: loops, conditionals, statements
вЂ“ Actions: File I/O, print to console, interact with system utilities
1.17. Lesson 4: Scripting, Troubleshooting, & Workflow
55
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.17.5 Common Scripting Languages
вЂў Bash
вЂў Python
вЂў Language affects the speed of development and performance of a task
вЂў Most tasks can be done using any language
1.17.6 Bash
вЂў Adds a bit of logic to automate your shell commands
вЂў Saves re-typing things
вЂў Powered by the tools you invoke from the shell
вЂў Scripts can be hard to read at first because they call to external utilities
1.17.7 Python
1.17.8 Why Python?
вЂў Easy to read
вЂў Easy to maintain
вЂў Quick to write
вЂў Lots of libraries
1.17.9 Troubleshooting
1.17.10 Informal method
вЂў Notice that something isnвЂ™t working right
вЂў Identify what should be happening
вЂ“ Define a success criterion (вЂњit works if...вЂќ)
1.17.11 If it used to work
вЂў Determine what changed
вЂ“ Version control history (Git bisect)
вЂ“ Emails from the system? Logs? (Check for cron jobs or config mgmt)
вЂ“ Ask others whoвЂ™ve been working on system
вЂў Use your own notes/documentation
56
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.17. Lesson 4: Scripting, Troubleshooting, & Workflow
57
OSU DevOps Bootcamp Documentation, Release 0.0.1
58
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.17. Lesson 4: Scripting, Troubleshooting, & Workflow
59
OSU DevOps Bootcamp Documentation, Release 0.0.1
60
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.17.12 If itвЂ™s never worked for you
вЂў Determine whether itвЂ™s possible at all
вЂў Find evidence of similar things working (code, blog posts, stackoverflow)
вЂў If thereвЂ™s no evidence of anything like this working, you might be Doing It Wrong (tm)
вЂў If thereвЂ™s documentation of something similar working:
вЂ“ Confirm that the docs are correct for the versions of things that youвЂ™re using
вЂ“ If they docs are wrong, fix them
вЂ“ If the docs appear right, figure out what differs between your code and the example
вЂў If thereвЂ™s sample code, make sure you can run it
вЂ“ Your goal is minimum viable test case
1.17.13 After finding the problem
вЂў Did the docs tell you how to fix it?
вЂў If you canвЂ™t fix the problem, identify why not, and then fix that
вЂў Ask for help
1.17. Lesson 4: Scripting, Troubleshooting, & Workflow
61
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂ“ Expert takes 5 minutes to answer a well-asked question
вЂ“ Newbie can waste hours
1.17.14 Formal method
(from this)
вЂў Identify the problem
вЂў Establish a theory of probable cause (question the obvious)
вЂў Test the theory to determine the cause
вЂў Establish a plan of action to resolve the problem and implement the solution
вЂў Verify full system functionality and, if applicable, implement preventative measures
вЂў Document findings, actions, and outcomes
1.17.15 How to get help
вЂў DonвЂ™t ask to ask
вЂў Summarize whatвЂ™s wrong
вЂў Summarize what youвЂ™ve tried and why it hasnвЂ™t worked
вЂў Make a specific request, politely
вЂў Pick the right place & time to ask
1.17.16 Documentation
вЂў Man pages
вЂў Wikis
вЂў Google (used wisely)
вЂ“ Assessing sitesвЂ™ applicability and reliability
* Who wrote it?
* When?
* Is the other content reliable?
* Is feedback from others visible? If so, what does it say?
1.17.17 Sources of trouble
When using something new
вЂў You probably misunderstood it.
вЂў Maybe their documentation was wrong.
вЂў If neither, then perhaps their code is wrong.
вЂў Submit a ticket or pull request to fix the docs or code
62
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
When something previously working breaks
вЂў Something changed
вЂў Someone updated something
вЂў Figure out who and why; document
1.17.18 Tickets
вЂў Ticket (often sysadmin) or Issue (often developer)
вЂў Ticket comes into tracking system, submitted by a user
вЂў Triage
вЂ“ Add details to tickets; consolidate duplicates
вЂ“ Contact submitter if more info needed
вЂ“ Add tags, milestones, priority, etc.
вЂў Ticket is assigned to someone, who fixes it
вЂў Someone else confirms that the fix works, then ticket is closed
1.17.19 Tickets vs. Issues
вЂў Workflow defined by tracker system
вЂ“ RT, Redmine, Chiliproject, GitHub issues, mailing lists
вЂў Issues/Bugs are developer work items which need to be included in a release of code
вЂў Tickets are sysadmin work items, often related to systems improvement or maintenance
вЂў CanвЂ™t log in because your account got reset: Ticket.
вЂў CanвЂ™t log in because the newest release of the software is incompatible with the old database format: Bug.
1.17.20 Some Examples
вЂў Trac
вЂў Chiliproject
вЂў RT
вЂў Bugzilla
1.18 Lesson 5: Web Applications
1.18.1 What is a web app?
Note:
вЂў talk briefly about responsibilities of each component
вЂў talk briefly about http
1.18. Lesson 5: Web Applications
63
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂў mention Virtual Environments (red box on diagram)
1.18.2 What is this Virtual Environment thing?
Note: Discuss system packages vs local, mention rvm
вЂў Separates application packages from system packages
вЂў Choose versions that differ from the system versions
вЂў Maintain a list of required packages for easy install
вЂў Install modules as a user
вЂў Pip and Gem
64
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.18.3 Frameworks
Ready-made building blocks take care of the tedious details:
вЂў Speaking HTTP
вЂў Connecting code to URLS
вЂў Talking to the database
вЂў Built-in development servers
Popular frameworks:
Django, CakePHP, Ruby on Rails, Flask
Note: Talk about major differences between frameworks, especially Django and Flask
1.18.4 The SystemView app
Display a filtered list of processes on the system
Note:
вЂў explain basic idea of the app
вЂ“ reference code from lesson 4
1.18.5 Setting Up
Update your vagrant file and vagrant up:
1.18. Lesson 5: Web Applications
65
OSU DevOps Bootcamp Documentation, Release 0.0.1
git pull
vagrant up
vagrant ssh
Note: Changes to the vagrant file: forward port 5050 to 5000 on the vm
1.18.6 Setting Up
Install packages
sudo yum install python-virtualenv*
sudo yum install git
Create a virtualenv
virtualenv systemview_venv
And activate it
source systemview_venv/bin/activate
Note:
вЂў students probably already have git?
вЂў discuss what virtualenv actually does, what is in it
вЂў env variables, etc
вЂў they can put the virtualenv anywhere, discuss locations
вЂў discuss, but donвЂ™t use virtualenv tools (mkvirtualenv, use, etc)
вЂў explain what source does
1.18.7 Get the Code
git clone git@github.com:DevOpsBootcamp/systemview.git
Note:
вЂў break here for github account setup, key location (where are they checking code out from? Where is their key
located?), etc
вЂў https://github.com/DevOpsBootcamp/systemview.git for anyone who canвЂ™t get their account/key working
1.18.8 Run the Code
cd systemview
python systemview.py
66
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.18.9 Fail
Oops!
Traceback (most recent call last):
File "systemview.py", line 2, in <module>
from flask import Flask, request, session, g, redirect, url_for, \
ImportError: No module named flask
Note: talk about missing modules, we need to install them, this is what the venv is for
1.18.10 Pip
A package manager for Python packages
вЂў Connects to PyPi, a vast repository of Python modules
вЂў Resolves dependencies, installs prerequisites
вЂў Can install packages from a list in a file
1.18.11 Install WhatвЂ™s Missing
Make sure you are in your virtualenv, then:
pip install flask
pip install argparse
Note:
вЂў How do you know if you are in the virtualenv?
вЂў can put requirements in requirements.txt for easy installation
1.18.12 Run and Test!
python systemview.py -i 0.0.0.0 -d
Now go to http://localhost:5050
Note:
вЂў talk about flags
вЂў go to terminal after this slide and talk about the code:
вЂ“ main module, templates, css, etc
вЂў Point out areas where bugs could be fixed or features added
1.18. Lesson 5: Web Applications
67
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.18.13 Branch and Modify
Create a Github issue for your changes
https://github.com/DevOpsBootcamp/systemview/issues
Create a branch for your changes
git checkout -b my_name
When you have made changes and everything works, push it up
git push origin my_name
Note:
вЂў talk about branching vs forking, get everyone working on a new feature or bug
вЂў use IRC handles for branch names to make sure you are unique and identifiable
1.18.14 Homework
Add a feature or fix a bug, push your changes up.
Github URL:
https://github.com/DevOpsBootcamp/systemview
Github issue tracker:
https://github.com/DevOpsBootcamp/systemview/issues
1.19 Lesson 6: Boot and the Filesystem Hierarchy
Note:
вЂў grub, filesystem stuff based roughly on FrostsnowвЂ™s talk
вЂў basics of kernel and differences between virtualization/physical (the picture that kevin draws)
1.19.1 The Linux Filesystem Hierarchy
Note: Based on WadeвЂ™s talk https://github.com/clinew/presentation_filesystems/blob/master/presentation.tex
WhatвЂ™s a filesystem?
In computing, a file system is used to control how information is stored and retrieved. Without a file
system, information placed in a storage area would be one large body of information with no way to tell
where one piece of information stops and the next begins.
вЂў (http://en.wikipedia.org/wiki/Filesystem)
68
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.19.2 Filesystem can mean:
вЂў How the systemвЂ™s files are arranged on the disk
вЂў How the disk actually holds the files
вЂ“ FAT and NTFS are old but Windows-compatible
вЂ“ ext3 is standard, ext4 is newer, xfs has fancier journaling
* journaling tracks changes before write
вЂ“ sysadmins will encounter NFS and its competitors like Gluster
Note: Moving from Windows?
вЂў Binaries, not executables.
вЂў Directories, not folders.
вЂў Read, not load.
вЂў Symbolic links, not shortcuts.
вЂў Write, not save.
1.19.3 The File System
$ ls
bin
boot
dev
etc
home
initrd.img
initrd.img.old
lib
lib64
lost+found
media
mnt
1.19. Lesson 6: Boot and the Filesystem Hierarchy
opt
proc
root
run
sbin
selinux
srv
sys
tmp
usr
var
vmlinuz
69
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.19.4 Installed programs and utilities
/bin
/sbin
/usr/bin
/usr/sbin
/usr/local/bin
/usr/local/sbin
вЂў PATH environment variable
$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
вЂў which command
$ which bash
/bin/bash
1.19.5 User-Specific Data & Configuration
вЂў Data stored at /home/<username>
вЂ“ Desktop environment creates folders Documents, Pictures, Videos, etc.
вЂў Configurations in dotfiles within home (/.)
вЂў Lost+Found is not your desktop trash can
вЂ“ Lost blocks of the filesystem.
вЂ“ Usually not an issue.
вЂ“ If your desktop provides backups of deleted files, theyвЂ™ll be somewhere in /home/<username>/
1.19.6 Where are drives mounted?
вЂў Raw device appears under /dev.
$ dmesg | tail
[260930.208715] sdb: sdb1
[260930.320756] sd 6:0:0:0: >[sdb] Asking for cache data failed
[260930.320765] sd 6:0:0:0: >[sdb] Assuming drive cache: write through
[260930.320771] sd 6:0:0:0: >[sdb] Attached SCSI removable disk
вЂў USB filesystem under /media, main disk /
вЂў You can manually mount devices with mount
вЂ“ вЂњEverythingвЂ™s a fileвЂќ
вЂ“ umount to unmount
вЂў /etc/fstab tells things where to mount
вЂў /etc/mtab shows where things are currently mounted
1.19.7 Space on drives
вЂў Use df to see disk free space.
70
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
$ df -h /
Filesystem
/dev/sda8
Size
73G
Used Avail Use% Mounted on
29G
41G 42% /
вЂў Use du to see disk usage.
$ du -sh /home/
21G /home/
вЂў Default output is in bytes, -h for human-readable output.
1.19.8 Three Tiers of Filesystem Hierarchy
вЂў /, essential for system booting and mounting /usr.
вЂў /usr, read-only system data for normal system operation.
вЂў /usr/local, locally-installed software.
вЂ“ Package managers usually install under / and /usr.
1.19.9 Common Directories
Directory
/bin
/include
/lib
/sbin
/boot
/dev
/etc
/home
/media
/mnt
Contents
Binary files
Header files for C/C++ programs
Libraries
Binary files for root (superuser)
Files essential for booting kernel, initramfs
Virtual filesystem, exports hardware devices
System-wide configurations
Individual usersвЂ™ data
Removable storage devices
Like media вЂ“ place to mount disks and things
1.19. Lesson 6: Boot and the Filesystem Hierarchy
71
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.19.10 Common Directories
Directory
/opt
/proc
/root
/run
/sys
/tmp
/var
/usr/share
/usr/src
Contents
вЂњAdd-on application software packagesвЂќ
Virtual filesystem exporting system data
homedir for root
Volatile information accumulated since boot
Virtual filesystem exporting kernel objects
Temporary files
Data which varies вЂ“ logs, mail, etc.
Architecture-independent, read-only data
Kernel source code
1.19.11 /proc has lots of useful system information
Which Linux kernel version are you running?
$ cat /proc/version
Linux version 3.5.0-17-generic (buildd@allspice) (gcc version 4.7.2
(Ubuntu/Linaro 4.7.2-2ubuntu1) ) #28-Ubuntu SMP Tue Oct 9 19:31:23 UTC 2012
Learn about systemвЂ™s hardware
$ less /proc/cpuinfo
$ less /proc/meminfo
Some parts of /proc can be written as well as read...
$ echo 3 > /proc/sys/vm/drop_caches # drop caches
1.19.12 Commands for working with filesystems
Creating filesystems
$ mkfs
Mounting filesystems
$
#
#
#
mount
-t for type
-o for options
requires device path and mount point
Loopback devices
$ losetup
$ /dev/loop*
# makes it look like a device instead of a file
1.19.13 devfs
sd*
sr*
/dev/null
72
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
/dev/random
/dev/urandom
/dev/zero
1.19.14 Blocks and dd
вЂў Block size is the size of chunks allocated for files
вЂў dd
вЂ“ Disk duplicator (or disk dump).
* if=<path>, input file.
* of=<path>, ooutput file.
* bs=<size>, block size.
* count=<size>, number of block to transfer.
$ dd if=/dev/random of=/dev/sda
# What will this do?
1.19.15 Filesystem Consistency
вЂў Metadata vs. data
вЂ“ Metadata is extra information the filesystem tracks about the file
вЂ“ Data is the fileвЂ™s contents
вЂў Filesystem is consistent if all metadata is intact
вЂ“ fsck is FileSystem Consistency Check
1.19.16 More about Journaling
вЂў Filesystem consistency tool; protections against system freezes, power outages, etc.
вЂў Replaying the journal.
вЂў ext3вЂ™s three modes of journaling:
вЂ“
journal Data and metadata to journal.
вЂ“
ordered Data updates to filesystem, then metadata committed to journal.
вЂ“
writeback Metadata comitted to journal, possibly before data updates.
1.19.17 The Boot Process
вЂў Bootstrapping
вЂў Steps in the process
вЂў Boot loaders
вЂў Startup scripts
вЂў Boot levels
1.19. Lesson 6: Boot and the Filesystem Hierarchy
73
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.19.18 Bootstrapping
Note: kernel loaded into memory, initialization tasks, and available to users
Init
вЂў kernel spawns init which is always PID 1
вЂў controls the boot process
вЂў can be a simple script to a binary
вЂў Pull itself up by its own bootstraps
вЂў Automatic and manual booting
вЂў Driver Loading
вЂў Period of vulnerability
вЂ“ configuration errors
вЂ“ missing hardware
вЂ“ damaged filesystems
вЂў init вЂ“ Always Process ID (PID) #1
вЂ“ First process to start
вЂ“ Either a binary or can be a simple script (even a bash shell!)
1.19.19 Steps in boot process
Note:
Kernel
вЂў 1st stage вЂ“ bootloader, 2nd, boot the kernel
вЂў boot from boot loader
вЂў load into memory
вЂў located in /boot/ on Linux
Hardware config
вЂў locate & initialize hardware
74
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂў print out what it does
System processes
вЂў init, kswapd, pdflush, etc
вЂў init only real process
вЂў Others look like processes for scheduling (appear as [kswapd] with ps)
1. Kernel initialization
2. Hardware configuration
3. System processes
4. Operator intervention (single-user)
5. Execution of start-up scripts
6. Multi-user operation
1.19.20 Booting
Note:
On hardware specific to UNIX (i.e. Sun)
вЂў firmware knows how to use devices
вЂў talk to the network
вЂў understand filesystems
вЂў all accessible via the commandline
1.19. Lesson 6: Boot and the Filesystem Hierarchy
75
OSU DevOps Bootcamp Documentation, Release 0.0.1
BIOS smarter than they used to be
вЂў Not standardized
вЂў Most servers support PXE
вЂў PCs vs Proprietary hardware
вЂ“ BIOS, UEFI, OpenBoot PROM, etc
вЂў BIOS
вЂ“ Basic Input/Output System
вЂ“ Very simple compared to OpenBoot PROM / UEFI
вЂ“ Select devices to boot from
вЂ“ MBR (first 512 bytes)
вЂў UEFI
вЂ“ Unified Extensible Firmware Interface
вЂ“ Successor to BIOS
вЂ“ Flexible pre-OS environment including network booting
1.19.21 Boot Loaders (Grub)
Note:
Grub
вЂў next generation PC boot loader
вЂў no need to вЂњre-run grubвЂќ config updates
вЂў Grub config
вЂў disks are index based from zero
вЂў grub install commands
вЂў netboot, pretty, serial
вЂў device.map, grub.conf
вЂў robust with weird disk geometry
вЂў Grand Unified Bootloader
вЂў Dynamic fixes during booting
вЂў Can read the filesystem
вЂў Index based вЂ“ (hd0,0) = sda1
вЂў Grub вЂњversion 1вЂќ vs. вЂњversion 2вЂќ
вЂ“ Version 2 has more features, but more complicated
вЂ“ Latest Debian, Ubuntu and Fedora use v2
76
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
grub> root (hd0,0)
grub> setup (hd0)
grub> quit
(Specify where your /boot partition resides)
(Install GRUB in the MBR)
(Exit the GRUB shell)
grub-install
1.19.22 Single User Mode
Note:
Show on VM
вЂў enter grub, hit ESC, pick kernel, hit вЂњeвЂќ for edit
вЂў use arrows
Typically ask for root password
вЂў What is it used for?
вЂў Troubleshoot problems
вЂў Manual Filesystem Checks
вЂў Booting with bare services
вЂў Fix boot problems
вЂў Add вЂњsingleвЂќ to kernel option
вЂў Solaris/BSD
вЂ“ boot -s
1.19.23 Startup Script Tasks
Note: Verbose and print out description of what its doing.
Old days were to manually adjust scripts, not anymore. Most are configurable now.
1.19. Lesson 6: Boot and the Filesystem Hierarchy
77
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂў Setting up hostname & timezone
вЂў Checking disks with fsck
вЂў Mounting systemвЂ™s disks
вЂў Configuring network interfaces
вЂў Starting up daemons & network services
1.19.24 System-V Boot Style
Note:
вЂў System-V Most common today
вЂў Show system changing between different run levels.
вЂў Slightly different between Distros
вЂў Linux derived from System-V originally
вЂў Alternative init systems
вЂ“ systemd - Fedora 15+, Redhat 7+ and Debian* (dependency driven)
вЂ“ upstart - Ubuntu, Redhat 6 (event driven, faster boot times)
Run levels:
level 0
level 1 or S
level 2 through 5
level 6
sys is completely down (halt)
single-user mode
multi-user levels
reboot level
1.19.25 /etc/inittab
Note: Look at inittab
вЂў Tells init what to do on each level
вЂў Starts getty (terminals, serial console)
вЂў Commands to be run or kept running
вЂў inittab not used with systemd or upstart
78
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
# The default runlevel.
id:2:initdefault:
# What to do in single-user mode.
~~:S:wait:/sbin/sulogin
# What to do when CTRL-ALT-DEL is pressed.
ca:12345:ctrlaltdel:/sbin/shutdown -t1 -a -r now
# terminals
1:2345:respawn:/sbin/getty 38400 tty1
T0:23:respawn:/sbin/getty -L ttyS0 9600 vt100
1.19.26 init.d Scripts
Note:
sshd init script
вЂў case statement
вЂў functions
вЂў chkconfig
вЂў One script for one service/daemon
вЂў Start up services such as sshd, httpd, etc
вЂў Commands
вЂ“ start, stop, reload, restart
вЂў sshd init script
$ service sshd status
openssh-daemon (pid 1186) is running...
$ service sshd restart
Stopping sshd:
Starting sshd:
[
[
OK
OK
]
]
1.19.27 Starting services on boot
Note: Show sshd script show list, adding, removing, enabling, disabling
вЂў rclevel.d (rc0.d, rc1.d)
вЂў S = start, K = stop/kill
вЂў Numbers to set sequence (S55sshd)
вЂў chkconfig / update-rc.d
вЂ“ Easy way to enable/disable services in RH/Debian
вЂў Other distributions work differently
1.19. Lesson 6: Boot and the Filesystem Hierarchy
79
OSU DevOps Bootcamp Documentation, Release 0.0.1
$ chkconfig --list sshd
sshd
0:off 1:off 2:on
3:on
4:on
5:on
6:off
$ chkconfig sshd off
$ chkconfig --list sshd
sshd
0:off 1:off 2:off 3:off 4:off 5:off 6:off
1.19.28 Configuring init.d Scripts
Note: show sendmail & network config examples for CentOS
/etc/defaults seems to be more common between UNIXвЂ™s
вЂў /etc/sysconfig (RH) or /etc/defaults (Debian)
вЂў source Bash scripts
вЂў Daemon arguments
вЂў Networking settings
вЂў Other distributions are vastly different
$ cat /etc/sysconfig/ntpd
# Drop root to id вЂ™ntp:ntpвЂ™ by default.
OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid -g"
1.19.29 Shutting Down
Note: Modern systems are less touchy with hard resets, but still need to be careful. Only for emergencies.
Shutdown -h
вЂў Not Windows, donвЂ™t reboot to fix issue
вЂў Can take a long time (i.e. servers)
вЂў Reboot only to
вЂ“ load new kernel
вЂ“ new hardware
вЂ“ system-wide configuration changes
вЂў shutdown, reboot, halt, init
вЂў wall - send system-wide message to all users
$ wall hello world
Broadcast message from root@devops-bootcamp (pts/0) (Fri Jan 31 00:40:29 2014):
hello world
80
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.19.30 Homework
1.20 Lesson 7: Databases
1.20.1 Filesystems Review Questions
вЂў What might happen to a busy ext2 volume on power loss?
вЂў ext3?
вЂў ext4?
Note:
вЂў ext2 may not be consistent upon restart
вЂў ext3 and ext 4 are not
вЂў but consistency only guarantees metadata is intact
вЂў What might happen to a busy ext2 volume on power loss?
вЂў ext3?
вЂў ext4?
1.20.2 But what about our poor data?
вЂў Possibly gone, like the wind
вЂў Or worse: Half completed writes!
вЂў General purpose operating systems, by design, donвЂ™t understand structured data
1.20.3 Enter the Database
A database systemвЂ™s fundamental goal is to provide consistent views of structured data using the tools the
operating system makes available.
Chief among them is fsync(2)
Note: fsync instructs the operating system to flush all writes to disk before returning
1.20.4 Structure
SQL databases are structured around Relational Algebra
вЂў Tables
вЂ“ Columns are fields
вЂ“ Rows define a relation between fields
вЂў A Primary key is a set of columns that uniquely identify rows in a table
вЂў A Foreign key is a column that matches the primary key of another table
1.20. Lesson 7: Databases
81
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.20.5 Relational Algebra Visualized
Note: joins are the principle use of relations.
1.20.6 Installing MySQL
$ yum install mysql-server
$ /sbin/service mysqld start
$ /usr/bin/mysql_secure_installation
82
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.20.7 Managing MySQL
$ /sbin/service mysqld status
$ mysqladmin -p ping
$ mysqladmin -p create nobel
1.20.8 Configuration
вЂў /etc/my.conf
вЂў The most important MySQL tuning rule:
вЂ“ almost always prefer InnoDB
Note: weвЂ™re going to add: default_storage_engine = InnoDB
1.20.9 Users & Permissions
$ sudo mysql -p
mysql> CREATE USER вЂ™vagrantвЂ™@вЂ™localhostвЂ™
IDENTIFIED BY вЂ™passwordвЂ™;
mysql> GRANT ALL PRIVILEGES ON nobel.*
TO вЂ™vagrantвЂ™@вЂ™localhostвЂ™
WITH GRANT OPTION;
1.20.10 Importing Data
$ wget http://osl.io/nobel -O nobel.sql
$ mysql -p nobel < nobel.sql
$ mysql -p nobel
SHOW TABLES;
DESCRIBE nobel;
1.20.11 Basic Queries
4 basic operations on data:
вЂў SELECT
вЂў INSERT
вЂў UPDATE
вЂў DELETE
1.20.12 SELECT
1.20. Lesson 7: Databases
83
OSU DevOps Bootcamp Documentation, Release 0.0.1
SELECT
yr, subject, winner
FROM
nobel
WHERE
yr = 1960;
1.20.13 Practice
вЂў Who won the prize for Medicine in 1952?
вЂў How many people were awarded the 1903 Nobel in Physics?
вЂў How many prizes were awarded to Linus Pauling?
вЂў How many people have won more than once? (Difficult)
1.20.14 INSERT
INSERT VALUES
(вЂ™2013вЂ™,вЂ™LiteratureвЂ™,вЂ™Herta MГјllerвЂ™)
INTO
nobel;
Note: this data stops at 2008, so lets insert some 2009 awards
1.20.15 Practice
In 2009:
вЂў Barack Obama won the Peace Prize
вЂў Elinor Ostrom and Oliver E. Williamson won the prize in Economics
вЂў http://en.wikipedia.org/wiki/List_of_Nobel_laureates
1.20.16 UPDATE
UPDATE
nobel
SET
winner=вЂ™Andrew RyanвЂ™
WHERE
subject=вЂ™PeaceвЂ™ AND yr=вЂ™1951вЂ™;
Note: obviously Andrew Ryan deserves the peace price for his work in the Rapture planned community
1.20.17 Practice
вЂў Brigid Tenenbaum Medicine prize in 1952
84
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.20.18 DELETE
DELETE FROM
nobel
WHERE
yr = 1989, subject = peace;
Note: peace prizes can be controversial, and perhaps thereвЂ™s a political interest in censoring our database?
1.20.19 Further Reading, Resources, etc.
вЂў Codd, E.F. (1970). вЂњA Relational Model of Data for Large Shared Data BanksвЂќ. Communications of the ACM
13 (6): 377вЂ“387.
вЂў sqlzoo.net
вЂў CS 440: Database Management Systems
1.20.20 Hands-On: Make a Database
вЂў Create a new database
mysql> create database systemview
mysql> GRANT ALL PRIVILEGES ON systemview.*
TO вЂ™vagrantвЂ™@вЂ™localhostвЂ™
WITH GRANT OPTION;
вЂў Grant a user privileges on your new database
Note: challenge them to do this based on the material in the last hour, maybe also demo the mysql console. Make
sure everyone remembers the username and password for the next step.
1.20.21 Databases in Applications
Applications love databases.
вЂў Application data - the information to be displayed and manipulated
вЂў User data - complex authentication and authorization
вЂў Logging, statistics, state and session data, etc...
Note: All the various things an app might use a database for - note that the vast majority of web apps use them for
something
1.20.22 Native SQL
Most languages allow you to speak directly to a database
Python:
1.20. Lesson 7: Databases
85
OSU DevOps Bootcamp Documentation, Release 0.0.1
#!/usr/bin/python
import MySQLdb
db = ("localhost","testuser","test123","nobel" )
cursor = db.cursor()
cursor.execute("SELECT subject, yr, winner FROM nobel WHERE yr = 1960)
data = cursor.fetchall()
for winner in data:
print "%s winner in %s: %s " % (winner[0], winner[1], winner[2])
db.close()
Note: Note the plain SQL statement, recognizable from earlier. Point out the cumbersome nature of creating the
connection, creating a cursor, sending the sql, getting data from the cursor (iterating over it if you want multiple
results), etc. Similar interfaces exist for virtually all languages.
1.20.23 Introducing the ORM
Object Relational Mapper
вЂў Maps an Object in an application to a database table or relationship
вЂў Talks SQL to the database, your favorite language to you
вЂў Lets you point to different databases with the same syntax
вЂў Intelligently manages transactions to the database
Note: Make sure people know what you mean by вЂњobjectвЂќ, mention possible difference between Postgres, sqlite,
MySql, etc. Objects may map to one table, but might also incorporate relationships. ORMs also often optimize queries
and manage transactions to make database queries as efficient as possible (like all other magic, though, sometimes this
can backfire).
1.20.24 Life With a Python ORM
Look, ma! No SQL!
for subject, yr, winner in session.query(Nobel).filter_by(yr=1960):
print "%s winner in %s: %s " % (subject, yr, winner)
Much easier to read and understand, but requires some setting up first.
Note: Of course we actually have to do a lot of setup work - setting up the model, engine, session, etc - but you do
that once and can interact with the database as much as you want, without worrying about the cursor or connection.
Note that we have no SQL in this statement, it is pythonic and has pythonic methods. The database table is now an
object.
86
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.20.25 Setting Up the Magic - SqlAlchemy
SqlAlchemy - a popular Python ORM, frequently used in Flask apps (like SystemView!).
To use it, weвЂ™ll need to:
вЂў Import sqlalchemy
вЂў Create a вЂњmodelвЂќ - a representation of our data in code
вЂў Create an вЂњengineвЂќ and connect it to the database
вЂў Create a session to store the model instances and transactions
Note:
Model A object with all the properties, attributes, etc of our data, can also include code to manipulate
that data in order to represent a specific view (i.e. automatically returning sorted results). ItвЂ™s just a
python class, instances are just python objects.
Engine This handles the authentication with the database, itвЂ™s like the MySQLdb.connect above.
Session An in-memory record of your changes to objects - all the orm objects you instantiate live int he
session, and are only saved to the database when you say so.
1.20.26 LetвЂ™s Databasify Systemview
Project:
вЂў Store search terms, then provide them as links on the search page, so you can just click the most common terms
you search for.
What else? Ideas?
Note: Solicit ideas for another column or two, maybe number of times the term is used (easy incrementing example),
or number of results from the least search.
1.20.27 Hands On
вЂў Install the following packages:
sudo yum install python-devel
sudo yum install mysql-devel
вЂў Check out systemview from GitHub (if you donвЂ™t have it already)
git clone git@github.com:DevOpsBootcamp/systemview
1.20.28 Hands On (Cont...)
вЂў Switch to вЂ�save-searchвЂ™ branch
git checkout -tb save-search origin/save-search
вЂў Activate your virtualenv
1.20. Lesson 7: Databases
87
OSU DevOps Bootcamp Documentation, Release 0.0.1
source <path to virtualenv>/bin/activate
вЂў Install the requirements
pip install -r requirements.txt
Note: Talk about git branches again, explain tracking, git pull for people who already have it cloned, etc. Talk about
the virtualenv, have people create a new one if they have lost the one they made last time. Talk about pip and what
requirements.txt is all about - point out how easy it is to set up an app this way. Make sure requirements.txt contains
sqlalchemy.
DANGER! - people will need mysql-dev package! name varies by distribution, for centos it is libmysqlclient-dev
1.20.29 Goals
вЂў Connect the app to your new database
вЂў Add a new column
вЂў Save data to that column whenever someone searches
вЂў Fetch the data from that column and display it on the search page
вЂў challenge: limit the returned result to only 5 terms
http://docs.sqlalchemy.org/en/rel_0_9/orm/tutorial.html
Note: The code in the repo should have a simple model with one column, вЂ�termвЂ™, you can make a models.py,
or just put it all in one file. If you separate them, talk about MVC. The code should start an sqlalchemy engine and
session, save the search term normalized (lowercased, stripped), the column should be set to unique. Make sure the
code handles the case of the term already existing in the database (when you add a count, increment the count when
the term exists). You should probably initialize the db directly in the code, otherwise youвЂ™ll have to open up a python
console, import the app and run the db update.
1.21 Lesson 8: Security & Authentication
1.21.1 What is security?
seВ·cuВ·riВ·ty
sikyoorit/
noun
вЂў the state of being free from danger or threat.
вЂў the safety of a state or organization against criminal activity such as terrorism, theft, or espionage.
вЂў procedures followed or measures taken to ensure the safety of a state or organization.
вЂў the state of feeling safe, stable, and free from fear or anxiety.
88
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.21.2 Types of security
вЂў Physical security
вЂў Software security
вЂў Network security
вЂ“ Active vs. passive
1.21.3 Authentication vs. Authorization
Authentication Are they who they say they are?
Authorization Are they allowed access here?
вЂў Authentication requires proof of identity
вЂў Authorization requires authentication, plus permission from an authority
1.21. Lesson 8: Security & Authentication
89
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.21.4 Identity
Persistent vs. authoritative
Imagine an identity thief who takes out lines of credit in their victimвЂ™s name then pays all the bills on time...
вЂў Is their identity persistent?
вЂў Is their identity authoritative?
How about a project maintainer who never uses their real name online, but uses the same handle and email address
across all sites?
1.21.5 Identity
How about a project maintainer who loses the domain which was hosting their email, and thus changes addresses
abruptly?
If youвЂ™re a sysadmin who works with multiple projects, you will run into these concerns often.
1.21.6 Principle of Least Authority
вЂў User & Group management
вЂў ACLs
вЂў File permissions
1.21.7 System Security
вЂў Close ports
вЂў Firewalls
вЂў Process isolation
вЂў nmap vs. fail2ban
90
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.21.8 Other Attacks
вЂў Social engineering
вЂ“ Pretexting
вЂ“ Phishing
вЂ“ Baiting
вЂ“ Quid Pro Quo
вЂ“ Tailgaiting
вЂў Zero-Day vulnerabilities
Note: Social engineering leverages those person-to-person skills us computer folks are so well known for not having.
Can someone let me borrow their laptop for a minute? I just want to tell my mom IвЂ™ll be home late tonight... honest!
Pretexting is when someone contacts you with a pretext. They sound like theyвЂ™re authentic because they know something about you which they probably got off Facebook or something else. вЂњHi, IвЂ™m calling from Chase Bank and I
noticed that your card might have been used at a location where fraud was committed. I have your name as Bob Smith
and your date of birth as May third, 1992. Can you read me your card number and the three digits on the back?вЂќ
Phishing is something weвЂ™ve all been warned about. We all know that eBay and Paypal arenвЂ™t going to email us asking
for our bank account information. Just donвЂ™t fall for it and youвЂ™ll be okay.
Baiting is a little more interesting. Ever walk down the street and notice a thumb drive or SD card on the ground? Hey
вЂ“ free thumb drive, right? LetвЂ™s just put it in the computer, what could go wrong? Plenty. If itвЂ™s too good to be true, it
probably is.
Quid pro quo is a trade вЂ“ IвЂ™ll give you something if you give me something. Would you trade your password for a
chocolate bar? According to the BBC, someone tried this outside a subway station in London in 2004, and more than
seventy percent of people took that trade! Thirty-four percent gave it up for free. DonвЂ™t do that.
I suspect many of you who live in the dorms have been told about tailgating. YouвЂ™re unlocking the dorm door and
someone comes up and says вЂњhey, I forgot my key in my room, can you let me inвЂќ or maybe theyвЂ™re wearing a DominoвЂ™s
1.21. Lesson 8: Security & Authentication
91
OSU DevOps Bootcamp Documentation, Release 0.0.1
uniform and are carrying a pizza. YouвЂ™re a nice person, you want to help them. DonвЂ™t do it.
And that leaves us with zero-day vulnerabilities. The term вЂ�zero dayвЂ™ refers to the amount of time that the folks who
write and maintain the code have had to fix it. If they donвЂ™t know about it, they canвЂ™t fix it. This is why itвЂ™s so important
for us to report vulnerabilities when we discover them as was discussed earlier.
1.21.9 What to do if you discover a vulnerability
First, test and document to verify that it exists.
Then, disclose it privately to those responsible for fixing it
Provide examples вЂ“ itвЂ™s basically a bug report, but through private channels (not public tracker yet!)
Give them time to release a patch before announcing it
Some places have bug bounties
1.21.10 Passwords
1.21.11 Good Passwording
вЂў http://bash.org/?244321
1.21.12 Server Side
вЂў Rainbow Table
вЂў Hashing / salt
вЂў bcrypt/ scrypt
92
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.21. Lesson 8: Security & Authentication
93
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.21.13 Password Managers
вЂў Password managers (LastPass, 1Password, KeepPass*)
вЂ“ Works with phones and other things
вЂў pass http://www.zx2c4.com/projects/password-store/
вЂў vim -x passwords.txt
вЂў http://world.std.com/~reinhold/diceware.html
Note: http://makezineblog.files.wordpress.com/2013/01/fractal-rainbow-table-runner-1.jpg We use passwords for everything we do online. Some (hopefully) semi random grouping of letters, numbers, and symbols which when combined with a username allow you to authenticate with a server or process. There are a couple common attacks on
passwords, the most common of which is called a dictionary attack. This uses the fact that words are easier to remember than random characters, so it abuses human memory in order to greatly reduce the search space for passwords.
pwgen + a password manager will help you have better passwords which you donвЂ™t even have to remember! DEMO
Storing passwords on the server side is a whole other matter. One of the primary issues of concern is what happens if
your server gets compromised. Lets say for instance that you just have a giant text file that has the form вЂњusername
passwordвЂќ on each row. This would be super fast to to lookup users in, but if that file ends up in the wrong hands, you
lose. A better option is to not store the passwords directly, but to store some representation of the password. This is
where a hash comes in. Essentially a hash is a one way function that is fast to calculate, deterministic in output, but
_very_ hard to reverse. If you store the hash of a password you can hash what they send you to verify who they are.
Again we must consider what happens if our database was compromised. Since these hashes are deterministic and
computing power is so cheap, we can precalculate what passwords correspond to what hashes, these precomputed files
are called rainbow tables. To avoid the issue with rainbow tables we вЂ�saltвЂ™ our passwords. This adds a small random
string to each password so that the search space for precomputing possible passwords becomes tera/petabytes large.
Enough about passwords, we now move into more interesting things called keys!
94
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.21.14 Keys
вЂў Better than passwords
вЂў Symmetric vs Asymmetric
вЂў Diffie-Hellman / RSA
1.21.15 Key Exchange
1.21.16 RSA
вЂў Math
вЂў Math
вЂў More Math
вЂў DonвЂ™t be shy
Note: Keys are password files. These can be used in place of a password for authentication and encryption.
Symmetric keys essentially work like passwords. They are basically a one-time pad where both parties need to know
the key to enable data to be stored and retrieved. Asymmetric keys work by encrypting with a public key (one everyone
can see), but only being able to be decrypted by the private key (which you shouldnвЂ™t show anyone).
1.21. Lesson 8: Security & Authentication
95
OSU DevOps Bootcamp Documentation, Release 0.0.1
The fundamental problem with communication is that if you donвЂ™t have a preshared key between two users, everything
you say is being listened to and presumably logged.
Diffie Hellman key exchange is probably the most important result in cryptography. It allows two users to communicate in plaintext (non-encrypted) and trade their public keys in order to generate a shared secret so then they can
communicate with encryption. RSA is an algorithm that follows Diffie-Hellman and is the most common way to do
key exchange.
1.21.17 SSH
вЂў Password vs Keys
вЂў Passphrases
вЂў authorized_keys
вЂў Automation
$ ssh -D 9999 manatee@freshblue.lake
$ ssh -R 2222:localhost:22 freshblue.lake
Note: ssh is secure shell and provides a shell to a unix machine over the вЂ�net by using RSA to encrypt communications
between a client and server. Passwordless login, refuse connections without keys, tunneling. Commands at the end
are run unecrypted.
Passphrases work by adding a password to a key file. Add your friends public keys to authorized_keys so they donвЂ™t
need a password to login.
ssh-agent, .ssh/config, /etc/ssh/sshd_config
DEMO Make ssh-keys, post to pastebin.osuosl.org
1.21.18 Brief History of Time (line of GPG)
вЂў P(retty)G(ood)P(rivacy)
вЂў Phil Zimmermann
1.21.19 GPG
вЂў E-mail privacy
вЂў Why you should use GPG
вЂў Why people donвЂ™t use GPG
вЂў Keys, signing, keyservers
вЂў Encryption
1.21.20 Ways to use GPG
вЂў Enigamail
вЂў mutt
96
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.21. Lesson 8: Security & Authentication
97
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂў Command line
$ gpg --encrypt manateessecrets.jpg.exe
1.21.21 Certificates and HTTP
вЂў Certificate Authorities
вЂў https
вЂў ssl/tls
$ openssl req -new -x509 -key /etc/ssl/private/privkey.pem \
-out /etc/ssl/certs/cacert.pem -days 1095
1.21.22 Man in the Middle
Note:
вЂў 650 CAs
вЂў Attacks on https/ssl
вЂў Future
DEMO sslsniff
98
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.21.23 WiFi
вЂў wep
вЂў wpa
вЂў wpa2
вЂў Wireshark
вЂ“ Demo
Note:
вЂў Attacks
вЂў mschapv2
DEMO Wireshark
1.21.24 Crypto-wares
вЂў Files
вЂ“ Tarsnap, SpiderOak, rsync over ssh
вЂў Communications
вЂ“ VPN
вЂ“ TextSecure/ RedPhone
вЂ“ Tor
вЂ“ https everywhere
вЂў Security
вЂ“ Metasploit, BEEF
вЂ“ AirCrack, sslstrip
1.21.25 Math!
вЂў Primes
вЂў Number Theory
вЂў Fields
вЂў Elliptic Curves
Note: DEMO rot13
1.21.26 One Last Thing
вЂў https://priv.ly ( proudly hosted at the OSL)
вЂў вЂњI have nothing to hideвЂњ
вЂў jeremykun.com
1.21. Lesson 8: Security & Authentication
99
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂў thoughtcrime.org
вЂў https://www.schneier.com/
1.22 Lesson 8: Web application security
Figure 1.4: image source: https://info.cenzic.com/rs/cenzic/images/2013-vulnerability-summary_290x250.png
1.22.1 Web application security
вЂў Who needs to worry about web application security?
вЂ“ Everyone!
100
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂў What kinds of attacks are seen in the wild?
вЂ“ Many!
вЂў What can devops do about these attacks?
вЂ“ A lot!
Note: Everyone needs to worry about web application security. You need to worry, because youвЂ™re learning how to
write web applications. You want to avoid making decisions which could lead to exposing vulnerabilities and letting
bad people use your service to do bad things. You also need to worry even if youвЂ™re not writing web applications,
because youвЂ™re using web applications. The web is still a wild and wooly place, and the last line of defense for the
user is their own common sense.
What kinds of attacks are seen in the wild? The image shows a dizzying array of acronyms and shorthand but weвЂ™ll be
going over those in a little more detail.
And what can devops like us do about these attacks? Plenty вЂ“ wait and see.
1.22.2 Code Injection
вЂў Attacks
вЂ“ SQL Injection
вЂ“ Cross-Site Scripting (XSS)
вЂ“ Cross-Site Request Forgery (CSRF)
вЂ“ Remote Code Execution
вЂў Defenses
вЂ“ Sanitize your inputs!
http://bobby-tables.com/ https://docs.djangoproject.com/en/dev/ref/contrib/csrf/ http://guides.rubyonrails.org/security.html
Note: These types of attacks consist of code that is introduced into the application causing unexpected behavior.
This code can be introduced unintentionally by typical users who use quotes or ampersands in their inputs as well as
intentionally by nefarious folks.
The comic demonstrates a classic SQL injection attack. Bobby took advantage of the schoolвЂ™s software not properly
sanitizing their inputs by including a command to drop the students table, causing the kind of chaos often seen in xkcd
comics.
Cross-site scripting works much the same way: someone posts a comment on a blog which includes Javascript which
gets executed when you view the comment. When it is executed, it does something horrible like send them your cookie
for that blog site.
1.22. Lesson 8: Web application security
101
OSU DevOps Bootcamp Documentation, Release 0.0.1
Cross-site request forgeries are similar but instead of Javascript youвЂ™ll see image links that really point to another site
like your bank, hoping that your cookies will let them transfer money from your accounts to theirs.
Remote site execution is what it sounds like: just like the SQL injection attack, but instead running a shell command
on the web server. I think by now you all have enough experience with running commands on your virtual machines
to know how bad that could be.
Luckily, each of these threats can be addressed the same way: listen to BobbyвЂ™s mom and sanitize your inputs! ThereвЂ™s
a web site dedicated to helping developers with SQL injection threats which IвЂ™ve listed above, but the same concepts
apply to the other threats. Want to stop cross-site scripting? DonвЂ™t allow users to contribute arbitrary Javascript in
comments. Want to stop cross site request forgeries? Make sure your GET requests are free of side-effects, and
include tokens in your forms. As a bonus, Django will do that last bit for you if you let it вЂ“ check out that second
link up there for more details. That third link is the Rails security guide and provides advice on these issues as well as
many others.
1.22.3 Web Server-Specific Attacks
Figure 1.5: image source http://news.netcraft.com/wp-content/uploads/2014/02/apache-vulns1.png
вЂў Version-Based
вЂў Configuration-Based
Note: There is a constant battle between developers and the bad guys вЂ“ one side discovers vulnerabilities, the other
side fixes them. One of the easiest things to do to keep the bad guys out is to use the most up-to-date version of your
web server, regardless of whether itвЂ™s Apache or IIS or nginx.
The graph above shows the most popular versions of Apache as of February 2014 according to Netcraft. Apache
encourages admins to run the latest major release of the 2.4 stable branch, which is Apache 2.4.7. How many of those
102
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
releases do you see in that image? ThatвЂ™s right вЂ“ none. Heck, two of the top fifteen are EOLed вЂ“ they arenвЂ™t even
receiving security updates any longer! This is bad. DonвЂ™t be like them.
But itвЂ™s not enough to run the latest version. You should also make sure your configuration files are updated as well.
Some default configurations will include accounts or passwords which can be guessed by hackers. Other times certain
features will be enabled by default, which can introduce vulnerabilities you donвЂ™t expect even though youвЂ™re not using
those features. Read the release notes when you update your software. Pay attention to details. They will. You should
too.
1.22.4 Problems with Design and Implementation
вЂў Authentication and Authorization
вЂў Session Management
вЂў Information Leakage
вЂў Unauthorized Directory Access
Note: The remaining threats facing the typical web developer come down to design and implementation problems.
The fine points of authentication and authorization have been discussed already: make sure that all your actions are
authorized by authenticated users and you should be okay.
Also, donвЂ™t let your cookies have infinite lifetimes. Better to have your users occasionally log in again than let
them be vulnerable to those cross-site attacks we covered before. Pro tip: PHP has a default setting for вЂњsession.cookie_lifetimeвЂќ of zero, which means they never expire. If youвЂ™re using PHP, fix that.
Information leakage is pretty sneaky. LetвЂ™s say your app allows users to request a password reset by entering their
email addresses. If your app behaves differently when valid and invalid addresses are input, congratulations, youвЂ™re
leaking information. Unauthorized directory access is a specialized form of information leakage вЂ“ while itвЂ™s nice to let
people know how to contact your staff, you might not want to let them download everyoneвЂ™s email address and such.
1.22.5 What Not to Do: The Exercise
1.22.6 Getting Up to Date
вЂў ssh into your vagrant environment
вЂў change directory to your local systemview repo
$ cd ~/projects/systemview
вЂў Make sure your local copy is up to date
$ git pull
вЂў If youвЂ™ve modified code youвЂ™ll need to follow these instructions
$ git stash save "some witty name about your work"
$ git pull --rebase
1.22. Lesson 8: Web application security
103
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.22.7 LetвЂ™s Check out DeanвЂ™s (not so) Awesome Code
$ git checkout <not so awesome code branch goes here>
1.23 Lesson 9: Networking
1.23.1 Who is this talk for?
Someone with little or no networking knowledge.
ECE/CS 372 at OSU covers this content, more or less
1.23.2 What is a network?
вЂњa group or system of interconnected people or thingsвЂњ
To us, a network is:
вЂў Electronic devices
вЂў Sending signals over wire, fiber, or radio
вЂў Communicating data using a standardized protocol
1.23.3 What is a protocol?
вЂњA set of agreed upon rules for communicationвЂњ
вЂў Defines sequence & format of packets being sent
1.23.4 The OSI Model
Open Systems Interconnection
вЂў Layers of abstraction
Note: вЂњCreate a layer of easily localized functions so that the layer could be totally redesigned and its protocols
changed in a major way... without changing the services expected from and provided to adjacent layersвЂќ
1.23.5 Layer 1: Physical
Networking Hardware
вЂў Connector shapes
вЂў Wire, optical fiber, or radio signal specifications
RS-232
104
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.23. Lesson 9: Networking
105
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.23.6 Layer 2: Data Link
MAC: Media Access Control
вЂў MAC address should be globally unique
вЂў ARP: Address Resolution Protocol (between layer 2 & 3)
вЂў NDP (neighbor discovery protocol) used in IPv6
вЂў Flow control & error checking
1.23.7 Layer 3: Network
Packet forwarding and routing
Network and host addressing
вЂў IPv4
вЂў IPv6
1.23.8 Layer 4: Transport
Interact directly with program same-order delivery, reliability, flow control, and congestion avoidance
TCP Transmission Control Protocol
вЂў used by HTTP, HTTPS, SMTP, POP3, IMAP, SSH, FTP, Telnet
UDP User Datagram Protocol
вЂў No error checking built in
вЂў No retransmission delays
вЂў VoIP, media, games
1.23.9 Get your hands dirty
In a linux terminal run::
ip a
These will display information about your network interfaces.
See also:
ifconfig
iwconfig
1.23.10 Example output:
user@host:~$ ip a
...
2: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN qlen 1000
link/ether 33:77:00:44:66:33 brd ff:ff:ff:ff:ff:ff
3: wlan1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
106
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
link/ether 24:77:33:44:55:66 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.55/24 brd 192.168.1.255 scope global wlan1
inet6 fe80::2677:3ff:fed4:538c/64 scope link
valid_lft forever preferred_lft forever
1.23.11 Netmask:
Decimal IP Address
192.168.1.55
255.255.255.0
Binary IP Address
11000000.10101000.00000001.00110111
11111111.11111111.11111111.00000000
Part of address
Network (Decimal)
Network (Binary)
Host (Decimal)
Host (Binary)
Corresponding address
192.168.1.0
11000000.10101000.00000001.00000000
0.0.0.55
00000000.00000000.00000000.00110111
Available Hosts: 192.168.1.[1-254]
Broadcast address: 192.168.1.255
1.23.12 Netmask Example:
Decimal IP Address
192.168.90.55
255.255.192.0
Binary IP Address
1.23.13 Netmask Example:
Decimal IP Address
192.168.90.55
255.255.192.0
Binary IP Address
11000000.10101000.01011010.00110111
11111111.11111111.11000000.00000000
Part of address
Network (Decimal)
Network (Binary)
Host (Decimal)
Host (Binary)
Corresponding address
192.168.64.0
0.0.26.55
1.23.14 Netmask Example:
Decimal IP Address
192.168.90.55
255.255.192.0
Binary IP Address
11000000.10101000.01011010.00110111
11111111.11111111.11000000.00000000
Part of address
Network (Decimal)
Network (Binary)
Host (Decimal)
Host (Binary)
Corresponding address
192.168.64.0
11000000.10101000.01000000.00000000
0.0.26.55
00000000.00000000.00011010.00110111
Available Hosts: 192.168.[64-127].[1-254]
1.23. Lesson 9: Networking
107
OSU DevOps Bootcamp Documentation, Release 0.0.1
Broadcast Address: 192.168.127.255
1.23.15 Routes
user@host:~$ route
Kernal IP routing table
Destination
Gateway
default
foo.osuosl
link-local
*
192.168.1.0
*
Genmask
0.0.0.0
255.255.0.0
255.255.255.0
Flags
UG
U
U
Metric
0
1000
2
Ref
0
0
0
Use
0
0
0
Iface
wlan1
wlan1
wlan1
user@host:~$ route -n
Kernel IP routing table
Destination
Gateway
0.0.0.0
192.168.1.1
169.254.0.0
0.0.0.0
192.168.1.0
0.0.0.0
Genmask
0.0.0.0
255.255.0.0
255.255.255.0
Flags
UG
U
U
Metric
0
1000
2
Ref
0
0
0
Use
0
0
0
Iface
wlan1
wlan1
wlan1
1.23.16 Bootstrapping
What happens when your computer connects to a network?
1. Duplex and speed negotiation
2. Static or dynamic configuration is applied
1.23.17 Static Configuration
Must in advance know:
вЂў IP Address
вЂў Netmask
вЂў Default Gateway
вЂў DNS Servers (optional in some cases)
1.23.18 Dynamic Configuration
All of the statically defined parameters are retrieved over the network via DHCP
But how do you communicate over the network without a network configuration?
1.23.19 Reserved IPv4 Addresses
вЂў 127.0.0.1
вЂў 192.168.0.0
вЂў 172.16.0.0
вЂў 10.0.0.0
вЂў 169.254.0.0
108
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.23.20 Public vs Private Address
NAT Network Address Translation
вЂў lose end-to-end traceability
вЂў hides internal network topology
вЂў allows use of private IPвЂ™s over public internet
вЂў conserves limited public IPвЂ™s
1.23.21 Network Devices
1.23.22 Network Devices
1.23.23 Control Layer
Connection oriented vs Connectionless
1.23.24 Collisions
CSMA CA All Wireless networks use this Carrier Sense Multiple Access with Collisions Avoidance
CSMA CD Carrier Sense Multiple Access with Collisions Detection
Why is this important?
http://articles.latimes.com/2007/aug/15/local/me-lax15
1.23. Lesson 9: Networking
109
OSU DevOps Bootcamp Documentation, Release 0.0.1
110
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.24 Lesson 10: Networking part 2
1.24.1 Agenda
вЂў Networking review
вЂў DNS
вЂў Web Server Setup
1.24.2 Networking Review
1.24.3 Networking Review
1.24.4 Networking Review
1.24.5 Networking Review
1.24.6 Networking Review
1.24.7 Intro to DNS
I get the OSLвЂ™s home page when visiting osuosl.org
1.24. Lesson 10: Networking part 2
111
OSU DevOps Bootcamp Documentation, Release 0.0.1
112
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.24. Lesson 10: Networking part 2
113
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.24.8 How do I know where the page came from?
HOST(1)
BIND9
HOST(1)
NAME
host - DNS lookup utility
SYNOPSIS
host [-aCdlnrsTwv] [-c class] [-N ndots] [-R number]
[-t type] [-W wait] [-m flag] [-4] [-6] {name}
[server]
DESCRIPTION
host is a simple utility for performing DNS lookups. It is
normally used to convert names to IP addresses and vice
versa. When no arguments or options are given, host prints
a short summary of its command line arguments and options.
1.24.9 OSL
$ host osuosl.org
osuosl.org has address 140.211.15.183
How did it know?
1.24.10 DNS
Domain Name System
вЂў Port 53
My laptop asked a DNS server how to get to osuosl.org.
114
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.24.11 Hierarchy of domains
1.24.12 Pop quiz
вЂў WhatвЂ™s 15 * 823?
вЂў 12345
1.24.13 Recursive DNS
вЂў Always gives real answer or error
вЂў Vulnerable to cache poisoning
1.24. Lesson 10: Networking part 2
115
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.24.14 Non-recursive
вЂў Uses cache or referral
вЂў Includes all authoritative-only
вЂ“ root and top-level domain servers
1.24.15 Another Quiz
вЂў WhatвЂ™s 15 * 823?
вЂў How did you know so quickly?
1.24.16 Caching
вЂў You cached the answer.
вЂў DNS can be cached at routers, ISPs, and DNS servers to improve performance.
вЂ“ Negative caching: Remember fails
вЂў TTL
1.24.17 /etc/resolv.conf
Configuration for BIND (Berkeley Internet Name Domain tool)
вЂў most common DNS resolver
вЂў current version is BIND 9
$ cat /etc/resolv.conf
# Generated by resolvconf
domain wireless.oregonstate.edu
nameserver 128.193.15.13
nameserver 128.193.15.12
Can only handle recursive name servers, no referrals
1.24.18 /etc/hosts
Used to skip looking up the DNS
Useful for testing web sites
Avoid malicious sites
$ cat /etc/hosts
127.0.0.1 devops-bootcamp32.osuosl.org devops-bootcamp32 localhost
localhost.localdomain localhost4 localhost4.localdomain4
::1
localhost localhost.localdomain localhost6 localhost6.localdomain6
116
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.24.19 Load Balancing
Multiple IPs bound to a single hostname are returned in random order
$ host google.com
google.com has address
google.com has address
google.com has address
google.com has address
google.com has address
google.com has address
google.com has address
google.com has address
google.com has address
google.com has address
google.com has address
173.194.33.131
173.194.33.132
173.194.33.133
173.194.33.134
173.194.33.135
173.194.33.136
173.194.33.137
173.194.33.142
173.194.33.128
173.194.33.129
173.194.33.130
1.24.20 Records
Note: rfc 1035
A hostname -> IPV4 address
AAAA hostname -> IPV6 address
CNAME like an alias, вЂњGo look up this nameвЂ™s recordвЂќ
PTR
вЂў Pointer to a canonical name, returns name and stops
вЂў Used in reverse DNS
SOA
вЂў Start of Authority for a zone (such as osuosl.org)
вЂў Administrator contact info, timers, serial number
MX Email (more on this next week)
1.24.21 Reverse DNS
Note: http://support.simpledns.com/kb/a45/what-is-reverse-dns-and-do-i-need-it.aspx
Reverse segments, then end with in-addr.arpa
$ host osuosl.org
# could also use dig
osuosl.org has address 140.211.15.183
$ dig 183.15.211.140.in-addr.arpa
...
;; AUTHORITY SECTION:
15.211.140.in-addr.arpa. 10795 IN
1.24. Lesson 10: Networking part 2
SOA ns1.auth.osuosl.org. hostmaster.osuosl.org. 1398356710 300 90
117
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.24.22 Web Apps: A Bit of Review
вЂў We created a python app called Systemview using the Flask framework
вЂў We tested Systemview by running FlaskвЂ™s built-in webserver on the command line
вЂў Systemview ran on a special port we had to open up on the virtual machine
1.24.23 What We Want To Do
вЂў Install a production-quality web server on a standard port
вЂў Serve Systemview using that web server
вЂў Party
1.24.24 Why?
вЂў FlaskвЂ™s web server is not robust or secure
вЂў We want to use standard ports for our web apps
вЂў We may want to run multiple apps on one server
вЂў Web server administration is cool
1.24.25 What is a Web Server
Note: Webserver software, not hardware
1.24.26 Webservers Talk HTTP
They donвЂ™t run code (well, they kinda do)
вЂў PHP, Python, Ruby, C donвЂ™t run in your browser
вЂў Separate servers (usually) run that code, and send the output of the code to the web server to send to your
browser
вЂў Sometimes those separate servers are web server modules
Note: Apache modules generally run in the apache process itself
1.24.27 A Digression: AJAX, JSON and APIs
вЂў Browsers render HTML/CSS (layout)
вЂў Browsers execute Javascipt (logic)
вЂў Javascript can dynamically update the layout
вЂў Javascript can handle user interaction
вЂў Javascript can call back to the server for more data
118
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.24. Lesson 10: Networking part 2
119
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂў Javascript can process data
вЂў Javascript is Client Side Logic
1.24.28 AJAX
вЂў Asynchronous Javascript And XML
вЂў An http request initiated by Javascript
вЂў Javascript listens in the background
вЂў The app sends a response containing data
вЂў Javascript processes the data
вЂў Traditionally XML, but now mostly JSON
Note: Lots of issues around security, javascript calling many servers, gathering data, calling servers outside the
domain of the originating page, etc. Install RequestPolicy and NoScript, just to see who that web page is talking to
while you read it.
1.24.29 JSON
JavaScript Object Notation
{"menu": {
"id": "file",
"value": "File",
"popup": {
"menuitem": [
{"value": "New", "onclick": "CreateNewDoc()"},
{"value": "Open", "onclick": "OpenDoc()"},
{"value": "Close", "onclick": "CloseDoc()"}
]
}
}}
1.24.30 XML
The same text expressed as XML:
<menu id="file" value="File">
<popup>
<menuitem value="New" onclick="CreateNewDoc()" />
<menuitem value="Open" onclick="OpenDoc()" />
<menuitem value="Close" onclick="CloseDoc()" />
</popup>
</menu>
1.24.31 APIs
вЂў When web apps talk to web apps
вЂў When javascript talks to a web app
120
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂў When curl talks to a web app
They talk HTTP, using clearly defined GET or POST params to initiate actions on the remote application.
curl http://graph.facebook.com/12345/friendlists
curl https://api.github.com/users/osuosl/repos
curl http://pub.sandbox.orcid.org/v1.1/0000-0001-7857-2795/orcid-bio
Note: Take a look at the source of a web page, look at all the javascript! How much of it is talking to Google, to
Facebook, etc?
1.24.32 LetвЂ™s Install a Web Server!
yum install httpd
1.24.33 Apache
WhatвЂ™s this httpd thing?
вЂњA patchy web serverвЂќ - born of many patches to NCSAвЂ™s HTTPD (1995)
вЂў Venerable, tested, solid
вЂў Old, complex, slow (not really that slow)
вЂў Many modules for executing code
вЂў Many modules for all kinds of other things too
1.24.34 LetвЂ™s Serve Some Web
ApacheвЂ™s DocumentRoot is the default place where it will look for files to serve. It maps вЂњ/вЂќ in the URL to a location
on disk
http://localhost:8080/index.html
^
"/" is the DocumentRoot
WeвЂ™ll write some HTML in the DocumentRoot for Apache to serve.
1.24.35 But First, Config Files
/etc/httpd/conf/httpd.conf
DocumentRoot "/var/www/html"
<Directory "/var/www/html">
Options Indexes FollowSymLinks
AllowOverride None
Order allow,deny
Allow from all
</Directory>
Note: Just looking, we are not editing the configs here. Note the DocumentRoot and Directory
1.24. Lesson 10: Networking part 2
121
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.24.36 Wait, What am I Writing Again?
HTML: Hyper Text Markup Language
Go to the DocumentRoot and create an html file:
cd /var/www/html
vim index.html
<html>
<head>
<title>This is only a test!</title>
</head>
<body>
<p>Nothing to see here, move along</p>
</body>
</html>
Point your browser to: http://localhost:8080/index.html
Note: HTML, is it code? Is it a language? Can you do logic with it? What happens if you forget the <html>? The
browser does the rendering, the web server doesnвЂ™t care, it just sends the data along. HTTP Content-Type header
says what kind of data.
1.24.37 Voila!
вЂў Apache receives a request for /index.html
вЂў It translates вЂњ/вЂќ into /var/www/html using the DocumentRoot directive
вЂў It looks in /var/www/html for the file вЂњindex.htmlвЂњ
вЂў It finds your file and sends its contents, along with HTTP headers, back to your browser
Note: Have a look at the page source. Edit the file, remove <html>, etc, look at source again. If time allows, use
developer tools, firebug, etc to look at http headers
1.24.38 But I Want to Run Code!
LetвЂ™s put some PHP code in the DocumentRoot:
vim index.php
<html>
<head>
<title>This is only a test!</title>
</head>
<body>
<?php print "Hey, this is PHP!" ?>
</body>
</html>
Then go to http://localhost:8080/index.php
122
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.24.39 What Went Wrong?
Apache doesnвЂ™t know what PHP is, it needs a module to execute the PHP code and return data it can serve
yum install php
service httpd restart
Note: Pop quiz - where do you look to find out what went wrong? Look at log files, talk about them, then look at
page source.
1.24.40 Voila, Again.
How does Apache know what to do with index.php?
/etc/httpd/conf.d/php.conf
<IfModule prefork.c>
LoadModule php5_module modules/libphp5.so
</IfModule>
<IfModule worker.c>
LoadModule php5_module modules/libphp5-zts.so
</IfModule>
AddHandler php5-script .php
AddType text/html .php
DirectoryIndex index.php
Note: CentOS, and most distribution system packages put these conf files for modules in place for you. httpd.conf
includes everything in conf.d - similar for Nginx
1.24.41 Ok, But I Want To Serve a Python App...
ThereвЂ™s a module for that! (Actually several, but we are going to use this one)
WSGI: Web Server Gateway Interface
вЂў Standardized interface for python apps to talk to web servers
вЂў Works with many different servers
вЂў Allows separation of python app and web server processes
Note: talk about mod_python - runs python scripts directly, not bad for single scripts, but unwieldy for applications
and frameworks.
1.24.42 Sounds Great, LetвЂ™s Go!
yum install mod_wsgi
LetвЂ™s clone the systemview app into a reasonable location while we are at it
1.24. Lesson 10: Networking part 2
123
OSU DevOps Bootcamp Documentation, Release 0.0.1
cd /var/www
git clone https://github.com/DevOpsBootcamp/systemview.git
cd systemview
git checkout wsgi
Note: Talk about the location - can be anywhere, but be consistent - /var/www is actually not in the web root, not
accessible by default, donвЂ™t put things under the docroot!
1.24.43 DonвЂ™t Forget Virtualenv!
(in the systemview/ directory)
virtualenv --no-site-packages venv
source venv/bin/activate
pip install -r requirements.txt
And lets make sure everything is owned by the web server:
chown -R apache ../systemview
Note: Web server user/group ownership is a major source of breakage - get cloning/pulling as the wrong user will
change perms on files, possibly breaking things
1.24.44 What Makes an App WSGI?
systemview.wsgi
activate_this = вЂ™/var/www/html/systemview/venv/bin/activate_this.pyвЂ™
execfile(activate_this, dict(__file__=activate_this))
import sys
sys.path.insert(0, вЂ™/var/www/html/systemviewвЂ™)
from systemview import app as application
1.24.45 Configuring Apache for Systemview
/etc/httpd/conf.d/systemview.conf
WSGISocketPrefix /var/run
WSGIDaemonProcess systemview user=apache group=apache threads=5
WSGIScriptAlias /systemview /var/www/systemview/systemview.wsgi
<Directory /var/www/systemview>
WSGIProcessGroup systemview
WSGIApplicationGroup %{GLOBAL}
Order deny,allow
Allow from all
</Directory>
(Look for this in systemview/docs/systemview.conf)
124
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
Note: This will go into a vhost some day
1.24.46 Even More Voila
http://localhost:8080/systemview
There are a lot of steps to getting this app up, wouldnвЂ™t it be nice to automate this?
Note: Future topics - configuration management and automated deploys, virtual hosts, best practices for app location,
Nginx, UWSGI, PHP-FPM, etc
1.24.47 Homework
вЂў Deploy SystemviewвЂ™s master branch with Apache (we merged the database code)
вЂў Read about Apache Virtualhosts
вЂў Install Nginx and UWSGI, deploy Systemview
1.25 Lesson 11: Devops & Configuration Management Intro
1.25.1 Session Summary
вЂў Devops introduction
вЂў Configuration management introduction
1.25.2 Devops Introduction
What is it?
вЂњA software development method that stresses communication, collaboration and integration between
software developers and information technology (IT) operations professionals.вЂќ [Wikipedia]
1.25.3 Definition of Devops
вЂў Software Engineering (Dev)
вЂў Technology Operations (Ops)
вЂў Quality Assurance (QA)
Figure 1.6: Wikipedia (cc)
1.25. Lesson 11: Devops & Configuration Management Intro
125
OSU DevOps Bootcamp Documentation, Release 0.0.1
Figure 1.7: photo by http://thoriseador.deviantart.com/ (CC)
1.25.4 The old view
вЂњDevвЂќ side being the вЂњmakersвЂќ
вЂњOpsвЂќ side being the вЂњpeople that deal with the creation after its birthвЂќ
This siloed environment has created much harm in the industry and the core reason behind creating Devops.
Burn down those silos!
1.25.5 History of Devops
вЂў mid-2000s
вЂњHey, our methodology of running systems seems to still be in a pretty primitive state despite years
of effort. LetвЂ™s start talking about doing it betterвЂќ
вЂў Velocity Conf 2008/2009 - increased presentations on вЂњAgile System AdministrationвЂњ
вЂў Agile 2008 Conf - вЂњAgile InfrastructureвЂќ BOF вЂ“ nobody showed up!
вЂў 2009 DevOpsDays in Ghent, Belgium - Patrick Debois
1.25.6 The Agile Approach
вЂў Iterative, incremental
вЂў Requirements change often thus need to be adaptive
вЂў Very short feedback loop and adaptation cycle
вЂў Quality focus
Manifesto:
вЂў Individuals and interactions over processes and tools
вЂў Working software over comprehensive documentation
вЂў Customer collaboration over contract negotiation
вЂў Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.
126
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.25.7 Adapting Agile to Ops
вЂў Widening the principles towards infrastructure
вЂњInfrastructure as codeвЂќ - i.e. configuration management
вЂў Integrating ops with dev, QA and product in the product teams
вЂў Continuous Integration
вЂњGive your developers a pager and put them on callвЂќ
вЂў Utilizing more specific metric and monitoring schemes
1.25.8 Better Tools enable Devops
Explosion of new tools over the past few years:
вЂў Release tools (jenkins, travisci, etc)
вЂў Config Management (puppet, chef, ansible, cfengine)
вЂў Orchestration (zookeeper, noah, mesos)
вЂў Monitoring & Metrics (statsd, graphite, etc)
вЂў Virtualization & containerization (AWS, Openstack, vagrant, docker)
1.25.9 ItвЂ™s not NoOps
вЂў Existing ops principles, processes and practices have not kept pace
вЂў Business & dev teams need more agility to keep up with competitors
вЂў Deep dev skill set + Deep ops skill set == awesomesauce
вЂў Ops people need to do a little dev
вЂў Dev people need to do a little ops
1.25.10 What is Devops Video
1.25.11 Devops Explained: No Horse Manure
1.25.12 Configuration Management
What is it?
вЂњConfiguration management is the process of standardizing resource configurations and enforcing their
state across IT infrastructure in an automated yet agile manner.вЂќ [PuppetLabs]
1.25.13 History of CM
вЂў mid-1990s вЂ“ вЂњsnowflake systemвЂќ; few systems
вЂў Rise of Unix-like systems and commodity x86 hardware increased the need
вЂў CFEngine вЂ“ First release 1993; v2 released in 2002
1.25. Lesson 11: Devops & Configuration Management Intro
127
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂў mid-2000s through present
вЂ“ More agile CM systems emerged developed with the cloud in mind
вЂў 2008
вЂ“ provisioning and management of individual systems were well-understood
1.25.14 Infrastructure as code
вЂў CM enables ops to define their infrastructure in code
вЂў Install packages, configure software, start/stop services
вЂў Ensure a state of a machine
вЂў Ensure policies and standards are in place
вЂў Provide history of changes for a system
вЂў Repeatable way of rebuild a system
вЂў Orchestrate a cluster of services together
1.25.15 CM Platforms
вЂў CFengine
вЂ“ Lightweight agent system. Manages configuration of a large number of computers using the clientвЂ“server
paradigm or stand-alone.
вЂў Puppet
вЂ“ Puppet consists of a custom declarative language to describe system configuration, distributed using the
clientвЂ“server paradigm.
1.25.16 CM Platforms (part 2)
вЂў Chef
вЂ“ Chef is a configuration management tool written in Ruby, and uses a pure Ruby DSL for writing configuration вЂњrecipesвЂќ. Also a client-server model.
вЂў Ansible
вЂ“ Combines multi-node deployment, ad-hoc task execution, and configuration management in one package.
Utilizes SSH with little to no remote agents.
1.25.17 Puppet Example
вЂў Install apache and start the service
вЂў Puppet Domain Specific Language (DSL)
package { "apache":
name
=> "httpd",
ensure => present,
}
128
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
service {
name
ensure
enable
require
}
"apache":
=> "apache",
=> running,
=> true,
=> Package["apache"],
1.25.18 Chef Example
вЂў Install apache and start the service
вЂў Ruby code
package "apache" do
package_name "httpd"
action :install
end
service "apache" do
action [:enable, :start]
end
1.25.19 CM Platform Comparison
вЂў CFEngine scales like mad, not very agile
вЂў Puppet
вЂ“ Uses a list of dependencies and figures out what order to run it in
вЂ“ The Puppet DSL can become a blocker and a problem, puppet also has scaling issues
вЂў Chef
вЂ“ Executes commands and scripts as they are listed with minimal amount of dependencies
вЂ“ Using ruby offers both its advantages and disadvantages
вЂў Each platform offers its own level of complexity
1.25.20 References
вЂў http://theagileadmin.com/what-is-devops/
вЂў http://itrevolution.com/the-convergence-of-devops/
вЂў http://en.wikipedia.org/wiki/DevOps
вЂў http://en.wikipedia.org/wiki/Agile_software_development
вЂў What is DevOps? - In Simple English (video)
вЂў DevOps Explained: No Horse Manure (video)
1.25. Lesson 11: Devops & Configuration Management Intro
129
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.25.21 Traditional Development Workflow
Scenario: Developer Mary Smith wants to deploy SystemView to a server administered by Ivan Bofh, a strict oldschool sysadmin.
email conversation link
1.25.22 Email #1
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
On April 3, 2013, at 4:22 PM, Mary Smith <msmith@cruftware.com> wrote:
Ops team,
As discussed in the release schedule distributed by Mr. Bossman on 2/5, the
development team is ready to deploy our flagship product SystemView this week.
We will need Python 3.4 an Virtualenv on the production server, as well as a
correctly configured Nginx vhost to direct users to the site.
When we log into the production server to deploy the appвЂ™s code, weвЂ™ll need
permission to write to /var/www and all of /etc for configuration reasons.
Please also create the user and tables detailed in the attached spreadsheet on
our MySql 5.7 database.
Mary Smith
Lead Developer, CruftWare SystemView product division
1.25.23 Email #2
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
On April 5, 2013, at 9:15 AM, Ivan Bofh <ibofh@cruftware.com> wrote:
Mary,
Our production systems are standardized to CentOS 6, so Python is only
supported up to version 2.6. The Python 2.6 version of virtualenv can be
installed after you work with legal to file documentation of a full security
audit of the package.
Providing any account, let alone root, to developers on a production system is
absolutely out of the question. Just document the appвЂ™s deployment process
clearly and weвЂ™ll handle it.
Ivan Bofh
Senior Systems Engineer, CruftWare
1.25.24 Email #3
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
130
On April 5, 2013, at 11:32 AM, Mary Smith <msmith@cruftware.com> wrote:
Ivan,
That sounds like it will be simpler to just install the dependencies directly
on the server instead of using virtualenv. I should be able to include this
in the Jenkins configuration, as long as the CI users is running as root.
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
Speaking of which, the development team will need access to Jenkins or other
continuous integration in order to automatically update the site when changes
are pushed.
Is mysql-dev installed yet? Also please confirm that the database is at
systemview-prod.mysql57.cruftware.com.
Mary Smith
Lead Developer, CruftWare SystemView product division
1.25.25 Email #4
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
On April 6, 2013, at 10:08 AM, Ivan Bofh <ibofh@cruftware.com> wrote:
Mary,
Why do you need to use Jenkins? I Googled it and it looks like a non-standard
and immature implementation of some of CFEngineвЂ™s features. Just send me a
CFEngine configuration file for the settings that you need. The updates can be
done with an SVN post-commit hook.
Due to administrative decisions that Mr. Bossman explained in a company-wide
memo a couple of months ago, absolutely no dev libraries may be installed on
production servers. Servers are for serving, not for compiling.
Ivan Bofh
Senior Systems Engineer, CruftWare
1.25.26 Email #5
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
On April 6, 2013, at 10:14 AM, Mary Smith <msmith@cruftware.com> wrote:
Do you at least have the Nginx vhost and uWsgi installation ready?
Mary Smith
Lead Developer, CruftWare SystemView product division
On April 6, 2013, at 1:53 PM, Ivan Bofh <ibofh@cruftware.com> wrote:
We donвЂ™t use Nginx or uWsgi. The specs should have said to convert the app to
work with Apache and mod_wsgi for production deployment.
Ivan Bofh
Senior Systems Engineer, CruftWare
1.25.27 Email #6
> On April 6, 2013, at 2:37 PM, Mary Smith <msmith@cruftware.com> wrote:
>
> Ivan,
>
> WhatвЂ™s the URL for the database?
>
1.25. Lesson 11: Devops & Configuration Management Intro
131
OSU DevOps Bootcamp Documentation, Release 0.0.1
> Mary Smith
> Lead Developer, CruftWare SystemView product division
>
On April 6, 2013, at 4:22 PM, Ivan Bofh <ibofh@cruftware.com> wrote:
Mary,
YouвЂ™ll have to contact Sharon Negative (snegative@cruftware.com), our DBA, and
file a ticket to get the database access. She wonвЂ™t be back from vacation for
another 2 weeks so it might take awhile.
Ivan Bofh
Senior Systems Engineer, CruftWare
1.25.28 DevOps Workflow
Scenario: DevOps-oriented developer Simon Johnson wants to deploy SystemView to a server administered by Ada
Opdev, a DevOps-oriented sysadmin.
irc conversation link
1.25.29 IRC #1
14:03 < JnomiS> AdaOpdev: hey, all the systemview tests are passing on
python 3.4
14:04 <@AdaOpdev> yay! IвЂ™ll spin up a VM on the cluster with the production
cookbook
14:04 < JnomiS> thatвЂ™ll be at sysview23dev.internal.ourcorp, right?
14:05 <@AdaOpdev> actually weвЂ™re migrating over to a new test cluster
14:05 <@AdaOpdev> could you use sysview23.dev.ourcorp instead?
14:06 < JnomiS> sure
14:06 <@AdaOpdev> itвЂ™s set up for passwordless ssh login with your ldap
account
14:07 < JnomiS> ok, awesome.
14:07 < JnomiS> thanks!
1.25.30 IRC #2
15:12 < JnomiS> hmm, IвЂ™ve been building mysql-python for the app with the
mysql dev libraries, but it doesnвЂ™t look like you have
those in production
15:22 < JnomiS> also, what database should i connect to from the dev
instance? IвЂ™ve been using MySql 5.7 in testing
15:25 <@AdaOpdev> just get me a list of the names and databases youвЂ™ll
need, and IвЂ™ll plug them into our MySql Chef cookbooks.
15:25 <@AdaOpdev> I checked the ORM docs, and youвЂ™re not actually using any
features that changed between MySql 5.5 (which is what
weвЂ™ve got in production right now) and 5.7
15:26 < JnomiS> okay, IвЂ™ll run the tests with mysql5.5
15:27 < JnomiS> everything seems to be working fine locally
132
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.25.31 IRC #3
16:00 < JnomiS> ugh, I totally forgot -- IвЂ™ll need to get root on the dev vm
so I can install uwsgi and nginx
16:00 <@AdaOpdev> We actually manage UWsgi and Nginx through chef as well.
Have you written cookbooks before?
16:02 < JnomiS> nope, IвЂ™ve always just used yours :)
16:05 <@AdaOpdev> http://reiddraper.com/first-chef-recipe/ and
https://www.digitalocean.com/community/articles/how-to-create-simple-chef-cookbooks
are a couple of good places to start
16:05 < JnomiS> thanks!
1.25.32 IRC #4
16:52
16:53
16:54
16:54
16:54
16:55
16:55
16:57
16:57
16:58
< JnomiS> do we have jenkins deployed anywhere yet?
< JnomiS> IвЂ™d like to get continuous integration set up
<@AdaOpdev> IвЂ™ve heard of Jenkins but not worked with it much
< JnomiS> yeah, itвЂ™ll automate that deploy hook mess we used to have
<@AdaOpdev> cool!
<@AdaOpdev> letвЂ™s talk to our boss about getting an instance
provisioned
< JnomiS> okay, IвЂ™ll email him about it and cc you
<@AdaOpdev> thanks
<@AdaOpdev> for now letвЂ™s keep using Chef for everything we can
< JnomiS> okay, sounds good.
1.25.33 Non-DevOps
вЂў Poor communication, territorialism (silos)
вЂў Development environment wildly different from production
вЂў Sysadmin averse to changes because environment is hard to test
вЂў Little willingness to cooperate or educate (trust/teamwork)
вЂў It can get even worse
вЂ“ More people in email thread = more confusion
вЂ“ Bikeshedding about top post vs bottom post
вЂ“ Mary is surprisingly clear about her exact requirements
1.25.34 DevOps
вЂў Developer tests on VM with same config as production
вЂў Realistic expectations about access and compatibility
вЂў More automation, configuration management
вЂў Sysadmin can debug the code and help developer
вЂў Developer and sysadmin help and educate one another (Chef, Jenkins)
вЂў Distributed tasks mean fewer choke points (single person who can block task)
1.25. Lesson 11: Devops & Configuration Management Intro
133
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.26 Lesson 12: Configuration Management
Note: starting off with cs312 content
1.26.1 Large site management
вЂў Configuration Management
вЂў Automation
вЂў Centralized
вЂў Standardization
вЂў History
Note:
вЂў Automating most of what you do
вЂў Scaling the configuration management is key
вЂў Centralized so that a team can work together
вЂў History via SCM (source control management)
1.26.2 What config management does
вЂў Verify permissions for security/mgmt purposes
вЂў Distribute config files and scripts
вЂў Control batch jobs
вЂў Ensure packages are up to date
вЂў Look for file changes
Note: Ensures that critical configs always are set like you want. No mucking around by a bad admin
1.26.3 Configuration Management
вЂў System-wide settings
вЂ“ sshd, ntp, ldap, etc
вЂў Group settings
вЂ“ Web, email, database
вЂў Standardize settings globally
вЂў Easy to troubleshoot
134
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.26.4 CF Management Tools
вЂў CFEngine
вЂ“ Mature, widely used
вЂў Puppet
вЂ“ Getting mature, different concept
вЂў Chef
вЂ“ Very new
Note: Different concepts Feature set different between them Each have their own issues
1.26.5 Puppet
LetвЂ™s install puppet
$ yum install puppet
1.26.6 Declarative Configuration
вЂў We вЂ�declareвЂ™ the desired state of the system
вЂў Puppet does the necessary work to make the system match our declaration
вЂў We can save these declarations in a repository, just like code
1.26.7 Puppet Resources
Puppet knows about вЂњresourcesвЂќ on the system
$ puppet describe -l
Look at all those things Puppet can manage out of the box. WeвЂ™re most interested in these:
file
group
package
service
user
-
Manages files, including their content, owner ...
Manage groups
Manage packages
Manage running services
Manage users
1.26.8 Resources Have Attributes
# letвЂ™s look at the vagrant user
$ sudo puppet resource user vagrant
user { вЂ™vagrantвЂ™:
ensure
gid
groups
home
password
=>
=>
=>
=>
=>
вЂ™presentвЂ™,
вЂ™500вЂ™,
[вЂ™wheelвЂ™],
вЂ™/home/vagrantвЂ™,
вЂ™$1$aDsSD/Uu$.tXG5wN.TSit1AP5ZyphB0вЂ™,
1.26. Lesson 12: Configuration Management
135
OSU DevOps Bootcamp Documentation, Release 0.0.1
password_max_age
password_min_age
shell
uid
=>
=>
=>
=>
вЂ™99999вЂ™,
вЂ™0вЂ™,
вЂ™/bin/bashвЂ™,
вЂ™500вЂ™,
}
We can declare a value for any of those attributes, and Puppet will make it happen.
Note: the password is a password hash, as appears in /etc/shadow - donвЂ™t put passwords in puppet manifests!
1.26.9 Puppet Manifests
Puppet keeps its declarations in manifest files. We can write a manifest to create a user:
$ sudo su $ vim users.pp
user {вЂ™yournamehereвЂ™:
ensure
=> вЂ™presentвЂ™,
home
=> вЂ™/home/yournamehereвЂ™,
groups
=> [вЂ™wheelвЂ™, вЂ™vagrantвЂ™],
shell
=> вЂ™/bin/tcshвЂ™,
}
1.26.10 Pull the Strings
Lets run our manifest.
> puppet apply user.pp
Notice: Compiled catalog for devops-bootcamp.osuosl.org in
environment production in 0.12 seconds
Notice: /Stage[main]/Main/User[yournamehere]/ensure: created
Notice: Finished catalog run in 0.13 seconds
Note: we are using stand-alone mode, manually running an individual manifest
1.26.11 Declarations Are Idempotent
Lets run our manifest again.
> puppet apply user.pp
Notice: Compiled catalog for devops-bootcamp.osuosl.org in
environment production in 0.12 seconds
Notice: Finished catalog run in 0.02 seconds
The state of the system is already what we declared it should be, so applying the manifest again doesnвЂ™t change
anything.
Note: idempotency is important, the puppet master daemon will run periodically, and it is important that running the
same commands over and over does not have cumulative effects
136
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.26.12 Packages and Services
We can declare that our system should have certain things installed and running.
apache.pp:
package{вЂ™httpdвЂ™:
ensure => вЂ™presentвЂ™
}
service{вЂ™httpdвЂ™:
ensure => вЂ™runningвЂ™,
enable => вЂ™trueвЂ™,
require => Package[вЂ™httpdвЂ™],
}
Note: The вЂ�serviceвЂ™ block makes sure that the httpd service is started, and that it is enabled, the вЂ�requireвЂ™ directive
tells the service that it must wait until the package вЂ�httpdвЂ™ is processed. Services are anything you would start with
вЂњservice x startвЂќ and packages anything you would install with вЂњyum install xвЂќ
1.26.13 Puppet Config
Where does Puppet keep its configuration files?
Note: the audience really ought to know where to start looking by this point
1.26.14 /etc/puppet
$ ls /etc/puppet
auth.conf modules
puppet.conf
вЂў puppet.conf - systemwide configuration
вЂў auth.conf - puppet agent configuration
вЂў modules - weвЂ™ll talk about that later
Note: there isnвЂ™t much of anything we need to worry about in any of the config files
1.26.15 The Site Manifest
We
want
to
move
beyond
running
individual
manifests
on
вЂ�/etc/puppet/manifests/site.ppвЂ� is the place to put your siteвЂ™s configuration.
the
command
line.
$ mkdir /etc/puppet/manifests
$ vim /etc/puppet/manifests/site.pp
1.26.16 But First, Nodes
вЂў Nodes are defined in the site manifest
вЂў A node is a single machine, identified by its FQDN (Fully-Qualified Domain Name).
1.26. Lesson 12: Configuration Management
137
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂў You can define many nodes.
вЂў You can add declarations to a node definition.
вЂў A special вЂ�defaultвЂ™ node will be used if a nodeвЂ™s name canвЂ™t be found.
We will put our configurations in the default node for now.
Note: a node can inherit from another node, but this is discouraged
1.26.17 An Example Site Manifest
node default {
file {вЂ™/etc/issueвЂ™:
path
=> вЂ™/etc/issueвЂ™,
mode
=> 644,
ensure => present,
content => "Welcome to the DevOps Bootcamp VM.\n",
}
package{вЂ™httpdвЂ™:
ensure => вЂ™presentвЂ™
}
service{вЂ™httpdвЂ™:
ensure => вЂ™runningвЂ™,
enable => вЂ™trueвЂ™,
require => Package[вЂ™httpdвЂ™],
}
}
Note: have we talked about /etc/issue? The file resource lets you declare the filename, ownership, and contents. You
can also have it copy files from the module onto the node instead of manually inserting content here.
1.26.18 The Master and the Agent
Puppet uses a Master/Agent architecture.
вЂў The Master reads the вЂ�site.ppвЂ� and listens for an Agent to contact it.
вЂў Agents run on nodes, they contact the master to get their configuration
вЂў Master and Agent can be on the same machine.
вЂў When they are on different machines, they need an SSL certificate to authenticate
Run the master on your vm:
$ puppet master
Note: the master will background by default and log to syslog, but you can run it in the foreground with вЂ“nodaemonize and get extra logging on stdout with вЂ“verbose
138
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.26.19 The Agent
The agent will look for its master on the host вЂ�puppetвЂ� by default. Lets add the hostname вЂ�puppetвЂ� to our local
host definition in /etc/hosts, so it will look on the local machine.
$ vim /etc/hosts
127.0.0.1
devops-bootcamp.osuosl.org devops-bootcamp localhost
localhost.localdomain localhost4 localhost4.localdomain4 puppet
^^^^^^
Now run the agent in test mode:
$ puppet agent --test --verbose
Note: the agent will also background by default, the вЂ“test flag prevents that and shows us what is going on. In a
production environment, the master and agent would always be running in the background, usually started as services
on boot.
1.26.20 Modules
We can keep adding configurations to site.pp, but itвЂ™s going to get long and messy. LetвЂ™s use modules instead.
вЂў Modules are classes
вЂў Modules encapsulate a set of related configurations
вЂў Modules make it easy to apply configurations to many nodes
вЂў Community created modules already exist for almost everything
Note: community or puppetlabs modules vary in quality, always read the docs thoroughly
1.26.21 Module Structure
/etc/puppet/modules/
modulename/
files/
some_file
manifests/
init.pp
some_other_manifest.pp
Note: that files directory is served to the puppet agent like a fileserver, file resources can declare their source attribute
like вЂњpuppet:///modules/module_name/some_fileвЂќ and the file will be copied into place
1.26.22 The Bootcamp Apache Module
#
$
$
$
$
LetвЂ™s create a module for our Apache configuration.
cd /etc/puppet/modules
mkdir bootcamp_apache
mkdir bootcamp_apache/manifests
vim bootcamp_apache/manifests/init.pp
1.26. Lesson 12: Configuration Management
139
OSU DevOps Bootcamp Documentation, Release 0.0.1
class bootcamp_apache {
package{вЂ™httpdвЂ™:
ensure => вЂ™presentвЂ™
}
package{вЂ™mod_wsgiвЂ™:
ensure => вЂ™presentвЂ™
}
service{вЂ™httpdвЂ™:
ensure => вЂ™runningвЂ™,
enable => вЂ™trueвЂ™,
require => Package[вЂ™httpdвЂ™],
}
}
Note: it is good practice to namespace the class name of your modules, so instead of just вЂ�apacheвЂ™, we use bootcamp_apache, which wonвЂ™t collide with any other apache related module.
1.26.23 Site.pp Modularized
node default {
file {вЂ™/etc/issueвЂ™:
path
=> вЂ™/etc/issueвЂ™,
mode
=> 644,
ensure => present,
content => "Welcome to the DevOps Bootcamp VM.\n",
}
include bootcamp_apache
}
Note: the include statement assumes a module located in modules/ under the pupper config dir. The name is the class
name of the the module, which is not necessarily the directory name the module is stored under (but it is much easier
to name them the same)
1.26.24 Community Modules
We need MySql installed for our SystemView app, as well as a database, user, and permissions. We could do all that
with package, service and file resources, but there is a better way, the puppetlabs-mysql module.
https://github.com/puppetlabs/puppetlabs-mysql
(ItвЂ™s in Git, how convenient!)
$ cd /etc/puppet/modules/
# WeвЂ™ll clone into a directory named mysql, because thatвЂ™s the module name
$ git clone https://github.com/puppetlabs/puppetlabs-mysql.git mysql
We can include this moduleвЂ™s class into our site manifest or our own modules.
1.26.25 The Bootcamp Mysql Module
We want to create a database and users, so lets make a module and not clutter up the site.pp
140
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
$
$
$
$
cd /etc/puppet/modules
mkdir bootcamp_mysql
mkdir bootcamp_mysql/manifests
vim bootcamp_mysql/manifests/init.pp
class bootcamp_mysql {
class { вЂ™::mysql::serverвЂ™ }
}
::mysql::server causes Puppet to install MySql and makes available many methods for managing MySql.
Note: Calling the вЂ�mysqlвЂ™ class essentially includes that module, which includes a package declaration insuring mysql
is installed. It is easy to explore the module files and see what is in it.
1.26.26 Databases, Users, and Grants
class bootcamp_mysql {
class { вЂ™::mysql::serverвЂ™ }
mysql_database { вЂ™systemviewвЂ™:
ensure => вЂ™presentвЂ™,
charset => вЂ™utf8вЂ™,
collate => вЂ™utf8_swedish_ciвЂ™,
}
mysql_user { вЂ™vagrant@localhostвЂ™:
ensure => вЂ™presentвЂ™,
}
mysql_grant { вЂ™vagrant@localhost/systemview.*вЂ™:
ensure
=> вЂ™presentвЂ™,
options
=> [вЂ™GRANTвЂ™],
privileges => [вЂ™ALLвЂ™],
table
=> вЂ™systemview.*вЂ™,
user
=> вЂ™vagrant@localhostвЂ™,
}
}
Note: the mysql module has a lot of stuff in it, there isnвЂ™t time to get into it all.
1.26.27 Test It Out
$ puppet agent --test --verbose
1.26.28 Further Reading
вЂў http://docs.puppetlabs.com/learning/introduction.html
вЂў https://github.com/puppetlabs/puppetlabs-mysql
1.26. Lesson 12: Configuration Management
141
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.27 Lesson 13: Contributing to Open Source
1.27.1 Getting started in Open Source
вЂў CarlosвЂ™s CS419 in a nutshell
вЂў Relevant books
вЂ“ The cathedral and the bazaar (Eric Raymond)
вЂ“ Producing Open Source Software (Karl Fogel)
вЂ“ Open Advice (Lydia Pintscher)
1.27.2 Joining vs Starting a Project
вЂў Scratch an itch.
вЂў Research first
вЂў Revive dead project vs. rewrite
вЂў Get involved with communities even if you plan to start your own
вЂ“ Learn from their examples
1.27.3 Know your licenses
Note: I am not a lawyer.
http://opensource.org/licenses is pretty cool
вЂќ Freely used, modified, and sharedвЂќ
MIT/X11 Short, permissive, says attribution and no liability. DoesnвЂ™t discuss copyright. Can convert to
Apache 2
Apache Short, permissive, goes in every file, grants patent rights from contributors to users, author keeps
copyright. Plays nice with GPL3 (?)
BSD Attribution, keep copyright, no liability
AGPL Demands source distribution even when software not distributed (for cloud/hosted)
GPL Viral, copyleft. Viral = infects entire program if it links to GPL library or uses a single line of
GPLвЂ™d code
LGPL Fixes library linking issue with GPL
CC Non-code content
вЂў MIT/X11
вЂў Apache
вЂў BSD
142
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂў AGPL/GPL/LGPL
вЂў Creative Commons
вЂў http://choosealicense.com/
1.27.4 Assessing a new community
вЂў Elitism vs welcomeness
вЂў Communication style
вЂў Documentation and guides
вЂў Faster or slower change?
1.27.5 Getting involved
вЂў Lurk more
вЂў Make accounts
вЂ“ Mailing lists
вЂ“ IRC channels
вЂ“ Wikis
вЂ“ Issue trackers
вЂў Your nick is your reputation
вЂў ItвЂ™s okay to make mistakes... But learn from them.
1.27.6 Finding a project
Note: First contributions will be metric of how nice they are to newbies
1.27. Lesson 13: Contributing to Open Source
143
OSU DevOps Bootcamp Documentation, Release 0.0.1
144
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
ThereвЂ™s a thing where older project members get grumpy at newbies because theyвЂ™ve answered the question over and
over... read docs/faq then improve them
вЂў Openhatch
вЂў Easy bugs
вЂў GSOC submitters who didnвЂ™t get enough interns
вЂў Search by language
вЂў Search by project type вЂ“ find something that interests you (web dev? bioinformatics? video games?)
вЂў Your immediate payment for contributions will be satisfaction, so pick something satisfying
1.27.7 First steps
Note: It will feel like you have only a vague idea what youвЂ™re doing. This means youвЂ™ve found a project thatвЂ™s
challenging and that youвЂ™ll learn from.
вЂў Lurk awhile then ask
вЂў Write a test
вЂў Fix a typo
вЂў Deploy and update the installation docs
1.27.8 DevOps Concerns
вЂў Configurations often managed in public repos
1.27. Lesson 13: Contributing to Open Source
145
OSU DevOps Bootcamp Documentation, Release 0.0.1
146
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂў Root canвЂ™t be handed out to just anyone
вЂў Build trust, contribute to project consistently
вЂў Practice with the tools they use
1.27.9 Your Homework
вЂў Find a project that youвЂ™d like to get involved with this summer
вЂў Join IRC, mailing lists, etc.
вЂў Pull the code and run its tests using what youвЂ™ve learned
вЂў Find something you can contribute to the project
вЂў Discuss how itвЂ™s going in #devopsbootcamp on irc.freenode.net
1.27.10 Questions?
Any questions about anything from this year?
вЂў Conferences: OSBridge, OSCON may have free expo hall passes
вЂў In Corvallis? Want to come to the OSL and see what we do, pair program, etc.?
вЂў No meeting next week вЂ“ please leave feedback!
1.28 Lesson 14: Review
1.28.1 TodayвЂ™s Goals
вЂў Working VM
вЂў Run Systemview app
вЂў Connect to Systemview from browser in host
вЂў Understand whatвЂ™s happening
1.28.2 How to set up a VM
1.28.3 Enable VirtualBox (VT extensions)
Install VirtualBox
Install Vagrant
1.28.4 Linux
# clone
git clone https://github.com/DevOpsBootcamp/devopsbootcamp-vagrant.git
# start up
cd devopsbootcamp-vagrant
1.28. Lesson 14: Review
147
OSU DevOps Bootcamp Documentation, Release 0.0.1
vagrant up
# access vm
vagrant ssh
1.28.5 IRC
вЂў Use the LUG guide
вЂў #devopsbootcamp on irc.freenode.net
вЂў Also join the Mailing List
1.28.6 GitHub Account
SSH keys
ssh-keygen -t rsa
Your public key is in ~/.ssh/id_rsa.pub by default.
GitHub -> Account Settings (icon in upper right) -> SSH keys -> Add SSH key
1.28.7 Joining the Github Org
вЂў Ping Ramereth in the IRC channel to add you
1.28.8 Installing the Web App
1.28.9 Prerequisite tools
On host machine, in devopsbootcamp-vagrant directory:
git pull
vagrant up
vagrant ssh
1.28.10 The Magic Script
Now that youвЂ™re in the guest machine:
git@github.com:DevOpsBootcamp/catch-up.git
cd catch-up
./catch-up.sh
1.28.11 Branches
Start one with
148
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
git checkout -b branchname
Which are you on
git branch
Switch with
git checkout branchname
1.28.12 Connecting to the App
In the guest machine, with virtualenv activated, python systemview.py
Point the browser of your host machine at http://127.0.0.1:5050
If changes in the app donвЂ™t show up in your browser, use F5 to hard refresh
1.28.13 Where am I?
1.28.14 In virtual machine?
вЂў Did you vagrant ssh?
1.28.15 In a repo?
вЂў git status
1.28.16 On a branch?
# Show current branch
$ git branch
# create new branch, called branchname
$ git checkout -b branchname
1.29 Lesson 16: Email
вЂў Email service
вЂ“ How it works
вЂ“ Configuration Postfix
вЂ“ Planning
1.29. Lesson 16: Email
149
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.29.1 Email: System Components
вЂў Mail User Agent (MUA)
вЂў Mail Transport Agent (MTA)
вЂў Delivery Agent (MDA)
вЂў Access Agent (MAA)
Note:
MUA
вЂў lets users read & compose mail
вЂў Thunderbird, mutt, etc
MTA
вЂў routes messages to other machines
вЂў sendmail, postfix, exim, qmail
MDA
вЂў places messages in local store
вЂў mail.local, procmail
MAA access to mail store (i.e IMAP, POP)
1.29.2 Email: System Components
Note: The most confusing part about email is understanding the routing. Knowing the different components is
important to fully grasping it.
150
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
1.29.3 Transport Agents
Accept mail form user agent
Postfix More common, easier to configure & use
Sendmail Highly configurable, steep learning curve
Exim Similar to Postfix
Qmail Logging is horrid, but some people like it
Note: Postfix is the easiest to learn and understand, but queue management is a вЂњblack boxвЂќ
Sendmail & qmail is great for high volume sites, but postfix/exim still perform great.
Sendmail has great options for queue management
Features to look out for:
вЂў SASL (authenticated SMTP)
вЂў Queue Management
1.29.4 Delivery Agents & Message Stores
вЂў procmail вЂ“ great filtering
вЂў maildrop вЂ“ newer procmail-like
вЂў mail.local
вЂў Message Stores
вЂ“ mbox вЂ“ one large file, locking problems
вЂ“ maildir вЂ“ one file per message, great for IMAP
Note: Consider scaling issues for the mailstore.
Generally maildir is the best & most compatible option
1.29.5 Anatomy of a Mail Message
вЂў Envelope
вЂ“ Destination email address
вЂў Headers
вЂ“ Record of variety of important information
вЂ“ Great for tracking down problems
вЂў Body of the message
Note:
Headers:
вЂў Know how to identify and track queue idвЂ™s
вЂў Originator starts at the bottom
1.29. Lesson 16: Email
151
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂў Headers can be forged
вЂў X- Headers non-RFC headers
вЂў Message ID is always unique
1.29.6 MTA Log Files
вЂў Track emails via queue ID
вЂ“ Look something like: 03CE18819A
вЂў Tracking via message ID
вЂў Informational fields
вЂ“ to, from, status, relay, etc
вЂў Log files differ between each MTA
Note: Being able to read log files is important.
1.29.7 Configuring Postfix
вЂў /etc/postfix
вЂ“ main.cf вЂ“ main config file
вЂ“ master.cf вЂ“ postfix process config file
вЂ“ /etc/aliases вЂ“ local email forwarding
вЂў Set to relay email to central MTA
вЂ“ relayhost = [smtp.osuosl.org]
вЂ“ myorigin = osuosl.org
вЂ“ /etc/aliases вЂ“ root: foo@gmail.com
Note:
relayhost: [smtp.osuosl.org] vs. osuosl.org
вЂў [smtp.osuosl.org] goes directly to smtp.osuosl.org
вЂў вЂ�osuosl.orgвЂ™ does DNS lookup and uses MX
Make sure you run вЂњnewaliasesвЂќ after updating /etc/aliases
Reloading postfix is ideal too
To test email: echo вЂњthis is a testвЂќ | mail root@localhost
1.29.8 Sendmail
вЂў Config files created via m4
вЂ“ Makefile
152
Chapter 1. Contents:
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂў Always edit the .mc files not the .cf files
вЂў Remember to rebuild .cf files with make
вЂў Extremely configurable
Note: Config files in /etc/mail usually Primary file to edit should be sendmail.mc
1.29.9 Email: Viruses & Spam
вЂў Virus
вЂ“ Clamav
вЂ“ Ensure freshclam is running too
вЂў Spam
вЂ“ Spamassassin
вЂў All-in-one
вЂ“ Amavis
вЂў Check abuse emails
Note: Make sure you have enough CPU & RAM for Spam checking Neglecting abuse emails may get you blacklisted
For larger infrastructures, have dedicated machines to process spam Important to keep these updated
1.29.10 Email: Infrastructure Implementation
вЂў Small sites
вЂ“ Can have MTA/MDA/etc all on the same server
вЂў Medium sites
вЂ“ Separate MTA from MDA
вЂў Large sites
вЂ“ Split outgoing mail and incoming
Note: Consider resources, redundancy, & scalability. MDA is hardest to scale.
вЂў Look at Cyrus Murder for large scalability
вЂў dovecot is another option
1.29.11 Email: Security
вЂў On General servers:
вЂ“ Only listen on localhost
вЂ“ DonвЂ™t allow other hosts to relay through it
вЂ“ Relay all outbound mail through central host
1.29. Lesson 16: Email
153
OSU DevOps Bootcamp Documentation, Release 0.0.1
вЂў On Email servers:
вЂ“ Restrict relaying to trusted networks
вЂ“ Implement antivirus & spam protection
Note: Always test new configurations to ensure spammers canвЂ™t relay mail through your server Having dedicate
outbound servers will ensure they always catch spam/viruses/etc
154
Chapter 1. Contents:
CHAPTER 2
Indices and tables
вЂў genindex
вЂў modindex
вЂў search
155
OSU DevOps Bootcamp Documentation, Release 0.0.1
156
Chapter 2. Indices and tables
Bibliography
[Wikipedia] http://en.wikipedia.org/wiki/DevOps
[PuppetLabs] http://puppetlabs.com/solutions/configuration-management
157

Download Report