Malcolm McRoberts's List: DevOps - Provisioning

• I’m not sure if you were doing this for a different reason, but there is a way to do automatic ssl signing with puppetmaster. You just need a file called autosign.conf in /etc/puppet. You whitelist FQDNs that you’ll auto sign for. It accepts wildcards, too. I hope that saves you some work!

• But you can also classify nodes based on a pre-existing external data source, like an LDAP database or a set of flat files describing your infrastructure.

• node_classifier
• node_classifier[8989]: DEBUG: Node configuration (53b0b7bf-723c-4a0f-b4b1-082ebec84041): classes: ['hadoop::namenode']

• In 2008 our software creation and deployment process we went from 38 documents and gates to well over 40. Yay, moving backwards! In the DoD releasing software requires multiple levels of tests and approval, all within distinct organizations that don’t much communicate.
• • Forge.mil is for government open source projects. They look to create community tools and foster a bit of colaboration. The vice chairmen of the cheifs of staff supports this project.
   
  • Race is a government program for Cloud resources and accreditation assistance.
   
  • Testforge.mil is another project that loks toward improving system provisioning, automated workflows, and test as a service.
• CONS3RT is a library of all the assets that you have from dev to operational deployment. Assets can be things like servers, databases, monitoring tools, etc…

• Lessons learned
   ——–

   
   
  • Automation is huge.
   
  • Testing early and often. It is easier to fix the engine when it on the assembly line then when it is on the dealers lot. Simple concept. Virtualization and CI allow you to test often with minimal barrier to doing so.
   
  • Leanness. This goes back in part to the process stuff. It is a matter of keeping the process simple
   
  • Flexibility, team trust and commitment. We fall off the wagon sometimes. We need to be able to call eachother out on things so we can keep improving. We need to trust that we will stay committed and on track when thing fail – which they will.
   
  • Persistence. Stay the course when people push back on you. Managent, peers, and others.
   
  • Trust, with customers and sponsors.
   
  • Top cover, someone over the top that can allow you to be persistent and help cover challenges from further above.

2 more annotations...

• you aren't going to treat those virutal servers as precious and unique snowflakes. That means you are going to need configuration management and automation
• your large scale applications need to be "fault-tolerant, and/or self-healing."
• This means that sysadmin skills need to be applied to development and the application needs to be designed alongside the infrastructure.

1 more annotation...

• “The cloud is not about technology, but about a new operations model for software and hardware, which makes DevOps possible,”
• Knowledge of programming, APIs and protocols is required for load distribution and that drives good cloud economics.
• The DevOps toolkit includes configuration management tools like Puppet, Chef and cfengine; indexing tools like Splunk; and scripting languages like Perl and JavaScript.
• Computing in the cloud requires system administrators to manage thousands of servers at once, so there’s a growing need for APIs that can help manage infrastructure

• An OpenFlow-based, software defined network would make networks more programmable, enabling networking teams to collaborate more directly with DevOps teams.

    

  “It’s a key part of how we see networking evolve over time, with DevOps automating policy in the network,” said Shah.

• How can network engineers work with a DevOps movement?

    

  As DevOps evolves, networking pros must learn scripting, as well as configuration and deployment tools.

4 more annotations...

• The theory of DevOps is predicated on the idea that all elements of a technology infrastructure can be controlled through code. Without cloud that can’t be entirely true. Someone has to back the servers into the data center and so on.
• everything becomes repeatable.
• You take the repeatable elements of Dev and apply them to Ops. Starting a server becomes a repeatable testable process.
• Scalability; this is another thing you get with DevOps + Cloud.
• DevOps + Cloud is self healing.
• ou can detect VM failures and automagically bring up a replacement VM for example.
• Continuous integration and deployment.
• We are running CI and delivering code continuously not just at the application level but at the infrastructure level when you fully go DevOps and Cloud.
• The Cloud platform has 2 key components to it that turn virtualization into Cloud. One of them is locational transparency.
• Configuration management is one of the most fundamental elements allowing DevOps in the cloud.

8 more annotations...

• Primarily we’ll use it as a means of programmatic execution of Systems Administration actions on clusters of servers.

• Jesse Robins describes the goal brilliantly:

   

  “Enable the reconstruction of the business from nothing but a source code repository, an application data backup, and bare metal resources”
• One of the principles of Extreme Programming that I find both challenging and elegant is the principle of the simple design. When writing software we should seek to Do The Simplest Thing That Could Possibly Work (DTSTTCPW).

• "Infrastructure as code". I love the phrase. Where devops is a word that is sadly open to so much (mis)interpretation, "Infrastructure as code" is pretty clear. Treat your infrastructure as code. Programmable. Testable. Deployable.

• [Y]ou take your software to the wall that separates development and operations, and throw it over and then forget about it. [...] Giving developers operational responsibilities has greatly enhanced the quality of the services, both from a customer and a technology point of view.[…] You build it, you run it.

   

• First and foremost, build/deploy instructions provided in textual form can’t be easily consumed by a machine
• Secondly, because they are not fully formalized, they can’t be as easily shared
• Thirdly, since they are not code, they are not as easy to test and verify test coverage for, or to fork and merge, or to port to other platforms.

1 more annotation...

• Engineers handle entire features themselves, from JavaScript to Database code. Engineers are also responsible for testing, bug fixes, and post-launch maintenance of their own work. There is QA at Facebook, just not as an official team. In effect, there is mandatory code review for all changes.