Blog Archive

If Corona Were the Flu
Trying to understand the Corona virus and how it is affecting society is an exercise in frustration. Sifting through all of the news, blogs, tweets, gossip to get to accurate and useful data is hard. We are getting told "Social Distance", "Stay at Home", "Flatten The Curve". But Read More ›

AWS MySQL Choices: MySQL, RDS, Aurora oh my
My migration to AWS is to the point where I have to decide on what database to use. Or should I say what version of a database to use. Our current DB is MySQL, so I want to stick with compatible options. And there are three: self-managed MySQL, RDS MySQL, RDS Aurora. Which to choose? Read More ›

Can SaltStack do AWS, what's in a name
Two days of aws-formuala being live and I found it was broken. Because of names. More precisely the collision of of names in different VPCs. Aren't VPCs completely independent? Well yes, except for names it appears. The normal methodology for a salt state is to check if something needs to be updated and then update it if it does. It appears that this methodology is breaking down in the AWS state/module. I think this is partly due to how AWS is structured and part to difficulties with the boto module. Read More ›

Can SaltStack do AWS, the VPC
We started on Rackspace back when it was still trying to compete with AWS in the IaaS. As a startup, Rackspace worked just fine. But if a company grows and needs more and their IaaS vendor is not growing their offering, there comes a tipping point. Security was our tipping point. We could not get the security we needed without either expensive dedicated hardware or a way to convoluted network architecture. And so we are moving to a new IaaS vendor, AWS. To get started at AWS I created a VPC by hand: subnets, routing tables, gateways, security groups. Add in a salt server and I am able to use Salt to spin up and manage my AWS servers. Read More ›

Can Ops use Dev Techniques Part 3, Deployment Workflow
The last piece of the development process I want to look at is the Development Workflow. What is the process for getting the code from the developer and onto a physical server where customers can use it. Generally speaking companies have a set of environments that code will flow through before it gets into production. Lets keep this simple with three environments Read More ›

Can Ops use Dev Techniques Part 2, Code Workflow
Another piece of the the development process the the code workflow. We have bunch of developers writing code with the goal of providing new and hopefully useful features to our customers. So we need a way to organize how all these separate pieces of code get merged into that finished product. Read More ›

DevOps, Plums and Apricots or Pluots. Can Ops use Dev Techniques?
Is DevOps shoving plums and apricots into a bowl and pretending they are the same thing or is it breeding something new and giving it a new name like pluot? When DevOps was coined in 2009, it was a label for a bottom up movement: people in the IT trenches trying to figure out how to make things work better. Agile programming had changed development and Ops was having trouble keeping up. In the last 6 years the term has definitely gone corporate but retains its roots in the trenches. So as a trench dweller I have to wrestle with the practicalities of DevOps every day. Today I am wanting to see if the DevOps theory that Ops can use the same processes as Dev really works in practice. Read More ›

Chatops at 6 months. How has it changed development. Getting There
Using ChatOps to move from Ops-Serve to Self-Serve obviously took some technical work. But as stated before I am focusing on the developer in this series, so I want to look at what it took to transition the development team to using this new tool. Read More ›

Chatops at 6 months. How has it changed development. After
In my last post I looked at where the development process was before ChatOps was discovered and implemented. My how things have changed in these six months. Developers can do a lot more now. Read More ›

Chatops at 6 months. How has it changed development. Before
Yesterday one of my developers created a server environment to test a feature branch and had a problem: one of the 4 servers in the environment did not get built. A short investigation showed a typo on the ChatOps command was the cause. Read More ›

DevOps Solo. Oh, The places you'll go
It was unexpected and a little ironic that the quote I found that best sums up my DevOps experience is from Dr. Seuss. Read More ›

Saltstack expanding includes to remove data redundancy
Last post I created a pillar merge strategy that allowed data at different scopes and include it on all the servers relevant to that scope. After implementing this strategy and using it for a while I started having the needing for the same data in multiple different pillars. Read More ›

SaltStack and a well formed hostname prevents merge chaos
The pillar merge used in the last post allowed us to have global data and environment specific data. Now I want to add in another: datacenter specific data. Read More ›

Saltstack and a well formed hostname keeps your environment data organized
Previously the well formed hostname has been used to: 1) map servers to roles 2) map application servers sharing roles to the appropriate role and to the application specific data needed to customize each server 3) map servers to salt environments Read More ›

Saltstack and a well formed hostname keeps environments clean
The well formed hostname in my last post helped connect servers to the appropriate pillars/roles while keeping the top file simple. One part of the well formed hostname used, three to go. Read More ›

Saltstack and a well formed hostname simplifies your top
When I first started out using SaltStack, I setup a role based configuration where each server has one or more roles and those roles contain everything required to setup the server. There is the **common** role which as you might expect from its name is applied to all server. Read More ›

Becoming Hubot or How I make Hubot look really smart
When my teammates talk to me about problems with Hubot I learn about what is and is not working well, which often leads to code updates to make Hubot easier to user. Recently my conversations are indicating teammates are having difficulty with getting the hubot syntax correct. Read More ›

Trust but verify? Maybe, but better to not trust and sanitize
It would be nice if the old Regan slogan, which is an even older Russian proverb, was sufficient for our chatops security mantra. I would like to be able to always trust the system's users, especially because chatops users are trusted employees. Read More ›

Pencils and Pens will be Illegal
As a citizen of the United States of America and a technology worker I am of course interested in Dianne Feinstein and Richard Burr's recent bill **Compliance with Court Orders Act of 2016**, particularly since Dianne Feinstein is my Congresswoman. While I appreciate the efforts of our Government to protect citizens, I am concerned what affect this bill will have on the US. Read More ›

Security, Security, More Security
The security put in place last time was not enough for me. While it provides that no one can affect production servers except those in the hard-coded Hubot admin group, I don't like that anyone who has a slack account can affect the test environment. Read More ›

Hubot and Slack Security concerns
The last post used middleware to restrict what room commands can run it. But there is still no restrictions on who can use a command and what server they can run it on. Read More ›

Using Hubot Middleware to Restrict Rooms
Today's post is a small diversion from the security topic I planned because something came up that I wanted to address first ... where hubot could be accessed from Read More ›

Hubot Global Functions
Researching global functions in Hubot lead me down two interesting paths: Classes and Middleware. Middleware is pretty well documented on the Hubot Scripting page. Classes on the other hand not so much Read More ›

Hubot Carbon Copy a Channel
My pondering on how to clearly log Hubot initiated saltstack changes to a Slack channel without having a bunch of **help** and **pug me** in the channel lead to developing a method to carbon copy (cc) Read More ›

Hubot Saltstack Integration
Before finding ChatOps our strategy was to create a salt-api in node.js that exposed just the needed parts of Saltstack via a RESTful API that returns JSON. This would allow a developer to integrate Saltstack with our internal management tools with minimal understanding of salt Read More ›

Chatops and Hubot the basics
Hubot was really not hard get started with. I started with it on my MacOS laptop. I already had these dependencies installed: node.js 5.x, npm, git. So what was left from the hubot installation instructions Read More ›

Chatops Integration with Automation Tools
Discovering ChatOps and a couple of hours of googling/reading lead me down the garden path of how to Integrate. I am using SaltStack for IT Automation ( sorry no Puppet/Chef here). SaltStack has served that role admirably. And now I want to move forward with ChatOps. Read More ›

ChatOps FOMO
A month ago I discovered the term **ChatOps** and was a little surprised it has been around for over 3 years. Guess I have missed out, but not any more. Time to get me some ChatOps Read More ›