Open Communities With Andrew Zigler

1 februari 2024 | 35 min

Openness plays a significant role in propelling DevOps and organizational processes forward. This is not to imply that everything must be open, but the default should be openness unless a valid reason indicates otherwise.

Andrew Zigler, developer advocate at Mattermost, and Matty from Arrested DevOps recently shared insights on this subject. They discussed creating impactful developer advocates, managing community writing programs, and dealing with the challenges of open source communities.

The Importance of Open Source in Communities

Andrew emphasizes that the loudest and most contributory voices in open source projects are usually the paid internal staff. However, he champions setting up pathways in the community to validate the experience of all contributors and reward them with anything from thought leadership, platforms, or even swag. The key is to influence individuals at all levels of engagement and ensure that they feel they own part of what they are contributing.

One of the challenges he identified is over-influencing which often stems from the fact that the paid staff are the ones driving the open source project vehicle. This imbalance usually drowns out the voices of other contributors, particularly those who may not have the luxury of dedicating as much time and energy to the project as the paid staff.

Andrew suggests a solution: the company creating more developer advocates through the multiplier effect. This means ensuring that everyone across the board understands the importance of the open-source community and empowers them to contribute. The more developers contribute, the larger and more diversified the community becomes, leading to better outcomes and solutions.

The Critical Role of Leadership in Open Source Communities

Matty highlights how vital leadership is in these initiatives. By allocating resources, prioritizing open source community engagement, and maintaining a strategic focus, leaders can do much to foster a healthy open-source community. Successful leaders understand that engagement levels differ, so they create opportunities for different levels of contributors to partake and contribute to the community.

To ensure the project remains harmonious and aligned with company goals, the leadership should give equal weight to both staff and contributors’ voices. In the end, everyone involved in the project is part of the community.

Engineering Blogs: The Balance of Output

The conversation took an interesting turn when they started discussing engineering blogs, a tricky subject for many organizations. Matty points out that these blogs have the tendency to publish sporadically, often dominated by lengthy droughts of content or a sudden overflow of posts.

Such inconsistency happens when the contributors, mostly engineers, write when they can spare the time. Balancing this dynamic is crucial, and one suggested solution is to involve people whose primary job is creating content. They can collaborate with subject matter experts to create consistent, relevant content.

Conclusion

Operating under a default open environment for your projects does not mean that everything has to be open. Nevertheless, transparency and openness should be the norm unless necessary otherwise. By dealing with the occasional echo chamber and understanding that contributions will always ebb and flow, the community will thrive and keep moving forward.

In line with the open source spirit, scaling advocacy is crucial in DevOps. It involves not only the individuals whose title is developer advocate but everyone within the company. By creating more advocates and amplifying community efforts, the DevOps movement continues seamlessly.

Links

Matty’s blog post about SharePoint (including broken images!)
Community Pulse podcast

Machine Learning Ops With Chelsea Troy

18 januari 2024 | 48 min

It's Been Ten Years of ADO, Charlie Brown

4 januari 2024 | 109 min

So You’re in Charge Now… With Ben Greenberg

22 december 2023 | 48 min

DevOps Isn’t a Department With Jeremy Duvall

7 december 2023 | 30 min

Runtime Analysis With Brian Kelly

23 november 2023 | 39 min

Complexity With Michael Stahnke

9 november 2023 | 47 min

The Database Calls Are Coming From Inside the House With Grant Fritchey

26 oktober 2023 | 42 min

Platform Engineering Goes to Flavortown With Matt Kurtiz

5 oktober 2023 | 47 min

What's Up With Open Terraform?

21 september 2023 | 39 min

The New DevOps With Adam Jacob

7 september 2023 | 58 min

Purposeful Personal Brand With Cassandra Faris

24 augusti 2023 | 52 min

The Importance of Your Personal Brand

Whether you’re working for a startup or a corporate giant like Visa, Target, or JP Morgan Chase, your personal brand helps to define your professional identity. This brand is not just about showcasing your GitHub contributions, but about telling your unique story. It’s about how you solved a particular problem or contributed to a project, not just about the technologies you used.

Building Your Brand Story

Building your personal brand starts with self-reflection. Here are three questions to ask yourself:

Who are you professionally? Identify your top three technical specialties, professional specialties, and team contributions.
Who are you personally? Identify three interests and hobbies, your most important beliefs and values, and aspects of your social, family, or community life that you want to connect with people over.
How do you connect with people? Identify shared interests, the advice or information you’re seeking, and what you want to know about other people.

Answering these questions gives you a wealth of material to draw from when telling your story and helps you identify your strengths and areas of expertise.

Show, Don’t Tell

When you’re creating your brand content, remember the old adage: show, don’t tell. Instead of proclaiming yourself a “thought leader,” demonstrate your expertise through your work. Share your accomplishments in a way that highlights how your work benefited others, not just yourself. This approach makes your story more engaging and relatable.

Embrace Vulnerability

Being open about what you don’t know can be a powerful part of your personal brand. It shows that you’re a lifelong learner, open to new ideas and willing to grow. Plus, asking for help or resources can lead to valuable connections and insights.

Pay It Forward

Sharing your knowledge and helping others is a powerful way to build your personal brand in tech. It demonstrates your expertise and your willingness to support your peers. This approach is not about trading favors but about creating a positive ripple effect in your community.

In conclusion, building a personal brand in tech is about more than just showcasing your skills. It’s about telling your story, connecting with others, and contributing to your community. By being authentic, open, and generous, you can create a personal brand that truly stands out.

Links to Resources Mentioned

Keep a Brag Book
KubeCampus: Free Kubernetes Training
Purposeful Personal Branding - Nov 2018 Slides 13-15 have the 3 questions

DevOps World is back for 2023, and you won’t want to miss out on this one-of-a-kind event! This year’s program is packed with exclusive insights, immersive workshops, and unparalleled networking opportunities taking place across multiple cities in the US, UK, and Asia. Elevate your DevOps game and register using the following links: NYC area, Chicago, Silicon Valley, Singapore, and London.

Everything's a Product With Sarah Morgan

10 augusti 2023 | 51 min

Cloud Native Security With Michael Isbitski

27 juli 2023 | 56 min

DevOps With Better Marketing With Pete Cheslock

29 juni 2023 | 57 min

It's Rough Out There With Sidney Miller

15 juni 2023 | 52 min

Data! Data! Data! With Francesco Tisiot

1 juni 2023 | 47 min

Platform Engineering With Daniel Bryant

18 maj 2023 | 49 min

Into the VOID Report With Casey Rosenthal and Courtney Nash

13 april 2023 | 60 min

Continuous Feedback With Roni Dover

28 juli 2022 | 51 min

Engineers Are People With Dagna Bieda

21 mars 2022 | 49 min

We Have More Work to Do With Tim Banks

2 mars 2022 | 54 min

Why Are We Still Talking About DevOps and Security

16 februari 2022 | 53 min

Deserted Island DevOps 2021

7 januari 2022 | 57 min

2021 Year-End Wrap-Up

31 december 2021 | 87 min

Technical Agile Coaching With Emily Bache

22 november 2021 | 35 min

The Reality of DevSecOps With Steve Giguere

22 oktober 2021 | 46 min

Brigade With Kent Rancourt

16 september 2021 | 35 min

Words Are Hard With Emily Freeman

8 september 2021 | 42 min

Multicluster Service Mesh With Phillip Gibson and Annie Wang

4 augusti 2021 | min

Foundational Practices With Johan Abildskov

15 juni 2021 | 61 min

Drawing DevOps With Ashton Rodenhiser

6 maj 2021 | 53 min

The Edge of Now With Cat Swetel

21 april 2021 | 38 min

Seasons of Community With Katy Farmer

31 mars 2021 | 44 min

All Things Docker

13 mars 2021 | 34 min

Learning to Learn and Learning to Teach With Shelby Spees

8 februari 2021 | 58 min

The Six Plots of Tech Twitter

25 januari 2021 | 61 min

Doing Releases Right With Scott Hain

5 januari 2021 | 49 min

2020 Year-End Wrap-Up

18 december 2020 | 85 min

Breaking Down Gates With Tim Banks

17 november 2020 | 51 min

Devopsdays Chicago 2020

9 november 2020 | 68 min

Tea and Anarchy With Alice Goldfuss and Ian Coldwater

22 oktober 2020 | 38 min

State of Open Source Security With Alyssa Miller

12 oktober 2020 | 56 min

Incident Retrospectives With Amy Tobey, Alex Hidalgo, and Rein Heinrichs

25 september 2020 | 59 min

Don't Worry, Do Care With Aaron Blohowiak

13 september 2020 | 54 min

Service Mesh With Michelle Noorali and Delyan Raychev

5 augusti 2020 | 36 min

Developer Experience With Stephanie Stimac

27 juli 2020 | 49 min

Deserted Island DevOps

26 maj 2020 | 70 min

Security Chaos Engineering With Aaron Rinehart

18 maj 2020 | 55 min

Helm Community With Matt Farina, Karen Chu, and Matt Butcher

30 april 2020 | 40 min

What's the Deal With AWS Billing...? With Corey Quinn and Pete Cheslock

23 april 2020 | 54 min

WebAssembly, Krustlet, and the Future

14 april 2020 | 43 min

Secure by Design

9 april 2020 | 56 min

Secure By Design

Guests Dan Bergh Johnsson, Daniel Deogun, and Daniel Sawano join host Jessica Kerr to discuss their book Secure by Design.

Daniel: “There’s a lot of good designs which come naturally to us as programmers but which has the interesting side effect that they also prevent security-related bugs.”

Domain Primitives

The panel discusses domain primitives as an example of coding practices that naturally provide security through good design.

Dan Bergh: “It’s a good starting point to understand that using domain-driven design not only makes your code more expressive, solves more domain problems. Even though these designs were not crafted to address security to start with, they’ve also had that as a side effect.”

Jessica: “I love that what you’re recommending in this part is to think harder about what you do want in the system, express that in the code, and suddenly a bunch of things that you don’t want in the system just aren’t.”

Testing

The panel talks about the ways in which testing contributes to secure design.

Daniel Sawano: “It tends to be so much easier and more robust if you start defining your own domain types.”

Immutability

The panel discusses the benefits of immutability.

Dan Berg: “It’s possible to…configure and mutate them until they are kind of safe-ish.” Jessica: “Kind of safe-ish?” Dan Berg: “Well, we are on a DevOps podcast.”

Logging

The panel talks about the security implications of logging practices.

Daniel Deogan: “One thing that’s very important is that if you log input directly into your logs, it becomes an attack surface for second-order injection attacks.”

Dan Bergh: “It’s a perfect launchpad for doing a really, really hard attack inside your system.”

Daniel Deogan: “The common mistake that many developers do is that they more or less dump inputs blindly.”

Jessica: “We have this illusion that logging is simple, but it isn’t.”

Cloud Thinking

The panel discusses the chapter on cloud thinking.

Dan Bergh: “In a way, we’re instructing the system to become more intelligent.”

Symmathesy!

The book is available online in its entirety.

Whose Transformation Is It Anyway? With Andrew Clay Shafer

25 mars 2020 | 38 min

Making DevOps Beginner-Friendly With Laura Santamaria

18 mars 2020 | 48 min

We Are Always Learning With Patrick Debois

17 februari 2020 | 54 min

Team Topologies

11 februari 2020 | 53 min

Historical Context

The panel discusses the origins of the book Team Topologies. The project started with a blog post.

Matthew: “Back in 2013, I actually wrote a blog post in my personal blog. I actually wrote it in a rage.”

In 2015, Manuel joined the team to help expand on the ideas from that blog post and create Devops Toplogies.

Manuel: “What the hell are you calling a DevOps team? DevOps is not about creating a new team called DevOps.”

DevOps Topologies

The panel discusses the impact of DevOps Topologies and some of the companies that have used it, including Netflix and Conde Nast.

Matthew and Manuel explain how the project has evolved over time as DevOps Topologies was being deployed in the real world.

Matthew: “It’s not just a set of patterns or templates. We wanted to provide an organizational capability for detecting when things have changed and have gone wrong.”

The panel discusses Conway’s Law and its implications for DevOps.

Manuel: “Teams are the means of delivering value.”

Flow!

The panel discusses the importance of flow in both living systems and organizations.

Manuel: “It’s a more experiment driven approach where we have this goal or this need we need to meet and then allowing the teams to find the right solution.”

Jessica: “In modern systems, the flow is of changes to the flow of the product. It’s a very different level of work.”

The panel discusses the need for different team configurations that are constantly evolving.

Matthew: “It seems quite important to understand different kinds of dynamics in the organization at different times.”

Three Team Interaction Modes

The panel discusses the three team interaction modes laid out in the book: collaboration, as-a-service, and facilitation.

Four Team Topologies

The panel discusses the four team topologies in the book: value stream aligned teams, enabling teams, platform teams, and complicated subsystem teams.

Jessica: “The limitation of a team is cognitive load. It’s not resources, it’s not pizza.”

Manuel: “Although pizza is very appealing.”

Manuel discusses Dunbar’s Number and how that concept can put useful constraints on teams.

Buy the book!

Episode images by Jessica Kerr. Show notes by Tyler Wilson.

Communities Are Made of People With Jono Bacon

17 januari 2020 | 53 min

2019 Year-End Wrap-Up

23 december 2019 | 52 min

Learning Stuff With Ali Spittel

10 december 2019 | 53 min

Kubernetes & the Future

4 december 2019 | 45 min

Kubernetes Security

27 november 2019 | 23 min

Kubernetes Best Practices

18 november 2019 | 40 min

Devopsdays Philadelphia 2019

3 november 2019 | 31 min

The Meltwater Transformation

25 oktober 2019 | 61 min

In this episode, host Jessica Kerr is joined by guests Gene Connolly and Joan Freed to discuss DevOpsiCon and their DevOps transformation at Meltwater.

DevOps at Meltwater

The panel discusses how DevOps has transformed the working environment at Meltwater. When Joan started at Meltwater, engineering and operations were kept entirely separate.

Joan: “There was this huge wall because engineering knew how things were built and operations knew the production systems…but there wasn’t any cross-pollination there.”

The lack of communication between the teams caused friction and slowed down product delivery. The process could take as long as six months.

Joan: “We’re a Software as a Service company so we always want to build products that are sticky and keep our customers around.”

Joan discusses how Meltwater began to move to a more agile business model. Getting engineers and operations people in the same physical location to work this out was important. Creating truly cross-functional teams couldn’t happen over Skype alone. DevOpsiCon was born to facilitate this transition.

DevOpsiCon!

Buy-in wasn’t immediate throughout the firm. Teams were updating legacy systems throughout the company and some were more focused on work than enablement.

Joan: “The first one was a little bit sneaky. The next one was in Manchester and that one was a little less sneaky.”

The panel discusses the importance of investment in enablement. This allows the feature teams to focus on the features they were actually building instead of all the underlying infrastructure.

Joan: “Having them build everything they need is not realistic.”

The fact that Meltwater has offices all over the globe makes any wholesale shift of company culture difficult. The panel discusses the need for face-to-face interaction to facilitate change.

Gene: “There are these events like DevOpsiCon where we solve the problem by bringing a broad set of people together.” DevOpsiCon has grown beyond just developers. Product, UX, support, and even sales have joined the event to help build relationships and give insight.

Jessica: “That is very DevOps in the sense that DevOps says that you need to take these concerns that cannot be separated…and you can’t put those responsibilities on different teams and make them fight.”

Unconference

The panel discusses the value of DevOpsiCon as an unconference. Attendees collectively set the agenda on each day of the event.

Gene: “That format has been so much fun and created so much value.”

Gene: “The event becomes the conference you didn’t know you needed at the start. It adapts collectively to what the organization needs at that moment.”

Ongoing Transformation

Gene discusses how allowing teams the freedom to experiment can yield results that a top-down structure could not provide.

The panel talks about the structure of mission, framework and support teams.

Joan: “By keeping things loosely coupled, it gives teams a lot more freedom to make the technology choices that are best for them.”

Ownership Challenges

The panel discusses the ownership challenges of moving from the datacenter to the cloud specifically and large enterprise changes more generally. The investment in enablement is critical.

Joan: “There’s very much a not built here mentality that can take place.”

Gene: “Software can benefit from being owned by many people.”

The panel talks about overhead costs to both software and human changes.

Gene talks about the challenges of moving to a full mission responsibility model.

Gene: “You need to find efficiencies more effectively than you’ve ever had to find efficiencies before.”

So! Has It Worked?

Gene: “Oh wait, you’re looking to me for an answer?”

The panel discusses the results of the ongoing DevOps transformation at Meltwater. The six month process for production changes has transitioned to a constant stream. Beyond the productivity, it has also been a quality of life improvement for the employees.

Gene: “We’ve got a Slack channel for all the production changes and it is constant activity in it all week long of changes going out.”

Joan: “We’re now getting hundreds of features out every six months to our customers.”

The panel discusses the ways in which transformation is, or should be, a constant process. The change goes on.

Find out more about Meltwater, their DevOps transformation, and the process of an unconference on their blog.

Episode art by Evelyn Kerr.

DeliveryConf

14 oktober 2019 | 35 min

Devopsdays Cape Town 2019

2 oktober 2019 | 37 min

State of Devops 2019

26 september 2019 | 53 min

In this episode, hosts Jessica Kerr and Matty Stratton are joined by guests Dr. Nicole Forsgren and Mr. Jez Humble, two of the authors of the 2019 Accelerate State of DevOps report.

Historical Context

The panel discusses the history and purpose of the report. This is the sixth year it has been produced. How did the report start and what questions is it seeking to answer?

Nicole: “Once upon a time, kids, making software was sad. Making software used to light people on fire!”

Nicole: “So why do we even do this DevOps thing? It’s because we want to make that software process easier and better.”

Jez: “These approaches have worked even in highly regulated environments. They’ve worked everywhere.”

Nicole explains that capabilities and practices are more important than tools.

Nicole: “There’s no such thing as DevOps in a box.”

The group discusses the limitations of systems data vs survey data, and the importance of collecting both. Survey data is low resolution but high signal and vice versa.

Jez: “Over-precision is something that’s a real problem in our industry.”

Nicole: [“How to Measure Anything” - Douglas W. Hubbard] ( https://www.amazon.com/How-Measure-Anything-Intangibles-Business-ebook/dp/B00INUYS2U )

The panel discusses the importance of starting from hypotheses instead of looking for any spurious correlations that exist in highly related systems.

What’s New This Year?

The panel discusses some of this year’s findings. The “retail apocalypse” over the last decade has had some surprising effects. Regulation has less of an impact than many industries assume.

Jez: “We do see high performers in large companies who are highly regulated.”

Matt: “People will sit there and say well, I have over 5000 employees and I can’t DevOps so now I have an excuse, and that’s not the case.”

Enterprise!

Change management processes are one of the key areas of difference between large and small enterprises. Jez discusses how to make that process more lightweight even in a large organization.

Jez: “Risk is also about upside risk. If you can’t move fast at delivering software, that’s a risk to your business.”

The panel discusses the importance of keeping a holistic view even in a large enterprise with specialized roles.

Nicole: “You go from change management theater to strategic change management for a massive organization. It’s scary but it is also dope!”

The report has found year after year that a slower change management process can paradoxically result in more instability, not less. The panel discusses the reasons this experiment was not successful and how enterprises can implement more nimble processes going forward.

Productivity

The panel talks about what productivity actually means. The report uses an interesting definition: “Productivity is the ability to get complex, time-consuming tasks completed with minimal distractions and interruptions.”

Nicole: “You may be just closing the tickets that are meaningless but are easy to close.”

Research shows that real productivity reduces burnout and improves work-life balance. The group discusses how to separate that real productivity from gamification and busywork. Jez talks about scaling up and how centers of excellence may not be as useful as previously thought. Nicole points out that the report has given a lot of starting points for people to begin making changes in their organization.

Remember to read the report!

Devopsdays Chicago 2019

17 september 2019 | 30 min

Devopsdays Minneapolis 2019

16 augusti 2019 | 39 min

Protocols and Sympathy With Martin Thompson

20 juli 2019 | 50 min

Pushing Left With Tanya Janca

13 juni 2019 | 49 min

Catching Up With Steven Murawski

23 maj 2019 | 36 min

Principal Engineering With Silvia Botros

21 april 2019 | 52 min

Making DevOps Magic With Arup Chakrabarti

5 april 2019 | 50 min

Unreliable Things Can Be the Most Valuable Things

30 mars 2019 | 54 min

Certification Fun With Jay Gordon

22 mars 2019 | 38 min

Shiny Objects With Jessie Frazelle and Andrew Clay Shafer

3 mars 2019 | 61 min

The Database Calls Are Coming From Inside the DevOps

8 januari 2019 | 52 min

2018 Year-End Wrap-Up

21 december 2018 | 72 min

Fireside Chat With Jessica Kerr

18 december 2018 | 41 min

Shouting at the DevOps With Corey Quinn

3 december 2018 | 44 min

Devopsdays Chicago 2018

14 november 2018 | min

DevOpsDays Kansas City 2018

14 november 2018 | min

Ignite 2018 Catch Up With Jessica Deen

16 oktober 2018 | min

Ignite 2018 Catch Up With Ed Thomson

16 oktober 2018 | min

Ignite 2018 Catch Up With Steven Murawski

15 oktober 2018 | min

Ignite 2018 Catch Up With Donovan Brown

15 oktober 2018 | min

Fireside Chat With VM Brasseur

23 augusti 2018 | min

Fireside Chat With Christine Spang

22 augusti 2018 | min

Devopsdays Salt Lake City 2018

20 augusti 2018 | min

Devopsdays Minneapolis 2018

8 augusti 2018 | min

Let’s Be Careful Out There

1 augusti 2018 | min

Devopsdays Amsterdam 2018

22 juli 2018 | min

Theatre Geeks Unite and Tech Over the World

9 maj 2018 | min

GOTO Chicago 2018

28 april 2018 | min

Punk Rock DevOps With Jay Gordon

12 mars 2018 | min

Devops Weekly With Gareth Rushgrove

1 mars 2018 | min

Hot Takes With Charity Majors, Jill Jubinski, and Eric Sigler

26 februari 2018 | min

Fireside Chat With Andrew Clay Shafer

10 februari 2018 | min

Career Change Into DevOps With Michael Hedgpeth, Annie Hedgpeth, and Megan Bohl

8 februari 2018 | min

Gophers: Brian Ketelsen & Erik St. Martin

23 januari 2018 | min

2017 in Review

8 januari 2018 | min

Matt, Trevor, Bridget, and Joe discuss 2017.

What were some of your favorite episodes?

Bridget: the GOTO Chicago eps were fun! And I love the 1-1 fireside chats.
Trevor: I enjoyed the Twitter banter around getting the live call show with Dr. Nicole Forsgren going, sad I had to miss it! There’s also a ChefConf episode sitting on my Surface that I just found the charger for!
Matt: The Live call in show with Dr. Nicole Forsgren. Also the recent fireside chat with J. Paul Reed. And the crazy-go-nuts overload show of Windows Stuff.
Joe: Live eps were easier to edit.

Let’s talk numbers

Website

22K visitors to the website (up from 16K visitors in 2016)
Little less than half of traffic comes from search
Twitter still accounts for about 6% of all traffic

Episodes

263,558 listens in 2017 (232,200 listens in 2016; 204,001 listens in 2015)
Most listened-to episode in 2017 (and also the most-watched YouTube Video!) was Old Geeks Yell At Cloud With Andrew Clay Shafer & Bryan Cantrill (https://www.arresteddevops.com/yelling-at-cloud/)

What happened with you in 2017?

Matt

Got a new job (recently). Now DevOps Evangelist with PagerDuty
Spoke at DevOpsDays Denver. Went to DevOpsDays Hartford and MSP and Madison.
Pulled off an acceptable DevOpsDays Chicago

Bridget

New job in September! Microsoft dev advocate
Kept scaling devopsdays up (51 events on 6 continents)
Way less conference speaking; was on more conference program committees

Trevor

Wrapped up first year at Chef!
Spoke at DevOpsDays Dallas
Moved back to Chicago!

Joe

Iceland!
Tesla roadtrip!

Referenced in this ep:

Past year-in-review eps:

Community & Event Stuff

If you have an upcoming conference you would like to see promoted on ADO, you can fill out the handy form at arresteddevops.com/conf

Upcoming conferences

Open CFPs

lots of DevOpsDays
SREcon - March 27-29 2018 - CFP closes Wednesday, November 29, 2017, 11:59 pm PST
ScaleConf Colombia - April 27 - 28th 2018 - CFP closes Jan 15
Velocity San Jose - June 12-14 2018 - CFP closes Jan 17
ChefConf 2018 - May 23-26, CFP open now, closes Jan 10th

Discount codes: ADO2018 for 20% off lots of devopsdays, 10% off ChefConf

Work/Life Balance With Sasha Rosenbaum, Bill Weiss, and Katie Prizy

7 januari 2018 | min

Inner Source to Open Source With Aaron Rinehart

5 december 2017 | min

Fireside Chat With Grandpa Paul

4 december 2017 | min

Fireside Chat With Alice Goldfuss

28 november 2017 | min

Devopsdays Madison 2017 With Emily Freeman, Joshua Zimmerman, & Christian Herro

31 oktober 2017 | min

Velocity With Inés Sombra & James Turnbull

25 september 2017 | min

Live DevOps Call in Show With Nicole Forsgren

30 augusti 2017 | min

CI, CD, Oh My! With Jez Humble

23 augusti 2017 | min

Devopsdays Minneapolis 2017 With Bryan Liles, Jessie Frazelle, and Andrew Clay Shafer

29 juli 2017 | min

Go Small or Go Home With Daphne Chong and Kenny Bastani

2 juli 2017 | min

Tupperware Party With Jérôme Petazzoni, Mark Heckler, and Jennifer Heckler

28 juni 2017 | min

Risky Business With Nicole Johnson, Matt Curry, and Anthony Lee

19 juni 2017 | min

Devopsdays Toronto 2017 With Roderick Randolph, Arthur Maltson, Aaron Aldrich, and Amy Mansell

13 juni 2017 | min

When the Levee Breaks With Jeff Smith and Mark Imbriaco

4 juni 2017 | min

Let’s Do the Devops Again With Nicole Forsgren & Tim Gross

19 maj 2017 | min

Old Geeks Yell at Cloud With Andrew Clay Shafer & Bryan Cantrill

8 maj 2017 | min

Enterprises With Bryan Liles

17 april 2017 | min

Startups With Charity Majors & Nicole Forsgren

4 mars 2017 | min

Microsoft Redux With Liam Bennett, Brandon Olin, Reuben Dunn, Glenn Sarti, and Chris Hunt

26 januari 2017 | min

Devopsdays Cuba 2016

17 januari 2017 | min

2016 Year-End Extravaganza

31 december 2016 | min

Matt, Trevor, and Bridget chat (at length) about podcasts, podcast recording, and podcast recording software. Oh, and the highlights of 2016 if they get around to it. (Don’t miss the supercut of all 2016’s cold opens, which was edited by Joe, even though Matt takes credit for it!)

What were some of your favorite episodes?

Bridget: So many great ones! Kicked off the year with Andrew Clay Shafer and Kelsey Hightower, and we kept going at that pace!
Trevor: Soon-to-be-finished Singapore PowerShell Summit Episode
Matt: Personal Brand episode, Who Owns Your Availability (Bridget says “omg, left pad, we need to have this episode NOW”), CareerOps

Let’s talk numbers

Website

16K visitors to the website
46% of traffic comes from search
16% comes from referrals, mostly Twitter (Twitter is 6% of all traffic)
1% of our traffic comes from the hugo site itself (our site generator)

Episodes

232,200 listens in 2016 (204,001 listens in 2015)
Most listened-to episode in 2016 was Application Configuration with Adam Jacob and Tim Gross
Most watched YouTube video in 2016 was Bridget’s Fireside Chat with Bryan Cantrill

Website updates

Didn’t change much, but we list episode numbers now (maybe we’ll add dates sometime)

Other improvements

Matt did a session with Daniel J. Lewis of the Audacity to Podcast (we were a featured podcast eval in Podcasters Society) and we got a whole bunch of ideas on how to improve the show - some of which we have already started to implement). You can check out our outstanding issues on GitHub
Welcome to Joe as our main audio editor!

What happened with you in 2016?

Matt

Didn’t travel much - only spoke once twice - once at Pink16, and once at a Cloudbees conference in Chicago
Only went to one devopsdays - Chicago. I’m failing!
Hey, I got married

Bridget

28 talks. Give or take. A few of those were at devopsdays - I made it to London, Toronto, New York, Detroit, Havana, Philadelphia, Madison, Sydney - and of course ran Minneapolis.
Counting North America Joe and I hit 5 continents this year. And I was going to travel less. Instead, all the airline status. Rethinking next year. Maybe get into webinars?
Devopsdays: growing! New core team! New cities!

Trevor

I spoke 3 times this year, but got to attend so many more conferences than before
I was literally on the other side of the planet to my usual place of existence for the first time.
I moved to LA
I work for Chef now!

Check Outs

Bridget

Systems We Love - videos at https://twitter.com/SystemsWeLove/status/809117528374972416

Trevor:

[Westworld](http://www.startrek.com/article/klingon-mug-and-collector-lapel-pins-ready-to-beam-up Klingon Blood Wine Mug)

Matt:

There’s this thing called Minecraft I “discovered” via my kids.
Also reminder about code.org and everyone is talking about Hour of Code
It’s been super retro time for Matt as he re-read a bunch of the Dragonlance books, and discovered a bunch he hadn’t read, including The Soulforge.

Community & Event Stuff

If you have an upcoming conference you would like to see promoted on ADO, you can fill out the handy form at arresteddevops.com/conf

Open CFPs

lots of DevOpsDays
Velocity San Jose until Jan 10th
ChefConf 2017 - closing on January 18, 2017
Monitorama - May 22-24 - until Feb 1

photo credit 1, photo credit 2

Devopsdays Sydney 2016

6 december 2016 | min

Discovery With Julia Evans

16 november 2016 | min

Chatting With Pauly Comtois

5 november 2016 | min

DevOps in the Windy City With Jeff Smith, Jerry Cattell, and Sameer Doshi

24 oktober 2016 | min

The Art of Monitoring With James Turnbull

2 oktober 2016 | min

Don’t forget to check out the book itself! The Art of Monitoring.

Back in the day, James also wrote a book called Pro Nagios 2.0.

Three stages of monitoring maturity:

Manual, user-initiated, or no monitoring (aka Bridget’s example of “we know things are broken because the customer calls us to complain”)
Reactive
Proactive

“You will eventually get to CPU, memory, and disk, but a lot later after you start with the things you should really care about” - James

“Too much monitoring is binary - this thing either works or it doesn’t” - James

Also mentioned:

Community & Event Stuff

Where we’ll be for the upcoming fortnight

Matt is getting married at the Jim Beam distillery on Saturday
Bridget will miss the bourbon wedding as she’s heading to Joe’s family reunion followed by GOTO Copenhagen.

If you have an upcoming conference you would like to see promoted on ADO, you can fill out the handy form at arresteddevops.com/conf

Upcoming conferences

For any devopsdays, try the code ADO2016! It should get you 20% off. Also now works on O’Reilly Security conference.

For more DevOps awesomeness, check out the Chef Community Summit, October 26th and 27th in Seattle, WA. This Open Space event provides a great opportunity to connect with the DevOps Community and Chef Engineers over two days of engaging sessions and hallway discussions. Bring your ideas, passion and excitement for Chef and DevOps to this highly interactive event. Go to summit.chef.io to register for this awesome event and use the code ARRESTEDDEVOPS to get 10% off your ticket!

Open CFPs

A lot of devopsdays CFPs closing soon - see devopsdays.org/speaking
OSCON’s CFP closes Oct 25

Check Outs

James

Skyliner.io
Jason Dixon
Vote, but not for Trump

Bridget

Cloud Foundry Summit is going on in Frankfurt right now
honeycomb - explorable operations metrics from Charity Majors
MicroBadger - for docker image inspection Liz Rice & Anne Curry at Microscaling Systems

Trevor

Windows Server 2016 is GA!

Matt

InSpec has shipped 1.0! You can check it out at http://inspec.io/ InSpec is compliance as code – a human-readable language for automating the continuous testing and compliance auditing of your entire infrastructure. You can also use it to verify if your servers and applications are configured correctly.
Self-promotion: working on a shareable theme using hugo for podcasts. Check it out at github.com/mattstratton/castanet

DevOpsDays DFW 2016

14 september 2016 | min

Fireside Chat With Bryan Cantrill

14 september 2016 | min

Devopsdays Chicago 2016 With Nell Shamrell-Harrington, Jill Jubinski, and Michael Stahnke

12 september 2016 | min

Operationalizing Open Source With Michael Hedgpeth and Doug Ireton

30 augusti 2016 | min

Application Configuration With Tim Gross and Adam Jacob

1 augusti 2016 | min

Devopsdays Minneapolis 2016

30 juli 2016 | min

ChefConf 2016 With Jon Cowie, Fletcher Nichol, and Annie Hedgpeth

27 juli 2016 | min

CareerOps With Jill Jubinski and Peter Burkholder

14 juli 2016 | min

Finding Signal in the Noise, With Aneel Lakhani and Jason Dixon

5 juni 2016 | min

Devopsdays Toronto 2016

28 maj 2016 | min

Containers and Security With Ben Hughes and Jessie Frazelle

12 maj 2016 | min

Speaking at Conferences With Ryn Daniels

16 april 2016 | min

Who Owns Your Availability? With Charity Majors and Pete Cheslock

25 mars 2016 | min

Building Your Personal Brand With Andrea Javor and Michael Hedgpeth

23 mars 2016 | min

Getting Down With GitLab With Job Van Der Voort

10 mars 2016 | min

Open Your Stack With JJ Asghar

26 februari 2016 | min

Vendors: Frenemies or Friends? With Michael Ducy of the Goat Farm

12 februari 2016 | min

Chocolatey Goodness With Rob Reynolds

30 januari 2016 | min

Open Source With Phil Dibowitz

20 januari 2016 | min

Platforms With Kelsey Hightower and Andrew Clay Shafer

13 januari 2016 | min

2015 in Review

31 december 2015 | min

Events

ChefConf (all three hosts were speakers, and we actually were in the same place at the same time!)
DevOpsDays Rockies (Matt was a speaker)
ALM Forum (Matt gave his 5 love languages talk)
DevOpsDays Minneapolis (Bridget was an organizer, Matt did an ignite about @petechesbot)
ThatConference (Matt gave a talk)
DevOpsDays Detroit (Matt gave a couple talks)
Bridget spoke at about 23 events, including oscon, velocity NY and Amsterdam, Chef Conf, QCon, and more. And she’ll be giving a tutorial at oscon in May.
Bridget also went on tour with her team, which was pretty awesome.

Memorable Episodes

Dave Zwiebeck and Mike Rembetsy joined us to talk about Blamelessness (Matt points customers to this episode all the time)
Jefferey Snover and Jessica Devita came on the show to chat about Microsoft stuff
We talked Docker with James Turnbull
We finally got around to having Andrew Clay Shafer on the show
Talked about culture change with Bill Joy (Matt also sends customers to this episode all the time)
Discussed the basics of ITIL and how it can work (or not) with DevOps
Bridget enjoyed such moments as talking cognitive neuroscience with Courtney Nash & Lindsay Holmwood and talking about failure in distributed systems with Kyle Kingsbury.

ADO Updates

No new hosts added this year, but it’s the first full year being “staffed up”, whatever that means
We sometimes do eps without all the hosts! Which allows for a bit more scheduling flexibility
Redesigned the website! This means now if you find errata in the show notes, etc, you can go right ahead and send the fixes to us via pull requests at github.com/arresteddevops/ado-hugo
Stats!
- 14,361 visits in 2014
- 24,293 visits in 2015
- 76,655 listens in 2014
- 204,001 listens in 2015
- Most listened to episode - the Docker episode (devops means docker, right?)
- In 2014, averaged between 2,500 - 3,000 audio listens/downloads
- In 2015, we now average between 5,000 and 7,000 downloads (double!)
- Most watched YouTube video was the Microsoft episode
We’ve had a lot of awesome linkage this year, and we appreciate everyone who links to our epsiodes!

Links

The Chef Prince of Azure - Trevor’s talk at ChefConf 2015
Cooking Up Drama - Bridget’s talk at ChefConf 2015
DevLOLOps: How To Automate Your DevOps Hilarity - Matt’s talk at ChefConf 2015
Velocity 2016 Call for speakers
DevOps In The Machine - Matt’s DevOpsDays MSP talk about @petechesbot
That Conference (Summer Camp For Geeks)
Matt on Channel9 at That Conference
The Audacity to Podcast

Check Outs

Bridget

https://cloud.gov/ - if the us government can innovate, you can too.

Trevor

Fallout 4 was pretty cool

Matt

The Force Awakens. Period.
devopsconferences.org You can follow the list at @devopsconfs. This account tweets alerts about CFP dates.
Speaking of people who got promoted after being on the show, recommend Sasha Rosenbaum’s talk “Single Person of Failure” at Devopsdays SV

Podcast Recommendation

Software Defined Talk (they need a more memorable URL)

Measurement and Sharing With Nicole Forsgren

15 december 2015 | min

DevOpsDays 2015 Year in Review With John Willis

1 december 2015 | min

Switching Teams: From Linux to Windows and Windows to Linux

30 november 2015 | min

What Is New at Puppet? With Eric Sorenson

25 november 2015 | min

Test Driven Infrastructure With Arthur Maltson and Michael Goetz

19 november 2015 | min

Tis the Season...For Scaling! With Rob Cummings and Matt Curry

13 november 2015 | min

ITIL Eye for the DevOps Folks With Steven Boyd

31 oktober 2015 | min

Creating DevOps Communities and Events With Andy Burgin, Dustin Collins, and Nathen Harvey

14 oktober 2015 | min

Infrastructure as Code With Joshua Timberman, Eric Sorenson, and Robyn Bergeron

1 oktober 2015 | min

Cognitive Neuroscience With Courtney Nash & Lindsay Holmwood

23 september 2015 | min

ChatOps Extravaganza With Jason Hand, Sasha Rosenbaum, and Peter Burkholder

10 september 2015 | min

Recording Live from DevOpsDays Chicago!

ChatOps is used by many teams and companies as the main communication tool for day to day chat, and their most important activities. In fact, ChatOps may be taking the place of email in the workplace for internal communication for tech teams as it helps communication during DevOps activities like deploys, code pushes, etc. This episode discusses best practices (if there are any) of ChatOps and how to make sure you are getting the most from your team communication tools.

Asynchronous vs. Synchronous Communication

25% of the work week is spent managing your inbox. You can actually increase productivity by moving to Sync communication…we think.
Sasha: ChatOps creates an enormous amount of noise while at the same time makes communication grouped and searchable.
Discussion suggests it is up to the user to mediate that noise, but is it the user, the culture, or the conversation itself that dictates the role of chat? Tivo is given as an example of user mediation: you recorded a shit ton of stuff, and watched only what you wanted.

The panel comes to the conclusion that important decisions should lean away from ChatOps, and into a more formal, permanent form of communication. “Important things will ‘re-bubble’ again,” but the chatroom is not the place if a team consensus is needed, especially if the team is remote.

Create a culture where ChatOps is used in the way you need.
Risky to go “Super Pendulum Swing” in one direction or the other.

What is ChatOps good at?

Solving the communication problem.
- Brings everybody into the same experience. Even if you are across Europe, or accross the room, you are having the same experience.
Great for in the moment Q & A.
- Even with one on one questions, if the answer is shared in a public channel, the information is given to all on the team which moderates the need for repeated questions, and increases team efficiency. You need to be constantly pairing. If you direct message someone, you are keeping that information from the team. “If you are not working in your chat tool, you are not collaborating.”
Shared History
- Makes communication searchable, and organized by topic, or at least team.
Rooms should be broken down to their smallest parts. Topics, Meetings, Projects, they should all be open spaces for all departments.
Getting messages/alerts from integrated tools is perhaps one of the most important features of ChatOps in DevOps: Jenkins, Github, Travis, etc.

What’s the Problem?

There are just too many messages. But they are necessary messages.
Internal ChatOps tool is almost useless when you are a consultant and you are all working on different clients.

Problems with Adoption?

In your organization, if you are considering chat tools for different purposes, use benchmarking and measurements to monitor your usage and data in each tool (in this case, chat vs. email)
Matt uses RescueTime (https://www.rescuetime.com/) religiously. His current rate of email vs Slack: 4 Hours in Slack, 48 minutes in email.
Sales, Support, etc. prefer email, but that will not change until their tools are integrated with Slack as well.

How to make use for permanent communication?

Have you adopted ChatOps for sustained messages and conversations that need to be kept?
Pinned Items are like the refrigerator…it’s emptied at the end of the week.
If it should live longer than a week, then it gets moved to the wiki, google doc, or the most appropriate space for the info.

How do you practice Chat-Zero? (Comment with your answer)

How do you go through every message in your chat?
How do you know what is important?

Last Thoughts:

Peter can’t wait for computers to be smart enough to interrupt us only when appropriate. “Who is the least invested in their work right now? Let’s notify that person.”
If you are considering it, do it. But be careful what you use, and how you use it.
Jason: It’s new tech, but its the old problem. ChatOps is just the newest efficiency on the line.

Sign up for the Banana Stand for the latest ADO news.

Podcast Me Maybe With Kyle Kingsbury

15 augusti 2015 | min

Building an Ops Team With Charity Majors, Patrick McDonnell, and MCR

29 juli 2015 | min

Charity Majors (Parse/Facebook) is an “Accidental Computerer.” She was the first infrastructure hire at Parse, which was acquired by Facebook in April 2013. Charity handles all of the backend operations and DBA work, and manages a team of 7 engineers.

Patrick McDonnell is a web operations manager at Etsy. He made the transition from individual contributor to management a few years ago.

Mike Rembetsy (MCR) is the VP of Technical Operations at Etsy. He was one of the first ops people at Etsy when he joined in 2008, and has helped grow the team to over 50 engineers since then.

Charity points out that your first question should be whether or not you actually need an ops team. She says, “There are a lot of places out there that think they need traditional operations engineers, when all they really need is someone to really care about their infrastructure… You should have genuinely hard operations problems before you even start looking to hire engineers.”

Once you do start down the path of building an ops team, MCR notes that you have to maintain it. You’ll need to grow the team, grow the individuals, grow the culture, which, if your company is in a startup phase, can be distracting to the overall goals.

“The glue that holds a team together is how well people interact with one another, how well they respect each other,” says MCR. “That’s what you build good ops teams on, and frankly, that’s what you build any good team on.”

As a growing company, we suggest you follow this process:

understand/establish your mission
communicate that mission clearly to the interviewing team
find people who can fulfill that mission
understand the fact that technical skills are great, but at the end of the day, it’s about the chemistry and culture of your team

Trevor agrees that cultural concerns are absolutely an issue, and asks for suggestions on how to interview for that.

MCR explains that at Etsy, they split their interview questions into technical questions and cultural questions about how the interviewees handled particular situations, as well as other skills.

Charity notes that good ops engineers are good at learning things, but some people freeze up when they’re put on the spot. She suggests providing interviewees with at least 50% of the questions beforehand, so they have a chance to prepare the preliminary information before interacting with her in a formal interview. “You want people to bring you the self that you’re going to be working with on a day-to-day basis,” she says, “not the self that is freaking out and wondering how they’re being perceived.”

Transitioning into the topic of management, Charity brings up the point that if, as a manager, you’re still responsible for key pieces of the infrastructure, you’re holding your team back in their technical development.

One of the jobs of a manager, MCR says, is to help motivate your team, and then step aside and let people get things done. In doing this, you let them succeed, as well as fail, and grow as a result of those experiences. He references Daniel Pink’s book, Drive, and the three pieces of science that motivate human beings: autonomy, mastery, and purpose.

“A manager’s role is a facilitator,” says Patrick. “Everyone should be doing what they think they need to do. It’s my job to remove obstacles that come in their way and make sure I can smooth things out if need be, but really, it’s to encourage and allow people to reach their full potential.”

Etsy has two separate career paths: one for individual contributors and another for management, which allows for the honing of particular skills specific to the end goal.

Charity agrees wholeheartedly with this approach, and says emphatically, “Management isn’t a promotion. It’s a career change.”

In addition, manager and leader is not synonymous. Some people are much better leaders than they are managers, and vice versa. It is also possible to be a leader while continuing as an individual contributor. In fact, Patrick points out that in some circumstances, you might actually lose influence when moving into a management role if you’re already a leader amongst your peers.

Bridget asks the panel, “What’s your best advice for someone in the position of building an ops team?” Patrick: “Focus on hiring good people. Hire people that you like. Hire people that you trust. Hire people that, maybe they need to do a little more research, maybe spend a little more time on StackExchange than the next person, but you know that they’re going to get the job done in the way that you need to get the job done.”

Charity: “Build your networks. Go to meetups, talk to people. Don’t just talk to the popular kids. Reach out to diverse communities and diverse crowds, and go meet people who are doing cool and exciting things, who are slightly off the beaten path. The more people you know, the better you’re going to be at hiring.”

MCR: “Make sure that you address conflict. Make sure you create a safe place for the people you do hire, to have open, honest conversations with one another. There’s nothing more toxic to a team than people chatting behind other people’s backs.

Check Outs

MCR: – Open Source Utility for ELK – 2012 Velocity Talk from Dr. Richard Cook: How Complex Systems Fail Charity: – Guided Meditation for Adults: “Breathe in strength, breathe out bullshit.”

Patrick: – Dr. Christina Maslov’s talk at Velocity: Burnout in Tech – 5-minute burnout self-assessment

Bridget: – DevOpsDays Minneapolis talks

Trevor: – Batman Arkham Knight – GitKraken

Upcoming Events:

DevOpsDays at #alltheplaces

Eating Sushi With Andrew Clay Shafer

18 juli 2015 | min

Transcript

Andrew Clay Shafer (@littleidea).

Coming to you live from DevOpsDays Minneapolis, Matt and Bridget sit down with Andrew Clay Shafer in front of a live audience to talk about the growth of DevOps, explain some commonly heard but not always understood terms, and more (after a brief detour on why episode numbers on podcasts are obnoxious, and why this episode is titled “Eating Sushi with Andrew Clay Shafer”).

Don’t know who Andrew is? He suggests you Google him, but then goes on to give a little bit of his background: he’s been involved in software development and technology for almost 20 years. After rooming with Luke Kanies, founder of Puppet Labs in college, Andrew got interested in operations and system administration.

O’Reilly’s Velocity Conference was also influential in Andrew’s growth, and Andrew reiminisces about his presentation on Agile Infrastructure in 2009. John Willis speaks up from the audience, and strongly suggests going to look at the slides.

Matt takes a moment to explain how Arrested DevOps started: “I started listening to John and Damon on DevOps Cafe and understood about 5% of what they were talking about… There are these things that we, as part of this community, tend to know, and what we try to do with this show is break it down for the people who don’t have the tenacity or stubbornness that I did.”

Along that vein, Matt asks Andrew to expand upon the “wall of confusion” idea that was referenced in his 2009 talk, and has become a commonly-used (but not always understood) term in DevOps lingo.

“It’s a jargony way to talk about the different incentives that exist between developers and operations,” says Andrew. “There’s a transition that happened as software became service-oriented, versus shipped on CDs, where the servers now become this critical part of the value chain, and if you deemphasize the system administration and operation of those servers, then you don’t actually have software. In the middle of these two worlds, where in one, systems administrators were for keeping the printers and the mail server up, to where they’re a critical part of the value chain in the new world, there are broken IT practices that don’t make sense when you’re trying to manage a service. It means recognizing that the best way to optimize a system isn’t to just throw random stuff onto production servers, and then make it ops problem, but to recognize that the infrastructure itself has become an application, and that you can manage these things as an application.”

Andrew points out that as much as he enjoys the attention (and who wouldn’t?), he was simply in the right place at the right time, and the right people listened to him. He connected dots to take advantage of tools and practices that were already used to manage software process, and bring that into the infrastructure and operations works.

Bridget asks, “You mentioned Agile, and you mentioned Scrum. I’ve heard you say that Scrum is a disease. Can you give us your thoughts on where that sort of stuff is going?”

“My personal opinion is that Scrum’s impact on software development is net negative,” Andrew says. “I think it’s particularly bad when people try to adopt it in operations. It’s really susceptible to problems when you have any interrupt-driven work whatsoever.” He suggests pursuing kanban and chatops, making work explicit and visible – tools that allow both you and your management to understand the full context and value of the situation.

The conversation transitions into talking about how to make actual changes to your operations and infrastructure teams, rather than always jumping through hoops to make the necessary changes to keep the pages up or the apps running smoothly. The answer isn’t to simply communicate how difficult it is to manage a system – upper management won’t understand the pain, and therefore won’t listen to the complaints. You have to involve the rest of the team so that they understand what you’re going through first-hand. You get empathy from suffering. This all plays back to Conway’s Law, as Andrew points out: “If you believe Conway’s Law is true (as I do), then you understand that your org structure (who communicaties with whom, who reports to whom, etc.) determines the outcome of any decision.”

Bridget brings up the point that this is the essence of dogfooding: requiring not only your engineers to be in the code, but your employees to be using the products that you’re creating, so that there’s a general understanding of why things work the way that they do, and a buyin for the necessary changes.

Bridget asks Andrew to expand more on what he thinks we are (and should be) optimizing for, which he touched on briefly during his talk at DevOpsDays.

Andrew counters that in order to do that properly, we need to first frame the context, which is a problem that plauges DevOps, Agile, and many other systems with which people are trying to transform their companies – you can’t do something prescriptive until you have enough context to understand where you’re starting from. For example, the diet and exercise program you’d give to someone who’s relatively healthy and active is very different than someone with a different set of circumstances. By the same token, you can’t prescribe a solution to an infrastructure problem without first investigating the roots causes and understanding what the foundation is.

However, if you model the world as everything is an agent trying to maximize some function, then the basic premise of your decisions is cause and effect. “Looking at the way people behave, and how this plays out within organizations, you might have very different patterns of interactions and patterns of health.” Andrew continues, “Therefore, what you’ll tell people do is very different from context to context.”

Despite all of the different scenarios, Andrew argues that there are three things you can always do:

Understand the incentives that people are motivated by
Align the incentives with behaviors
Radiate information to help people make different choices

Wondering what the Nash equilibrium and Pareto efficiency game theories are? Here are a few links:

Nash equilibrium: Wikipedia
Pareto efficiency: Wikipedia
Andrew’s slides on Leading a Learning Organization

Career Devops With Jeff Hackert

25 juni 2015 | min

Jeff’s background includes 30 years of experience working in people management. Specifically in developing the management skills, and careers of Software Engineering teams. He now works at Chef as the Director of Learning Experiences.

What is Career Development

Just thinking about career development in the workplace is not very common. Jeff discusses statistics around the specifics of Career Development and implementation strategies. For example, in one study, more than 60% of respondents said that Career Development was of “little to no” importance to their current employer. Less than 5% of employees at some organizations surveyed receive any career feedback from their current bosses.

Jeff: There are different approaches people usually take as they grow into their careers. The first is a “go with the flow” approach which can cause issues when leadership skills are not developed in response. A second approach is more proactive in which you plan out and describe your career and where you would like to go in the future.

What do you do?

The panel discusses their own careers and their attempts to summarize “what you do,” their weaknesses, and their strengths. Especially if the description is to be seen by the entire organization. Can you brag? Should you be vulnerable? For some, listing strengths as an engineer can actually be more difficult than listing weaknesses.

Jeff: “It’s a huge act of vulnerability to say who you want to be in an organization”

Jeff: “Every Engineer is responsible for their own career development […] and you are 100% responsible for the career development for every engineer on your team”

Can you be the Director of Flowers?

The group discusses the merit of titles and how relevant and useful they are within an organization. Jeff: “Titles don’t matter, and they absolutely matter.”

Ultimately, titles should be descriptive of what you want to do in that position and are indicative of positional authority more than anything else.

Management is not for everybody.

“It’s Not a Promotion - It’s a Career Change” — Lindsay Holmwood (http://fractio.nl/2014/09/19/not-a-promotion-a-career-change/)

Matt: “I don’t like managing people that’s not my thing.”

Jeff: Performance management is not the same as Career Development. Often, when Software Engineers get promoted to managers, coding becomes a secondary responsibility and People Management becomes a priority. For some, this is not the trajectory they want for their careers, and that’s ok. Providing context for feedback and guidance is a great way to identify these mismatches between current position and where someone wants to be.

Jeff: Using analytics “Project Oxygen” from Google describes 8 qualities of a good manager. ( http://www.nytimes.com/2011/03/13/business/13hire.html )

Jeff: “The Beatle Book” - Ken Schwaber ( http://www.amazon.com/Ken-Schwaber/e/B001H6ODMC )

Trevor: Doing an emotional check-in to describe your current emotional state is a powerful management and communication tool (from the Core Protocols by Jim McCarthy - http://www.mccarthyshow.com/online/)

Jeff discusses the usefulness of check-ins on every level within an organization.

You know you have a good manager when: 1. They expresses real concern with your career development, separate from quality conversations. 2. They create opportunities for you to realize your goals. (or at least get closer) 3. They have your best interest at heart.

…and then there’s the checkouts:

Disasters!

29 maj 2015 | min

Stephanie Van Dyk @sevandyk is an SRE at Google, and has also worked on healthcare.gov.

Mark Imbriaco @markimbriaco is co-founder and CEO of OperableInc. He’s worked previously at DigitalOcean, GitHub, LivingSocial, Heroku, and 37signals.

Bridget starts by asking Mark what Operable is all about. Mark explains that Operable is trying to help people who are on the “pointy end” of incidents. They’re trying to build tools that help people collaboratively fix problems. “There’s a lot of tools these days that do things like wake you up and alert when you when there is a problem,” says Mark, “but we think there’s a lot of room to help people actually solve problems.”

Stephanie briefly goes through some of the history of healthcare.gov, and how she first learned about it. Her position was unique, she points out. “We worked very hard, and very long hours… We were also in the fortunate position of having a lot of authority, which is important if you’re trying to fix a disaster. There’s a lot of problems to solve, and you don’t want any of your additional problems to be ‘Well, who gave you permission to do that?’”

“That’s a really good point – not all disasters are created equal,” Bridget notes, “and maybe we should take a step back and think, what are the ingredients that make something a disaster?”

Mark: I’m used to catestrophic problems that last for a few hours at most, or in the really bad case, mabe it lasts for two or three or four days, not something that goes on for weeks or months, so that’s a different perspective from what I have, so I’m super interested to hear about [healthcare.gov].

Matt: I think there’s the disasters where there’s a thing that happens, that’s maybe localized to one type of scenario; then there is what happens in an episode of This American Life, where it’s just one thing after another and everything unravels. There’s a quote from the episode that I like to think about when we think about these bigger disasters that are more than just an outage that may be far reaching:

“One ingredient of many fiascos is that great, massive, heart-wrenching chaos and failure, are more likely to fail, when great ambition has come into play, when plans are big, expectations are great, and hopes are at their highest.”

“I think you’re certainly right,” agrees Stephanie. “In order for something to be a disaster, the stakes have to be quite high… An outage that you find, and fix, and write a postmortem, and everyone learns something, and the users all get over their hurt, that’s not a disaster. That’s just life.

At times, there are incidents that leave scar tissue in their wake, making people wonder for years to come if they truly want to use certain products, or trust their data to a certain company. Mark reminisces: “The gutwrenching terror is, are we going to get our customers’ data back, or is it just gone? As an ops person, there’s almost nothing more terrifying than losing data.”

This provides a perfect segue into some of the non-obvious issues that arise with disasters. Stephanie brings up the point that you have to be prepared to regain the trust of your users. “How people think about your service is going to determine the fallout of it, and the impact. It’s interesting – it’s not something engineers like to think about very much, because they simply fix the problem. But someone has to be the one to reassure people that it’ll be ok.”

Mark agrees: It’s really, really hard to be in the middle of responding to a serious problem, and also have to be the person who needs to communicate about that externally. There’s so much good will that can be gained from being as transparent and public as you can about what’s going on, without pulling punches or hiding, even if things are really bad.

This is all well and good, but Bridget brings up a good point: “How do you know exactly how and what to communicate to people?”

Stephanie: There are definitely rings of communication. You have to be able to talk to the other engineers who are working on the problem, and those conversations are going to be very different than how you talk to your customers, even if you’re trying to be super open and honest. Your customers don’t care about where in the logs you found that tiny error. They care about when it’s going to be fixed, and whether you’re actually working on it… Also, the person who’s in charge of solving the outage should not be the same person who’s in charge of communicating about the outage. You should have different roles for that.

Mark agrees emphatically, and also noted that wording is incredibly important – not only what you say, but how you say it, and the words that you use. “There are three things I want to get across. The first thing is, I want to apologize to people. It has to be a sincere apology. The other thing I need to do is make sure people feel confident that I understand what happened. I need to display confidence, and a really firm grasp about the problem. The last thing I need to do is tell them what I’ll do to try to reduce the likelihood of something like this happening again.”

The conversation turns a corner as Matt asks how you plan for outages and prevent disasters. Stephanie jumps in, and reminds us all:

“If you don’t test your backups, you don’t really have backups. Similarly, if you don’t test your outage plan, you don’t have an outage plan.”

She suggests setting up brainstorming sessions with a handful of people from your team, appointing a “DM” (dubbed “Disaster Master” rather than “Dungeon Master” by Tyler), and running through possible scenarios. Keep an eye out for a Kickstarter in the near future ;)

There are definitely advantages to documenting incident reports along the way, but how do you balance the speed of talking through a solution out loud, and the value of face-to-face communication to build trust vs. the need to document things for posterity? Mark: How you interact on a day-to-day basis is also how you should communicate during an outage. The last thing you want to do is change your mode of communication when everything is falling apart and you’ve got high stress.

Matt posits that sometimes what’s a disaster for one company isn’t for another, because of their size, their logistical capabilities, etc., but also, sometimes what is being presented as a disaster isn’t actually all that bad.

Stephanie identifies the first benchmark as determining whether or not your users are hurting. “If they’re not hurting yet, you might have a disaster coming, but I don’t think it qualifies. But if your users are hurting, that’s when you really need to jump on board and get focused.”

Mark agrees, and adds that being able to quantify how many users are affected, and in what way they’re affected, is hugely important. “That’s different than monitoring. Monitoring may tell you that the server’s down, but it doesn’t tell you how many users that impacts.” He reminds us that when you’re working at scale, “services are down for somebody literally all of the time. “What is the threshold where it becomes a disaster? When do you need to start talking about it publicly and in status? Those are questions you really need to answer up front.”

Checkouts

Stephanie

catehuston.com, Accidentally in Code

USDS

Mark

Bridget

seedsavers.org - Organic heirloom seeds
“Common Ground and Coordination in Joint Activity” by David Woods et al

Trevor

Matt

Does Not Commute - iOS/Android game iOS | Android
Recreating highlights of Breaking Bad in GTA5 editor
I’m also obsessed with Hearthstone on iOS now (World of Warcraft card game thingy)

Dr. BOFH, or How I Learned to Stop Worrying and Love the DevOps

15 maj 2015 | min

Chris Read, Kevin Hubbard, and Yvo van Doorn are reformed BOFH’s (Basterd Operator’s From Hell).

Chris is back on the podcast again, this time talking about his expereince as a SysAdmin in past lives.

Kevin is currently the DevOps Engineer for BCycle at Trek Bicycle Corporation, and was a SysAdmin for 15+ years.

Yvo previously worked at classmates.com and McGraw Hill Corporation as a SysAdmin, and is now at Chef Software.

Before we get started, you’ll need to understand the origin story of BOFH. There were stories posted on Usenet back in the 90’s, supposedly authored by a computer operator named Simon whose sole purpose was to terrorize the users of his systems. The phrase “How great would my job be if it weren’t for the f***ing users!” resonated with many SysAdmins (and still does!).

Matt starts the episode off asking what it was like as everyone was starting out in this field.

Kevin: To me, it was sort of operating in a scarcity model. You had limited resources, and it seemed like anytime there was a new ask for an application, I immediately went to ‘How am I going to ask for the capacity to run this?’ and I would just get so frustrated. It boiled down to ‘How are we going to support this?’ That was my standard line when someone would bring up something new, and I wish I had trophies to give those people for all of their good ideas, because we just couldn’t get it off the ground with the resources that we had. It wasn’t a fun way to operate, but it was the most realistic view.

Chris: When in high school and university I was the System Administrator for the school systems. It was astounding seeing what damage to the system can be done – how people trying to do something could affect shared resources, and the after-effects of that. Most of the time it wasn’t malicious; it was due to ignorance, but it built up this mental attitude of ‘All users are just there to break things. We need to constrain them as much as possible, because when things break, we’re the ones that get shouted at.’

Matt: There was a belief that devs are stupid! All they’re going to do is break things, because they don’t care about the systems like a SysAdmin does, because they’re ours.

Yvo: Our devs were incentivized not to care because they were paid based on the amount of code they shipped. I’ve had some nightmare evenings trying to fix all of the problems.

Bridget then brought the conversation back around to incentives – are there situations when the incentives are diametrically opposed (or at least not aligend well) between the SysAdmins and the developers?

Matt brings up the point that developers are incentivized to build features, while SysAdmins are incentivized to bring stability, which at its most basic level is maintained by things not changing.

The viewpoint of ‘developers don’t know what they’re asking for’ is also a problem, Kevin reminds us. SysAdmins will often call the developer and explain why things work the way that they do, but won’t take the time to listen to the actual problem.

In reality, there’s a perception of other people touching “our stuff” and things will go wrong, but let’s face it: “there are all sorts of things that can go wrong that are often not a specific person’s action,” says Bridget.

Given that all of us here are supposedly reformed BOFH’s here, let’s chat about how things have changed, and what that process was.

Chris: I finally realized that my interactions with the developers were better if I went to them without the ‘clue stick’ and simply spoke to them, asking them if they realized the impact of their code. It finally clicked for me when I had to work together with the client-side SysAdmins as well as the developers at Thoughtworks. Our whole purpose was to get code written by two different development teams out into production, and it was only through being an advocate for both teams that I was able to build up a relationship with both teams and understand the value.

“It seems like DevOps has formalized the relationship between SysAdmins and developers,” says Kevin. “It seems like a much more natural, iterative process working with devs.” Because we’re working side by side, there’s much less going back and forth with having to figure out the direction and purpose behind projects, and simply getting to collaborate.

The “handing down stone tablets” philosophy not only no longer works… it has never worked!

In Yvo’s case, the change started to happen when the project management team was dissolved. Suddenly, a SysAdmin had to be a part of the development meetings, because there was no longer an intermediary passing information from one team to another. It immediately became more collaborative, and there was visibility into what was happening early on rather than being notified after everything was finished.

We’ve shifted from a mentality of ‘protection’ – teaching our “PFY’s” (pimply-faced youths) to protect their systems against the evil developers – to giving history lessons about how we got to the stage that we’re at now where we need to talk to all of the involved parties, and as Chris said, “having everyone focused on the goals, trying to see things from each other’s angles rather than antagonistcally.” This is how we, as a group, move forward.

“It takes a deliberate decision to shift,” Bridget observes. We have to be dedicated to teaching our PFY’s this new, collaborative way so that in the future, fewer people will start out with this BOFH mentality.

From there, we shift into How can we do better?

Matt asks, “We used to be this way – we’re better now – but what are some of the ways that we can still improve?”

“I want to be able to maintain this new flexibility that comes with DevOps, but I feel like there’s some decision-making that needs to be made as far as tools and standards go,” says Kevin. It’s a matter of balancing the old playbook of limited resources and mixing in the new cohesion and collaboration efforts.

We’ve done a great job of bringing in the greater teams of operations and developers, but most companies still have the one or two lone SysAdmins who are struggling on a daily basis to keep their heads above water. Yvo cautions that we need to bring them into the circle as well.

“If we can make their lives easier, they’re going to eventually go to another shop with the perception that being alone and supporting developers is not a bad thing, but right now they really don’t like life.”

Bridget’s money-back guarantee: “If you’re less BOFH-y, I promise you’ll be happier, or else we’ll pay for your therapy.”

Checkouts

Chris

Liz Keogh - Perverse Incentives
GOTO Conference Chicago 2015

Closing Keynote by Anita Sengupta from NASA JPL was awesome

James Lewis - “How I finally stopped worrying and learnt to love Conway’s Law”
DRW is hiring!

Kevin

Stache - new Trek mountain bike

Yvo

Hop Venom Double IPA - Boneyard</a

Bridget

Organic heirloom seeds
Common Ground and Coordination in Joint Activity by David Woods et al

Matt

Effective DevOps by Jennifer Davis and Ryn Daniels
asciinema

DevOps and Marketing

30 april 2015 | min

Shannon Smith is the marketing manager for 10th Magnitude, a consulting firm based in Chicago. She’s a self-described “liberal arts nerd” and fell into the tech world through her job at 10th Magnitude. She has a love for well-crafted sentences, song parodies, and general cleverness, which has translated into a love for marketing and the challenges that come with getting an effective message delivered to the “great abyss.”

Jason Hand is a DevOps Evangelist at VictorOps. He’s been in the IT space since college, but it wasn’t until joining VictorOps that he’s ventured into a wider role, working closely with both marketing and product.

Matt starts off this unique episode by posing an important question: “Why is DevOps something that people in marketing would even care about?”

Jason jumps in right away: “Obviously, the VictorOps tool is something we feel falls in line with a lot of the DevOps topics, in terms of the things we try to help engineers with. Being able to speak intelligently to the different subjects that fall within DevOps is very important, especially when it comes to marketing. As most of us know, we’re very averse to traditional sales and marketing tactics; we can sniff out those types of conversations before they even happen, and we’ll pivot and walk away.”

Shannon originally broached the topic during Open Spaces at DevOpsDays Chicago 2014. She was interested in the focus on community, but also intrigued by the jokes about how terrible traditional sales and marketing is. She wanted to know what she could do to combat the negative feelings surrounding these departments, as well as gain insight into what her team could do differently.

The group starts throwing out common pitfalls:

Using #devops on Twitter and thinking that in doing so, your company is part of the conversation
Packaging DevOps as a quick fix to deep-rooted problems
Trying to follow traditional marketing and sales “rules”

“Even for those of us who are heavily invested in this DevOps thing,” says Jason, “who talk about it on a daily basis, sometimes we don’t even understand everything – all the ins and outs of what is DevOps – because it’s not just a thing. It’s a way of getting things done. It’s much bigger than just one thing. We are very patient and empathetic to that thought process,” he continues, “because we deal with it all the time. That’s part of the challenge within marketing, is how to turn that conversation into a succinct way of explaining it.”

Shannon explains that part of the problem with having a succinct explanation for DevOps, as well as how to sell it, is that there are multiple target audiences. “While we do use more traditional marketing for ‘decision makers’ – people who are busy and don’t have the time to talk about these abstract concepts, we’ve recognized that there are two or three very separate audiences, and we have to make multiple different messages work.”

The point is, your message is going to be very different depending on whether your audience is practitioner, decision maker, or champion. It could be that your company caters to all three, but then you must have separate messages catered to each of these groups. Otherwise you’ll be trying to mandate change from the top-down rather than getting the buy-in from people doing the actual work, or vice versa.

In a conversation with Matt earlier in the week, Steve Pereira said, “I’m in favor of having marketers speak to what they’re actually selling, which is never actually ‘DevOps.’ DevOps is bathwater in which all of these things are babies.”

Given all of these different audiences, Bridget asks the “elephant in the room” question: “When you’re talking about the target markets, how do you build an understanding of DevOps among people who aren’t engineers?”

Jason mentions that at VictorOps, they have an emphasis on moving agiley across the company. Even the marketing team works in sprints, getting things done on a project basis and having team standups. DevOps isn’t just for engineers – it’s applicable for everyone across the company.

But how do we make that happen in a practical sense?

Companies that are doing this successfully often exhibit certain qualities:

Empathy to others, which allows silos to come down
Transparency between teams via chat clients and in person
Cross-company knowledge of what the top priorities are and what other departments are working on

This cross-company communication can be difficult at times, but ultimately it’s everyone’s responsibility to speak up if they see problems with the messaging. Bridget offers a solution: “If you’re going to have people external-facing in your organization, going out and evangelizing, they need to also be talking to the people inside the organization, who are developing the product or providing the professional services.”

With Jason’s role as DevOps Evangelist, he’s often out learning from the best. It’s his responsibility to bring the feedback directly back to the company and educate them on what he’s hearing from the community, and likewise it’s marketing and product’s responsibility to be open and willing to hear the feedback.

Collaboration and accessibility is what drives all of this, as Shannon pointed out. Being able to talk to people face-to-face is one thing, but using chat software to check in with the full company, or somehow checking in frequently with multiple teams is key.

For our listeners: If you work in a larger organization, we’d love to hear about how you’ve solved this problem of communication throughout the company. What’s resonated, and what hasn’t? Tweet your answers to us at @arresteddevops.

Main takeaways: Jason - Spend some time researching DevOps. Get to the DevOpsDays events that are happening in your area. That’s where I’ve learned the most. I’ve made some great friends and contacts. Also, these events have helped me have the sense of empathy not only toward the challenges of the marketing and sales teams, but also the conversations that are taking place among all of the business units, no matter what size company. I can now take what I’ve learned, both online and offline, and take those insights and thoughts back to my team. For the time-being, I’m the one internally who’s teaching DevOps best practices. It’s really a matter of absorption. You can’t sit behind your desk and understand DevOps. You really have to get involved with others.

Shannon - Truly the biggest thing that I’ve taken away in the past 9 months is really trying to build up our grassroots messaging and the way that we’re reaching out to people. I think the best way to do that is go to these community events – be involved, listen, share, have natural conversations, and be genuine. Having a presence in the community says a lot more than any little tagline.

Checkouts

*Shannon*

[Graze](https://www.graze.com/us) & [Naturebox](https://naturebox.com/homev2/) - when you need a 3pm snack
[Canva](https://www.canva.com/) - graphic design online tool
[Unbreakable Kimmy Schmidt](http://www.imdb.com/title/tt3339966/)

Jason

[Trello](https://trello.com/) - light-weight Kanban boards
[Buffer](https://buffer.com/) - tweet scheduler
[Mention](https://mention.com/en/) - keep a pulse on what topics people are engaging in
[DevOpsDays](https://www.devopsdays.org/) - local DevOps events

Bridget

[DevOps: A Brief History](http://peterjshan.com/posts/2015/04/devops-a-brief-history/)

Trevor

[David Bowie's Exhibition](http://davidbowieis.philharmoniedeparis.fr/en)
[Agent Carter](http://www.imdb.com/title/tt3475734/)

Matt

[Mac ID](https://macid.co/)
[Octotree Chrome Plugin](https://chrome.google.com/webstore/detail/octotree/bkhaagjahfmjljalopjnoealnfndnagc)
[PonyHoof](http://ponyhoof.little.my/)

What's New at Chef?

17 april 2015 | min

Seth Falcon is the Engineering General Manager for Chef Delivery.

Julian Dunn is the Product Manager for Chef Analytics.

Adam Edwards is the Engineering General Manager for the main Chef product.

Matt, Trevor, and Bridget gather at the Chef Newsroom at ChefConf 2015 to talk about the latest and greatest developments at Chef. They are joined by Seth Falcon, Julian Dunn, and Adam Edwards.

One of the biggest announcements at ChefConf 2015 was Chef Delivery. Matt asks what is Chef Delivery, and why is it interesting?

“Chef Delivery is a solution for continuously delivering infrastructure and applications, and it’s built on top of Chef,” says Seth, who’s heading up this new program within Chef. “We’re really excited about Chef Delivery. Successful software organizations use patterns to deliver software at high velocity collaboratively and safely. We’ve been able to distill some of those patterns into a workflow that we think will be easier for folks to adopt and learn.”

Seth continues: “The workflow is one that we’ve seen work, by working with customers to build these types of pipelines over and over again, and find those successful patterns. The overall workflow begins on a developer’s workstation, where they make a change, they do some local testing, and then submit it to the system, where some automated verification tests run. The job of those verification tests is to determine whether it’s worth the time of a human to do some code review on that change. Someone can then do some code review to approve the change. At that point, Delivery will build an asset for us that we could release. The workflow for building that asset is simple: do a merge onto the target branch (usually the master), rerun the same verification tests, which usually consists of unit tests, lint testing, and syntax checks, build that asset, and publish it into a repository where it can be fetched later. Then Delivery will provision an acceptance environment, should you need one, and deploy that asset into that acceptance environment, and run some tests to make sure that the deploy was successful. If it was, it waits there for further instruction. The last step is clicking the ‘Deliver’ button. That sets the system in motion to get the code all the way out. It first goes into a ‘Union’ stage, where if you had a number of projects that had some interactions, you’d be testing them together at their latest version, and making sure they’re good. If those tests succeed, Delivery rolls automatically for the rest of the stages – into a rehearsal environment, and then into a delivery environment.”

Interested in a more detailed description of the workflow? Seth continues to talk through some of the logistics in the podcast as Bridget and Matt ask questions about particulars. You can also check out this video about Delivery:

Seth concludes with, “A lot of what we’re providing here is an accelerant to teams, to give them that system that will allow them to move quickly and learn how to move quickly in that way.”

We transition into asking Adam Edwards, Engineering GM for the Chef product, what’s new in his world.

“There’s a lot of new stuff just coming out over the last few weeks,” says Adam. He goes on to detail just a few new features:

Policyfiles - solves problems with workflow and provides a way to specify in a specific file which cookbooks you want
Test Kitchen on Windows
PowerShell DSC Direct Integration
Chef Provisioning (neé Chef Metal) update - does more with containers and Azure
Chef Nightlies - releasing Chef Server nightly on apt and yum repos

From here, we move into Chef Analytics, and turn the mic over to Product Manager Julian Dunn. He gives us a quick background on Analytics, explaining that it “gives you a way to visualize, query, and report on the events stream that’s coming from your Chef data. Analytics lets you not only visualize that data, but track events in that stream, and then handle them in various ways.”

Matt brings up his favorite part of Analytics, and brings up how it’s closely affiliated to security: “You can take those same audit rules that you write in Analytics, and apply them through your pipeline to test them there. What’s nice is that it’s only one thing to write.” It keeps things simple and straightforward, rather than needing to search through a huge amount of rules and regulations to ensure that everything is accounted for.

Julian agrees: “You can think of security as just another aspect of quality. We understand that you can’t get quality if you try to bolt it on to the back of a system. How many of you have worked with applications that didn’t have tests originally, and the software is just poor quality? We tend to treat security in this backwards way, where we think if we don’t build it into the system, down the road we can just do an audit and we’ll magically get compliance and security. If it’s not a characteristic that’s already there, it’s very difficult to achieve those directives.”

In addition, Analytics allows customers to report metrics and characteristics about your infrastructure to business owners. “One of the things we often hear from customers,” Julian says, “is how do I measure how successful I am at Chef?” If you’re able to use Chef Analytics, it provides you with a direct way to illustrate the ROI of investing in this type of technology.

This business aspect of Chef Analytics was one of the core announcements regarding the Analytics Product Suite at ChefConf, specifically highlighting the integration with Splunk. Julian explains the rationale behind this integration:

Many enterprise customers are already using Splunk
Splunk makes it very easy to draw visualizations and make inferences from data

Matt segues into an overview of ChefConf 2015 and asks everyone for their insights on this year’s event.

Bridget: One thing that stood out for me at ChefConf is that there’s a very unified community feeling. At a lot of conferences (that are very wonderful conferences!), there are very specific tracks, or specific groups of people who end up mixing more than with others, and at ChefConf, it definitely seems like there’s a very broad community who are all interacting with each other.

Trevor: I’ve been rocking the hallway track… and like Bridget said, everyone’s been fantastic. I’ve spoken with so many people here who I thought would never want to give me the time of day outside of our show, where we kind of have them pinned in a corner (laughs). The Open Spaces in particular were fantastic. Brandon Burton brought up a very sensitive topic: mental health, burnout, suicide… and it was such a breathtaking and emotional experience to hear everyone share personal experiences around that.

Julian: It’s really great that even as we’ve grown as a company and a community, we’ve retained certain attributes about that community. I think one of those is that there’s a little bit of quirkiness, and a little bit of uniqueness, and fun. Backend automation and IT has not necessarily been the most fun arena to work in, and I think this is a breath of fresh air to that sector. I hope we’re able to retain those attributes even as we continue to grow.

Matt: Some of this experience is really hard to explain. You can’t explain what the Las Vegas strip looks like at night, even if you’ve seen it in movies, until you’re there. I’m not saying this is like going to Vegas, but just like no one can describe the Matrix to you, you have to experience ChefConf for yourself.

More information on ChefConf:

Links:

DevOps Culture Change With Bill Joy

1 april 2015 | min

How to Change the Culture of an Organization

“Change management rests with the how” – Bill Joy

DevOps revolves a lot around what an organization and culture should look like. We talk about it on just about every episode of this podcast. Something we tend to skate around though is the how. How do you change the culture of an organization?

Matt got to sit down and have an incredible conversation with Bill Joy of the Joy Group about the how of influencing change. We didn’t talk about what changes companies needed to make, we talked about how to get companies to make the changes. Bill shared the often overlooked fact that change influence isn’t restricted to top levels of power in a company.

If I were to ask you right now, “What would make your company better?” I’m pretty sure your mind goes into overload with all of the things that you would change if you could. The thing that most people fail to realize is that they can make the changes, or at least influence them.

It doesn’t matter what level of the executive chain you sit at, you have the ability to influence change.

Any senior leader can mandate any change. If you’re a senior level executive, you can walk into the office and implement new strategies, create a new company wide rule, begin rapid movement events, or whatever you see fit to accomplish your ‘what’.

But it can’t stop there.

In mandating a change, you have to be very aware of how it will affect the company itself; the infrastructure, the culture, anticipate any resistance, who are the key stakeholders are, etc. If you don’t allow for these, your change could be very well unsuccessful and you lose a great deal of credibility as a leader.

Non Executives Can Influence Change Too

You might remember back when we had John Allspaw on our Etsy episode. (Give it a listen, if you missed it.) When asked “How do you implement developments in an organization when I’m not on the top; I’m coming from the bottom of the chain.” Allspaw answered: “I don’t know. I’ve always been in charge.”

What a great position to be in. However, not everyone is a high executive with the capability to mandate change. This doesn’t mean that you can’t influence change.

The question arises then: How do you influence change in your organization from the bottom?

Ask yourself: Who are my key influencers?

Change management is the influence of authority. That said, as you talk to the person you have deemed to be your authority, you have to remember the power of empathy.

Patrick Debois once said “We should stop calling it DevOps and call it common sense.”

Here’s the problem with that, not everyone is going to see it your way. Common sense is a relative term. If during any of your influencing meetings you find yourself thinking “This is crystal clear to me, why aren’t they seeing it?” that’s more about you than it is about them.

When you make the decision to mandate, or in this case, influence change, it is your responsibility to present your ideas so that people can see them clearly. If they aren’t seeing them clearly, there is a good chance that you aren’t communicating them effectively and didn’t take into consideration who your audience is.

Identify Your Audience: It’s all in the personality

One of the most popular personality tests around is the MBIT (Myers Briggs) which identifies the strengths and weaknesses of a person’s general personality. There are a bunch of alternatives to the test, but one that works really well with change management is the DiSC Profile, which you can take for free here. The test measures your strengths in four areas: Dominance; Influencing; Steadiness; and Compliance. All of which have different traits associated with them.

Before you attempt to influence another person, you need to have a firm grasp on who you are. For example, if you’re heavily dominant, your natural approach isn’t going to be effective on someone who is compliant. You have to adjust your approach and presentation to your audience.

Once you understand yourself, know your triggers. As a dominant go-getter, someone who takes a while to process things or lacks energy may really test your nerves in a social situation. Realize this upfront, prepare for it.

Read; Assess; Adapt & Tweak

Make it a point to put some time into reading your key influencer. Take special note of the tone in their emails, the speed of their decision making, their general demeanor, etc. These are going to give you clear indications of what personality traits they have, and which you need to play on in order to effectively communicate with them.

From here, you learn to adapt. Adapting is essentially wrapping the package up. Maybe you thought at first that you would have to have a presentation based on making things less structured to allow for more success. But, you realized you’re in a room with the key influencer who is very compliant; he/she likes configuration and organization. You then have to be ready tweak your strategy to make the presentation more effective and geared towards them..

Taking On The Role of Consultant

Regardless of whether you're a hired expert brought on to assess the state of a company or if you're attempting to influence change internally, you're going to be taking on the role of the consultant. As the consultant, it's important that you remain committed to your views and intentions regardless of the initial feedback that you may get particularly if you're trying internally influence change. The key is to look at your key influencer as your client instead of your boss for the purpose of this project.

If your client (who happens to be your boss) shoots down your ideas and offers a counter plan, you have to be ready to channel some boldness and stand firm in your beliefs. Instead of arguing back and forth or worse, completely backing down because of his/her position above you in the company, you could say something like “I see what you’re saying, but if we do it your way, this is what is going to happen”, and then lay out how their plan is flawed.

It's Not The Size Of The Company That Matters, It's the Agility

When considering the speed and likelihood of a company to change, size shouldn't be the deciding factor. It's seems logical that a larger company would be more difficult to change than a smaller one, but often times it's the smaller company that shows more resistance. Don't let size lead you to any assumptions. Instead, there are 5 key areas that you should take a look at:

Risk Tolerance: Regardless of the size, does the company encourage risk? Do they punish risk-takers?
Speed of decision making: Is the company bound by politics? Hierarchy? Take notice to how the new hire process happens.
Levels of authority: Watch the style of how many people need to weigh in on a decision? When you go to your key influencer, is he/she going to refer the proposed changes to a higher level? Will that level then refer is even higher?
Empowerment: There is usually a tendency to refer empowerment higher. empowerment to the next level up. Can you start changing the direction of empowerment down in an organization? What you want to do as a consultant is be able to go in and say something like "Can't this be solved by level 2 instead of going to level 6?"
Voice of the customer: Does the organization's customer base see the company as helpful? Are they happy with the customer service? How do they give products? How do they handle product returns? Are they satisfied in general? What's often found is that organizations are more flexible with their external relations than their internal relations.

Compliance VS Commitment

Whether you’re doing the influencer or the implementer of change, you’re going to come across two types of people who “sign on” to your changes.

The Compliant One: The compliant person will sign the documents, put their time in, “do their TPS Reports”, as Matt said in the podcast, punch the clock day in and day out. They’ll do it because they need a job, or are too lazy to leave. Whatever the reason, they’ll make the changes, but won’t really care much about it.

The Commitment One: The commitment person will simply, “Believe in the TPS Reports”. They’re dedicated to the idea of change and are committed to the success of the company.

As you have more commitment, the success of your company will be exponential.

As Bill said; “Find satisfaction in the pursuit of commitment.”

The best change influencers are those who don’t see people as something they ‘have to deal with’. They’re authentically interested in them, and enjoy the pursuit of the how of change more than the change itself.

Checkouts

Bill

Matt

HabitRPG

Starting a New Devops Job

14 mars 2015 | min

Tonight’s episode featured a special guest host, Julian Dunn of Chef.

Our panel: Ryn Daniels - @rynchantress - a web operations engineer from Etsy. Jake Champlin - @grubernaut - an operations engineer from Minted. (Younger than Trevor!)

Matt asks Ryn what it was like going to a place like Etsy - the gold standard! - what they expected, and what it was like when they got there. Ryn says that years of reading codeascraft and seeing etsy engineers speak at conferences meant that they were excited to start but also felt a little impostory. “Everyone knows that devops is the internet and the internet is cats - I show up my first day and my desk is covered in cat pictures.”

Jake talks about how devopsdaysPGH was his first tech conference, where he got connected to the community and met his future boss Alex Nobert. Julian asks how Jake got connected to Minted, and Jake talks about Bridget introducing him to Alex and how Minted seemed to have a great culture. Trevor asks what stands out about Minted’s culture, and Jake mentions that it’s woman-owned and has a strong focus on the product, and how everyone’s motivated to make a quality product and it’s reflected in your infrastructure.

Bridget suggests this might be the inverse of Conway’s Law, while Matt asserts that most “inverse Conway’s Law” discussions just end up proving it. Jake mentions how the community of Minted votes on some products, and the engineering team tries to bring that same excellence throughout.

Bridget asks Ryn how Etsy’s engineering practices influence or are informed by the culture of the company. Ryn mentions Etsy’s B-Corporation status, which means they are focused on social good, transparency, giving back to the environment - and that’s reflected in a lot of what they do, from codeascraft to blog posts to involving sellers in the community.

Both guests give a short version of what their company does; Ryn says that aside from the monitoring and sending everyone to Velocity, Etsy is the largest marketplace for handmade and vintage goods. When Jake explains Minted, it’s revealed that as “lazy podcasters”, we didn’t realize that Minted was as artist-focused as it is, providing artists a platform and community - Bridget thought it was primarily for paper goods, while Matt thought it had something to do with money.

Bridget asks Ryn about their decision-making process to decide that Etsy was right for them. They said that even though Etsy folks they met at Velocity didn’t know them, they were welcoming and never condescended to them. (Also, pink-haired thought leadership.) They also admired how Ian Malpass was mentioning at devopsdaysMSP how he wanted to put together a class for effective male allies inside Etsy.

Similarly, when Jake went to devopsdaysPGH, the way people were open and welcoming is what made him know he was in the right place. (A restaurant-related digression led to a shout-out to Jon Cowie’s knife-spork.)

Matt mentions having the feels and being excited about getting a chance to work with Julian Dunn! and how when starting a new job, when you feel intimidated, people in a good culture are going to reach out to you and make you feel welcome. Julian asks our guests what the experience of starting at one of these devops gigs was like.

Ryn talks about how starting at Etsy is less-stressful than smaller jobs in the past (since payroll isn’t an open question) and then mentioned the engineering rotations that allow new people to “bootcamp” with other teams, to increase understanding across the site. They also mentioned the “first push” program, where everyone (even non-engineers) learns to deploy a change to the site.

Jake’s working remote, and his onboarding process was like being thrown into the deep end of the pool. He felt impostor syndrome at that point. Bridget agrees that it can be an intimidating feeling, and asks Jake how he deals with that. He says that he’s working with really smart people and is happy to learn new things every day.

Trevor asks about cultural cues. Ryn says that they have a chat-heavy culture, and because there are so many remotes, even the in-person team will communicate as if they are remote, so you get to know what’s going on with the team and what their favorite cat gifs are. Trevor mentions introducing hipchat to a client and how it caught on quickly. Jake mentions that although only ops and qa are remote, everyone at Minted uses hipchat. Matt believes it’s frustrating when there are people in an org aren’t on chat, and Jake agrees that having the whole org there (HR/payroll, etc) makes communicating easy, even as a remote employee.

Ryn: “I do get to sit ten feet away from John Allspaw, so I’ve got that going for me.” Matt: “If you’re playing the Arrested DevOps drinking game, that’s a drink.”

Julian asks what are some of the downsides of a chat-heavy culture. Bridget mentions that at DramaFever, people have sometimes found chat to be distracting, if people are wanting your attention all day, and Ryn points out that at Etsy, it’s considered okay if someone needs to be heads-down working on something, either turning off notifications or signing out of chat. Trevor mentions that working with a global team can mean unintended awakenings from 3am @-mentions, while Ryn and Jake are both in cultures that encourage not having chat on their phones. Bridget says that at DramaFever, the solution is spaces in someone’s name so as to not alert them during off hours.

Jake talks about how being in an oncall rotation instead of being oncall 24/7/365 is great, and after a week of oncall, at Minted they get the next Friday off (and their co-workers will kick them out of chat if they join). Bridget asks Ryn about their blog post on work-life balance, and Ryn says they wrote that about disconnecting, and Etsy also has a culture of asking people to take care of themselves (where they threatened to take away their VPN access because they were working while sick).

Julian, who reveals he is stranded in Chicago and that’s why he is on the show, turns the conversation back to finding good roles and asking what role a recruiter plays. Ryn says that an Etsy person being oncall and troubleshooting during a Sysdrink meetup in New York attracted a number of applicants. That, codeascraft, speaking at conferences - showing what an org does, instead of just saying “we’re hiring” - works better.

Bridget asks them what makes a job a place they know is right for them. Jake says the culture, and working on interesting things with smart people. Ryn agrees, and says that their first week at Etsy they got to start contributing right away. Bridget points out that new people have a power that people who have been there longer can never get back - the power of not knowing how things are “supposed” to be (and the ability to write documentation to better serve a “don’t know the answer” POV). Ryn says that breaking Nagios Herald their first week (and then they also got to experience Etsy’s culture around blamelessness.) Jake is in a more greenfield situation, and so he was able to start contributing immediately out of necessity.

Bridget asks for the panel’s best advice for people who are interested in a “devops” job, want to find such a job, etc. Apparently the answer is having Bridget introduce you to people and/or convince you that you’re totally good enough to work somewhere. [Note from Bridget: this may not scale.] Both Jake and Ryn point out that connecting with the community on Twitter is a really valuable place to start making connections. Matt points out that liking your co-workers isn’t so much “nepotism” - you don’t have to party with them - but you spend a lot of time with your co-workers, so you’re going to want them to be people you don’t hate. Matt says, “It’s not know the ‘right’ people, it’s just ‘know people’.” Jake points out that at PGH, Mark Imbriaco and Ben Rockwood spoke to him as if he was a friend, even though they are prominent in the community. Ryn encourages everyone to use Twitter.

Checkouts

Ryn

Jake

Julian

Trevor

Laphroaig PX Cask Triple Matured Scotch
The Book of Mormon
How we upgrade a live data center - http://blog.serverfault.com/2015/03/05/how-we-upgrade-a-live-data-center/

Bridget

Fat Bike Birkie - http://www.birkie.com/bike/events/fat-bike-birkie/
Bike camping in Napa
30 Days of Biking: http://30daysofbiking.com

Matt

The Goat Farm - Enterprise DevOps podcast by Michael Ducy and Ross Clanton.
DevOps Checklist - from our buddy Steve Peirerra

Docker! Docker! Docker!

27 februari 2015 | min

James Turnbull describes Docker as “a solution that is built by people to be usable by people, as opposed to some of the previous containerized solutions which were built by engineers to be usable by a very small subset of other engineers”.

When you might want to fly less… “when you start to recognize the airport lounge staff and the flight attendants on the New York-San Francisco route and they start to recognize you.”

James wrote The Docker Book to allow people of varying skill levels to quickly understand how to use Docker and what the practical applications could be for them. It’s intended to be a practical how-to guide.

At Kickstarter, developers have a dockerized replica of production on their laptops.

Matt asks if Docker can be only used in completely new deployments designed for Docker from the ground up. James points out that if you have existing infrastructure tools, it’s simple to create Dockerfiles from them.

The night before 1.0 launched at the first Docker conference in mid-2014, James removed all references to “don’t use this in production” from docker.com.

James mentions that Fig (soon to be renamed to Compose) helps with modeling multi-tier architectures locally.

James says, “People kinda forget the past and go, “oh my god Docker’s a pain in the ass to use”, and I’m like “compared to what, exactly? Compared to your previous build, or compared to you shipping around 10 ISO files and running Vagrant and 20 VMs on your local machine?”

He continues, “It wasn’t that long ago that the dark ages were real. I’m not suggesting that Docker’s a panacea, but it’s certainly a step in the right direction.”

James points out that something like Elasticsearch does well in Docker, since “it’s a bit of a fiddly thing to build, with the right version of the JVM, right version of Elasticsearch, prepping all the data, etc”.

James highlights continuous integration as a “sensational combination” with Docker.

On the controversy, James points out there will always be hype and people claiming “this is a revolutionary technology that will cure world hunger”. He says, “I’m fond of saying that Docker is a powerful tool to help you in your development life cycle […] not every workload in your data center is well-suited to Docker.” James doesn’t make technical architecture decisions based on the writing of tech journalists or blog posts, but rather by testing and evaluating the relative merits of a given solution.

In the case of Graphite, James would run carbon-relay and carbon-cache inside Docker containers, but he’d point them at a physical machine with SSDs to actually write the whisper files.

Matt read a blog post and reactions on reddit and wanted to see what James thought of the concerns around security and operability. James points out that empathy for developers is something sysadmins need to cultivate, because you don’t manage infrastructure for infrastructure’s sake.

James points out that the main reason developers ship code that doesn’t work in production is that they have no fucking idea what production looks like because there’s this grumpy asshole that manages production and they’re terrified to go ask them a question. Bridget says that as such a former grumpy asshole, she’s much happier when the devs aren’t afraid to talk to her.

James mentions that Docker containers are not virtual machines and should not be used to separate security concerns, and you should secure the host the containers are running on.

Matt: “I’m not suggesting that this [security concerns] is why DOCKER BAD…” Bridget: “Race conditions with devicemapper is why DOCKER BAD.”

James: “[PCI/DSS] is a low bar. If you followed simply the regulations for the compliance stuff that related to PCI/DSS, you would be running a massively insecure system.”

James points out that “owning” the standard gives one access to the marketing around an ecosystem. He also thinks that even if Rocket is a better technical solution, Docker has more traction.

Bridget: “So when I feel ranty about Docker and devicemapper, I should submit some pull requests.” James: “You should talk to Michael Crosby… Michael Crosby is currently in San Francisco somewhere going you motherfucker.”

James sees Amazon and Microsoft’s embracing of Docker as a great driver of revenue towards these cloud providers, if it gets developer code to production faster. They aren’t following hype; there are transparently obvious business reasons to do it.

In terms of skating to where the puck is going to be, James suggests looking at orchestration, software-defined networking, software-defined data centers - people building that sort of thing with Docker components. Docker Compose, Docker Swarm, people moving up the stack to manage different levels of abstraction.

James: “I challenge you to find a LAMP stack site where 80-90% of the configuration files aren’t identical - our secret knowledge of what to tweak isn’t as valuable as we think it is.”

Check outs

James

Jason Dixon’s Monitoring with Graphite book
The Art of Monitoring - James's upcoming book

Bridget

Spent a week with my Philly & NY co-workers, went to the 3rd annual DramaFever Awards show, sang more off-key karaoke.
Docker to Ducy Chrome plugin

Matt

Was in PDX for the first time last week for Agile Open Northwest. Led an improv open space. Got a tattoo. Met the founder of Voodoo donuts winding a clock. Such hipster. Very Portland.
kitty gem
Better Call Saul - new AMC show

DevOps in a Microsoft World With Jessica DeVita and Jeffrey Snover

13 februari 2015 | min

What are some of the challenges traditional Microsoft IT Pro’s deal with moving to a more automated DevOps pattern?

Jessica:
- Hard to tell which tools are really going to make their lives easier.
- Are the cultures of the companies benefiting the human side of the IT Pro?
Jeffrey:
- Because Microsoft has great GUI tools, they become the biggest strength and weakness of the DevOps/IT-Pro
- The process of using a GUI is much harder to replicate in documentation. Because most of the community uses powershell commands, Microsoft IT-Pros really need to get on board.
- IT-Pros are never done with learning. If you don’t want to learn anything new, get into the lumber business.

Microsoft has been making more open source integration moves, and changing philosophies to accept the Open Source community. “What up with that?”

Jeffrey: “The body follows the head”
It helps when you have a leader with a fresh approach who focuses on customer service and helping users within the community
Jessica: It is really exciting to get behind a leader that is welcoming to the communities.
It is refreshing to see Microsoft becoming a software company, not a “Windows software company”
Microsoft wants you to be successful. Tools such as RESTful APIs are becoming available across all OSs.

What is the acceptance level of the OpenSource movement within Microsoft?

Jessica: Whatever you’re running, we can host it for you

Traditional Configuration Management in Microsoft has been difficult. What are the plans?

Steve Morowski (http://stevenmurawski.com/) has good info for those interested in DevOpsing with Windows.
Current Microsoft tools are really good for enterprise, client management. Not so good for data center management.
We need something different, that is simple, and usable.
The problem is, everyone wants to do configuration their way. They want to be the CTO of their servers.
Jeffrey describes the creation, and idea conception of a Microsoft Configuration Management platform that takes into account the deep differences between Linux, Unix, Windows. Describing different tools currently available, their faults, and how they might be able to connect them for modern, DevOps oriented, Configuration Management.
The ability of chef and puppet, etc. are beneficial because of the ability of devs to pick it up, version it, and insert small parts of just what they need into the configuration.
Jessica: We are getting to the point where Microsoft DevOps engineers are adapting the powershell. Until the powershell is adopted by IT-pros, modern DevOps tools will be a difficult push.
You should already love powershell.

How can people get more comfortable with powershell?

Matt: It is not a scripting language. It is the way you interact with a system. Don’t write scripts in bash, write commands in bash that emulate the scripts.
Jeffrey: Don Jones: Powershell in a Month of Lunches (http://morelunches.com/2011/04/01/learn-windows-powershell-in-a-month-of-lunches-1st-ed/) Step by step people get it, or they don’t. Managers really need to promote and reward the people giving you the IT that you want.
Poweshell makes your environment repeatable, automatable, stable, etc. It is the future of the IT pro, and people must adopt it.

Are we automating ourselves out of jobs?

Jeff: The cloud is a great, cheap place to offer undifferentiated IT, however, if you can provide differentiated IT you are practically printing money vs. the cloud.
Jessica: We do need a healthy fear. Not of automation though. Be scared of more interesting things. You need to learn automation.

How do we work with Microsoft when its just not the best for DevOps-ing?

As more people us Microsoft, the more Microsoft changes. Jeff discusses the many ways in which Microsoft is using flexible R&D to make a push for DevOps tooling, as well as some tools coming down the pipeline.
Jessica: When choosing a tool, the longevity of the tool and the community around it is critical.

Why can’t I copy a file to a server using WinRM?

Jeff: Come talk to me at ‘Build and Ignite’.

Transcript

Checkouts

Jessica

The SoCal Linux Expo - Scale13 - a DevOps day Feb 20th - she has a discount code
The Field Guide to Understanding Human Error (Dekker)
Lean Enterprise book (Jez Humble)

Jeffrey

Hardcore History podcast - I’m in love with this podcast. Dan Carlin is an awesome storyteller.
Brain Science Podcast - (Cool podcast about the brain. I was just telling someone about this today)
http://www.microsoftvirtualacademy.com/training-courses/getting-started-with-powershell-3-0-jump-start This is the start of a 2 day training session on using PowerShell. It is one of the most widely viewed jumpstarts ever.
http://www.leeholmes.com/blog/2011/04/01/powershell-and-html5/ One of my all time favorite PowerShell scripts.

Trevor

FCC Ruling on broadband
Windows 10 on Raspberry Pi 2

Matt

Kitchen-windows is almost a thing! If you want to play with it, check out the Windows cookbook at http://github.com/opscode-cookbooks/windows
BitTorrent Sync - http://www.getsync.com/
Yes, Please by Amy Poehler

Hiring in a Post-DevOps World

23 januari 2015 | min

Incidents and Accidents: Examining Failure Without Blame

16 januari 2015 | min

Dave is at Next Big Sound, which does analytics for creative industries, and he’s seen a few orgs handle failure well, and a lot of organizations handle it poorly. He got interested in blameless postmortems and human factors in discussions with John Allspaw of Etsy, and Allspaw influenced him to read the work of David Wood and Sidney Dekker on human factors. He is writing a book for O’Reilly called Being Blameless.

MCR works at Etsy now, but has spent a lot of time consulting at various firms where he’s seen failure handled with blame. He points out what Rt. Lieutenant Colonel Scott Snook said in Friendly Fire, a book about when two US helicopters were accidentally shot down, that failure is part of complex systems.

MCR: “I work at Etsy, and that’s what we do - we examine failure as a learning opportunity.”

Dave is running his next workshop on Awesome Postmortems in NYC on February 12th, in which

Dave: “Sidney Dekker’s Field Guide to Understanding Human Error is probably the most important book for people like us, meaning people that are in the IT world - it’s very accessible and gives lots of examples from fields outside of IT, but they’ve very relevant to what we do.”

MCR: “Failure is gonna happen. It’s not a matter of if something is going to fail, it’s a matter of when it is going to fail.”

MCR mentions the different categories of failures - those that “fail closed”, that are easy to detect, like disk filling up, and “fail open” - the surprises. He mentions some of the techniques Etsy uses - an IRC warroom, Vidyo video chatting, to resolve an immediate issue. After the immediate issue is solved, the learning begins.

MCR: “We celebrate failure as much as we celebrate success here. […] The three-armed sweater is given to the person who most spectacularly impacted the website in the year.”

On the topic of why to do a blameless postmortem, MCR points out that it’s for learning, and there are both technical and human factors. Dave points out that blaming a person short-circuits the learning. Claiming that a person is the cause of the outage feels like a good story, but it’s not true.

Dave discusses root cause and mentions Allspaw’s excellent blog and a specific post about there being no such thing as a root cause, and Dave disagrees. He believes that outages are caused by change, and the systems with which we work are fundamentally changeable. “The impermanence of systems is the reason that they both function and malfunction.” Mike counters by saying, “Is there really a root cause for something that failed? If a hard drive dies, it’s the same hard drive. It hasn’t changed.” They both agree that it’s a philosophical rabbit hole.

MCR notes that as Etsy grows, they’ve found that user-impacting, service-degrading issues are when they do postmortems, and even if not user-impacting, if they can learn from a failure it’s worth doing one. Dave says, “The more we learn about the complex systems within which we work, the better we’re able to operate them.”

Within a week or two, according to Dave, is common practice of a time in which do the postmortem. MCR mentions that it’s important to write down the timeline almost immediately, definitely within a day or two, but doing it while someone’s amygdala is still triggered (and they are upset) is too soon. Dave points out that the facilitator of a postmortem sets the tone, including reminding people of hindsight bias, and at Next Big Sound they use a specific framework document which Dave will share. He also mentions defusing stress with empathy and humor.

On the topic of evaluating anything you do, MCR mentions that Etsy created Morgue because any department across Etsy can apply these techniques to learn. Dave points out they do retrospectives as well as prospective review at Next Big Sound. MCR says Etsy does both an architectural review and an operability review ahead of time. Dave mentions that answers in prospective reviews can be biased in a positive way, whereas in a “premortem” we imagine things going badly, and try to determine what could lead to that: in essence, harnessing hindsight bias to work for us.

Bridget forgets what decade it is and claims to have seen a presentation at devopsdays 2003. That would have been a nifty trick, since the first one was in 2009. :)

Check Outs

Dave:

Field Guide to Understanding Human Error, Sidney Dekker - new edition just came out Dec 28th, 2014!

Mike:

Bridget:

Did a lot of reading last week! Beside the aforementioned Dekker book, I read Web Operations (edited by John Allspaw and Jesse Robbins), Jon Cowie’s Customizing Chef, and the first three chapters of Jason Dixon’s Graphite book. All good stuff.
The Boundary Waters Canoe Area is great in the winter too: I like Gunflint Lodge: http://www.gunflint.com/

Trevor:

I was on vacation and delightfully disconnected. It’s been pretty awesome. Got a new Kindle and have been reading Game of Thrones before Matt accidentally (though at this point it’s my fault) spoils something.
Set up kegbot at our new office, will be doing it’s grand opening later today :) Metrics about office beer / root beer consumption to come!

Matt:

Been on vacation, which is great. Doing the Dadops thing although I was sick for most of it, which was not delightful. However, I did see Big Hero 6, and Trevor was right about that.
Book I’ve been listening to on audiobook is The Challenger Sale by Matthew Dixon and Brent Adamson http://www.amazon.com/The-Challenger-Sale-Customer-Conversation/dp/1591844355

A Year of ADO

1 januari 2015 | min

The Database: The Elephant in the Room

9 december 2014 | min

DevOps in the Enterprise

19 november 2014 | min

Git 101 With Emma Jane Westby

13 november 2014 | 58 min

Microsoft is open-sourcing .NET and creating the CLR for Mac and Linux .Net core5 is the new framework for 5, you can ship your own version of the app.

There is now a free version of Visual Studio — Visual Studio Community for open source developers and students. In fact, it is all happening out in the open on github.

Emma Jane is a long time listener of ADO! She has been teaching version control for many years with specific emphasis on the communication behind version control in teams. She has since switched to distributed version control such as git. Her aim in her teachings is to create resources that make git ”less painful” than it currently is.

Distributed vs Centralized Version Control

Emma Jane:

In distributed VC the DB that contains the changes, exists on the local system and I can have multiple connections to multiple DBs with other versions.
Centralized is all in a single DB, locally.

How is Distributed VC relevant to DevOps?

Matt: Many people hold the theory that you cannot have “The DevOps” without distributed version control. It implies communication through teams, so what is the validity of that statement.

Emma Jane:

Git is not the only VC option out there, but it is the most popular currently.
You need to assess your team and your project, along with the related expertise and community support, and go with the one that fits your needs.
As soon as you say “can’t” someone will prove you wrong.

Testing

Matt:

The whole basis of git-flow is based on the fact that you can’t trust your contributors. Especially with open source. It sends a message that says “I don’t have to test my shit, because you’ll do it for me.”

Emma Jane:

If we are talking about testing, you need to have full coverage testing of whatever your product. Many testing frameworks allows for 99.9% accuracy on the tests, but that .1% causes you not to trust your tests. This makes it really hard to get reliable CI into the dev process.
For that matter, Devs shouldn’t trust themselves when it comes to pushing code, you should always rely on testing because everyone is going to make mistakes, and humans might not catch them.
Git allows you to have control over the pushed code.

Trevor:

There should be no permissions. All developers should have the same permissions and the flow should go through QA.

How do you learn git?

Emma Jane:

All kinds of people are interested in learning git. But mainly:
1. Someone who is on subversion and wants to change to git
2. Someone who has been told to use git, but they don’t know how to run command line tools.
3. CTO or management types that know they want to use git, but they’re not really sure where to go from that decision.
In order to identify how your team will most efficiently use git, draw out your team flow and identify where efficiency is being blocked. Is re-basing causing problems? Is a PR sitting out there for too long? Use those as discussion points with your coworkers.
You cannot introduce creativity when you are just told to memorize commands.

Emma Jane: - Use Interactive Add! It allows you to split up your diffs into different commits. So you don’t end up committing a huge chunk of features that should most-definitely not be committed together.

How do I get set up?

Look for the right git-flow based on the type of deployments you are going to be using to release the software. Are your deployments feature based? or time based? How important is a rollback?

Your code should always be deployable in a CD framework. You are only rolling forward, you have one master branch, and feature branches, how can you have correct and fast CD if you have multiple branches before the CD process starts.

Your git setup should be directly related to your infrastructure. The git releases and flows of a team of 1 is going to be massively different than the git flow of a large team for a Could Provider.

Things you should know (about git):

Rebasing:

Rebasing allows you to recombine how your commit chunks are strung together. It takes all the commits of a branch and
Great for when you are adding too many commits.

Git Bisect

You can take out commits individually, and assess if the commits are in a working state.
However, if you do not have full commits, for example, commits when you are just thinking about something, it will be much harder to assess the state of the commits individually.

Source of Emma’s talk about Git: http://github.com/emmajane/gitforteams Post version: http://24ways.org/2013/git-for-grownups/ Recording: http://prague2013.drupal.org/session/git-makes-me-angry-inside

Emma’s rant about storing the history of your project: http://gitforteams.com/resources/evolution-social-coding.html

GitHub conversations: http://guides.github.com/introduction/flow/

Check-outs

Emma

Kaleidoscope mergetool - because of image diffs
Sketch training for techies - because of impostor syndrome for drawing
GNOME wins - go open source!

Matt

Teamocil - generator for tmux sessions
The NoPhone - kickstarter for a “technology-free alternative to constant hand-to-phone contact”
“It’s not a promotion, it’s a career change” - blog post by Lindsay Holmwood

Trevor:

Sandisk SSD
Android L: Inbox
Rosetta Probe

Managing Systems in the Cloud

15 oktober 2014 | 49 min

Devopsdays Chicago

9 oktober 2014 | 39 min

Conference Love

24 september 2014 | 57 min

Something About Security With Ben Hughes

9 september 2014 | 56 min

Dev to Ops

28 augusti 2014 | 52 min

The Sysadmin Show

20 augusti 2014 | 65 min

Help! I Need Somebody

4 augusti 2014 | 51 min

DevOpsDays Minneapolis

18 juli 2014 | 48 min

Continuous Delivery

15 juli 2014 | 51 min

How to Eff Up Devops

7 juli 2014 | 59 min

Software Deployment

23 juni 2014 | 46 min

Making the DevOps Transition

11 juni 2014 | 45 min

DevOps at Etsy: Not a Unicorn, Just a Sparkly Horse

21 maj 2014 | 58 min

Scaling the Application Mountains

8 maj 2014 | 51 min

Fast and Furious: Configuration Drift

29 mars 2014 | 65 min

Managing Your Mental Stack

11 mars 2014 | 62 min

All Together Now

23 februari 2014 | 60 min

DevOps Mythbusters

10 februari 2014 | 66 min

Continuous Integration – CI Told You So!

29 januari 2014 | 58 min

Agile and Devops

3 januari 2014 | 68 min

The Dev Show

2 januari 2014 | 56 min

Retro

Trevor completed another revolution around the sun. Got Chromecasts for some family members. He was really impressed with how easily it interfaced with their other devices. Matt celebrated the holidays and got a lot of Doctor Who stuff, including a new Sonic Screwdriver that his 4 year old sons are obsessed with. He also got the first part of his new tattoo done. And lots of PowerShell, cloud, and DevOps.

Outline/Questions

David discovers that his particular model of servers in his home lab are suspect to the NSA “DEITYBOUNCE” exploit. Discuss why is it necessary to learn/know the “low level” things - does a developer really need to know how to write a bubble sort? Do devs need to understand RAID? Dan is reading a book called Code which is relevant to this topic. Dan believes technology people should always be on a “quest” to want to be better, to need to be better. Matt: “There’s more to developing an application than writing the procedural code.” Matt quotes John Vincent’s blog post AGAIN about what DevOps means. Lots of discussion about special snowflakes without giving proper attributions to Sascha Bates. DevOps is not a role, it’s a culture. Dan wonders about the Tech Ops equivalent of CodeAcademy. Matt vaguely remembers something like this. He might be thinking about Ops School. Matt offers a challenge to any developers in the audience - in your next standup, grab a task from the board that is not a traditionally “dev” task. Let us know how this works when you try it!

DevOps Resolutions for 2014

David - To think about DevOps when Matt is not in the room. Just to keep fresh. Matt - To write some code for something that does some sh*t. And have David deploy it. Dan - To not be a #newb in Ops Trevor - To be more present in his mind when it comes to doing Ops-type things.

Check-Outs

Matt

6 Year Old Drumming Prodigy Shreds Welcome To The Jungle By Guns N Roses Chocolatey Boxstarter

Trevor

The Captain Kirk Problem: How Doctor Who Betrayed Matt Smith A Feast of Ice and Fire: The Official Game of Thrones Companion Cookbook

David

The book Clean Code by Robert C. Martin

Dan

Free/cheap Kindle e-books for dev and ops books (sort the Kindle store from low to high)

Does Testing Keep You From Making a Huge Mistake?

16 december 2013 | 67 min

What Is DevOps?

6 december 2013 | 37 min

Arrested DevOps

Arrested DevOps is the podcast that helps you achieve understanding, develop good practices, and operate your team and organization for maximum DevOps awesomeness.

Om podden

Avsnitt

Open Communities With Andrew Zigler

The Importance of Open Source in Communities

The Critical Role of Leadership in Open Source Communities

Engineering Blogs: The Balance of Output

Conclusion

Links

Machine Learning Ops With Chelsea Troy

It's Been Ten Years of ADO, Charlie Brown

So You’re in Charge Now… With Ben Greenberg

DevOps Isn’t a Department With Jeremy Duvall

Runtime Analysis With Brian Kelly

Complexity With Michael Stahnke

The Database Calls Are Coming From Inside the House With Grant Fritchey

Platform Engineering Goes to Flavortown With Matt Kurtiz

What's Up With Open Terraform?

The New DevOps With Adam Jacob

Links to Resources Mentioned

Purposeful Personal Brand With Cassandra Faris

The Importance of Your Personal Brand

Building Your Brand Story

Show, Don’t Tell

Embrace Vulnerability

Pay It Forward

Links to Resources Mentioned

Everything's a Product With Sarah Morgan

Cloud Native Security With Michael Isbitski

DevOps With Better Marketing With Pete Cheslock

It's Rough Out There With Sidney Miller

Data! Data! Data! With Francesco Tisiot

Platform Engineering With Daniel Bryant

Into the VOID Report With Casey Rosenthal and Courtney Nash

Continuous Feedback With Roni Dover

Engineers Are People With Dagna Bieda

We Have More Work to Do With Tim Banks

Why Are We Still Talking About DevOps and Security

Deserted Island DevOps 2021

2021 Year-End Wrap-Up

Favorite Episodes

Most Popular Episodes

Pet photos!

Moxie

Nimoy and Ripley

Technical Agile Coaching With Emily Bache

The Reality of DevSecOps With Steve Giguere

Brigade With Kent Rancourt

Words Are Hard With Emily Freeman

Multicluster Service Mesh With Phillip Gibson and Annie Wang

Foundational Practices With Johan Abildskov

Drawing DevOps With Ashton Rodenhiser

The Edge of Now With Cat Swetel

Seasons of Community With Katy Farmer

All Things Docker

Learning to Learn and Learning to Teach With Shelby Spees

The Six Plots of Tech Twitter

Doing Releases Right With Scott Hain

2020 Year-End Wrap-Up

Favorite Episodes

Matty

Trevor

Jessica

Bridget

Jeff

What happened in 2020?

Trevor

Jessica

Bridget

Jeff

Matty

90s Sci-Fi Roundup

Joe’s Top Babylon 5 Episodes

Joe’s Worst Babylon 5 Episodes

Breaking Down Gates With Tim Banks

Devopsdays Chicago 2020

Tea and Anarchy With Alice Goldfuss and Ian Coldwater

State of Open Source Security With Alyssa Miller

Incident Retrospectives With Amy Tobey, Alex Hidalgo, and Rein Heinrichs