This repo contains some of my favorite insights from DevOps Enterprise Summit 2016.
- Empathy with execs: competing priorities, delivering results, engaging workplace, too much work, risks, high rates of change, talent
- Empathy with middle managers: talent, features vs. technical debt, collaboration, shared outcomes, budget cuts, making the implicit explicit, empowering teams, in-sourcing
- Encourage a growth mindset
- Make work visible (i.e. Kanban)
- Seed teams with change agents
- Aligning into cross functional product teams w/people from various specialties
- What value is the silo protecting? How will that value be protected?
- “Cylinders of excellence”
- Culture of safety to experiment
- Room to experiment
- Build bridges between silos
- Non functional requirements in product backlogs
- Make your work visible to execs (backlog)
- Air cover, understand long term impact to people, turn over
- What are you willing to de-prioritize
- Survey to see how people are feeling
- Warning sign: if your backlog is growing, but your headcount is not (Aimee Bechtle)
- How do you earn trust?
- How transparent can you be?
- People want to look good
- Trust is a transitive property
- Company policies to build trust: are we doing what we said we’d do?
- You can't fire your way to reliable (David Blank-Edelman)
- Pioneer team, set them up for success, green field, let them work out bumps in the road and teach others
- Collect data and measure metrics, KPIs
- Find out what they care about
- Situational awareness
- Show progress
- Maturity model is a good way to quantify it
- API contracts
High performers have:
- 200x more frequent deployments
- 2,555x shorter change deployment lead times
- 3x lower change failure rate
- 24x lower MTTR
- Great things never come from comfort zones
- Try and fail, but never fail to try
- Whatever you do, make it great
- I'm not here to be average, I'm here to be awesome
- It's a misconception that changes lead to incidents
- The amount of time required to deliver changes goes up exponentially as the % load on the team increases
- Anyone can add value to any team from anywhere
- Automation is the Achilles Heel of DevOps
Hubot design considerations and best practices:
- Lightweight: small, on cloud, in Docker
- Automated build/deploy, tied to SCM
- Keep it simple
- Aviod creating a single point of failure
- Keep it chat tool agnostic
- Reuse code as much as possible
Security considerations:
- Express Framework enables basic auth for ports
- Implement nginx proxy pass for SSL endpoint
- Secure Hubot.env and store all ENV variables there
- App accounts are safer than personal accounts for integrations
- Implement SSO on chat platforms that support it
- Use the enterprise version of the chat tool
Lessons Learned Implementing a One Engineering System at Microsoft, Ed Blankenship and Sam Guckenheimer
- If you have more than about 4 open bugs per engineer, take a break from new features and fix them.
- Self forming teams. Once per month. Team leads pitch their projects. Developers put their 1st, 2nd, 3rd choice teams on stickies. Team leads sort people into teams. 95% success with getting people into their preferred teams. Just a few limits, like co-location.
- "GitHub is just better at version control than Subversion"
- It's also faster and better at merging
- "Give teams a half empty tool belt and they'll make their own tools. Then each team becomes a snowflake."
- "Instead of a plea to standardize, lure them with ease of use and low cost. (Make the "right" thing pleasant and people will do it.)
- They separated the presentation tier from the business tier and created separate paths to production for each.
- Being willing to run a test and potentially "miss" was a big psychological barrier they had to work through
- It's much more powerful when the practitioners are teaching than when the transformation guy is teaching
- Our backlog was growing, but our headcount was not
How to stop burnout and help people thrive
People burnout from:
- Work overload
- Lack of control
- Insufficient rewards
- Breakdown of community
- Absence of fairness
- Value conflicts
People thrive with:
- Leveled work
- Empowered
- Sufficient rewards
- Supportive community
- Fairness and transparency
- Aligned values
- Make tiny decisions. It's ok to not plan too far ahead. It's easier to make small decisions, and easier to change them when you're wrong.
- Fight hero culture. Enforce healthy balance.
- Recovery plans: If you have a plan that you haven't practiced, you don't have a plan.
- Test your run books monthly (rotating basis). Get new hires to practice them before they go on call. Be prescriptive so people don't have to make too many decisions at 2 AM.
- During an outage, offload communications to someone, so technical poeple can focus.
- Learn from your failures and successes, and share the results publicly.
- Apologize. And mean it.
- Demonstrate a thorough understanding of the problem.
- Explain what you're doing to reduce the likelihood of similar problems. Don't over promise.
- Collaborate by default. Visibility is the ultimate compensating control.
- "My plea for empathy as a core value during my keynote a #DOES16 yesterday wasn't just about DevOps. Even more important to me today." (in the context of the 2016 election results)
- URL: https://puppet.com/resources/whitepaper/2016-state-of-devops-report
- Continuous Delivery is key to being a high performance organization.
- "Work in Progress limits" only work in combination with other Lean practices, not in isolation.
- "Mean time between Vice Presidents should be longer than your project duration."
- "Conservation of management attention span": remove things that eat into managers' time and add little value.
- Operational heroics and firefighting lead to inhumane operations and burnout.
- DevOps adoption was successful; too successful.
- The Enterprise Strikes Back: overwhelming Scale, Speed and Stability issues due to explosive growth.
- "If you name servers and give them artisan care, they start to develop personalities."
- "Leadership requires People, Vision, and Courage."
- Leadership Challenges: Politics, New Leaders (disruption), Blame Bias.
- We are at our best when we are helping each other, serving each other, making a positive difference.
- Help improve the human condition, using technology.
- The best SLA is the ‘golden rule’ - provide a service level to others that you would want provided to you.
Domenica DeGrandis Time Theft: How Hidden and Unplanned Work Commit the Perfect Crime
- Video link: https://t.co/zfp2IdLghR