Previously, Nick discussed his adventures through our Learning is Good program in his Strange Loop Scoop. This year, a handful of us went to Velocity, a conference centered around systems engineering and building resilient systems at scale. Here are some of my favorite talks from the event earlier this summer.

10,000 messages a minute: Lessons learned from building engineering teams under pressure by Julia Grace, the director of infrastructure engineering at Slack, highlighted the importance of clarity and psychological safety within a team. In her talk, she described having high levels of clarity and psychological safety as the ingredients for a high performing team.

Last year, the New York Times Magazine published an article on psychological safety, defined as “a sense of confidence that the team will not embarass, reject, or punish someone for speaking up.” Grace expanded on this aspect, providing some concrete ways to build that safe space within your team:

  • Have icebreaker events so you get to know your coworkers better
  • Have an optional biweekly team lunch
  • Everyone in the team shares a goal where you need others to help you achieve, your teammates keep you accountable
  • Team emoji/mascot

Grace also emphasized the need for defining a clear mission that everyone in the team can be aligned on. Defining projects in terms of this mission will then help overall team dynamics and speed. We already do many of these things here at Yext, and I’ve always felt the psychological safety within my team. Nevertheless, the talk was a great reminder with some concrete action items that I will try to implement to build more psychological safety within my team.

The Road To Chaos by Nora Jones was an entertaining evangelistic presentation on how to get your company into Chaos Engineering. Jones is a senior software engineer at Netflix and recently came out with a book on Chaos Engineering. Her talk was well delivered and provided some simple steps to grow the culture at your company.

  • Introduce the concept of chaos engineering: Introduce chaos engineering to your team. Allow people to opt-out and communicate your intentions.
  • Make sure you monitor all services you plan to chaos test: Chaos is meaningless if you don’t already have good monitoring on your services. Having monitoring that reveals key aspects of your service will allow you to understand the weak points of your system.
  • Try to cause cascading failures/start adding failure injection capabilities: Try to create cascading failures and use these to build out resiliency in your services. Iterate!
  • Automate chaos: Make your chaos automatic and alert off this.
  • Targeted chaos: Focus your chaos efforts on specific software.

Chaos engineering is definitely on our roadmap and I’m excited to break some systems!