How They Test
A curated collection of publicly available resources on how software companies around the world test their software systems and build their quality culture.
Foreword
One of the outcomes of the various Taqelah meetups was that many software companies got to share their testing and quality culture with the community. It was absolutely fantastic to see the amazing stuff companies are doing to test their software, and ensure quality of their products and platforms.
Apart from this, many companies regularly come forward and share their best practices, tools, techniques and culture of software testing on various public platforms like conferences, blogs & meetups. The resources are there but dispersed, lost into the internet.
There is no single knowledge repository that gives a direct look at these best practices, tools, techniques and culture of software testing these companies adopt. This repository intends to do that.
Happy Learning!
Please note, that all the resources mentioned here are publicly available resources.
Kind of topics here
Testing / quality of software goes beyond traditional activities of pre-release functional testing and test automation. Hence the kind of topics you can expect to read about here are:
- Functional testing
- Non-functional testing
- Test automation
- Testing in CI/CD
- Release management
- Process/culture
- Testing in production (monitoring/observability, chaos engineering, site reliability engineering etc.)
- Customer/User service or support
Companies & how they test their software
Airbnb
Blogs
- Testing at Airbnb
- Better Android Testing Part 1
- Better Android Testing Part 2
- Better Android Testing Part 3
- Better Android Testing Part 4
- Better Android Testing Part 5
- Better Android Testing Part 6
- Better Android Testing Part 7
- Building an effective test pipeline in a service oriented architecture
- Tracking the money scaling financial reporting
Amazon
Blogs & Articles
- How does Amazon get by with so few hours spent by its QA teams relative to Microsoft?
- How Amazon handles a new software deployment every second
- How is software developed at Amazon
- Amazon's approach to automated testing
- Automated testing for Alexa
- Best practices for testing your Alexa skills
Videos
- Testing and Troubleshooting with AWS Device Farm
- Proactive security testing at AWS
- Advanced Continuous Delivery Best Practices at Amazon
- UI, Load and Performance testing at Amazon
- Best Practices for Benchmarking and Performance Analysis in the Cloud
- Large Scale Load Testing Amazon.com's Traffic on AWS
- Chaos Engineering on AWS
- Continuous Integration best practices for software development
Apple
Blogs & Articles
Videos
ASOS
Blogs
- Automated security testing using language you already know
- Behaviour driven automation testing in .NET Core using BDTest
- My Testing Safari
- Begin with the end in mind: one team’s journey towards Continuous Integration
- Gaining confidence in automated checks
- Witness the (Android) fitness
- How to test your React · Redux application
- How to write acceptance tests for a Facebook Messenger bot
- An introduction to Pact testing in .Net Core
- Finding balance as a Testing Specialist
- Continuous Testing in Customer Experience @ ASOS
- Responsive website testing with Sizzy
- Testing authorisation scenarios in ASP.NET Core Web APIs
Videos
Atlassian
Blogs & Articles
- Quality assistance: how Atlassian does QA
- Moving from quality assurance to quality assistance
- Developing quality assistance skills
- Quality Health Monitor: Gauging your team’s quality maturity
Videos
- Meet the Atlassian QE team
- The Future of QA at Atlassian - Atlassian Summit 2016
- Quality at Speed, How JIRA Does QA
- Verifying Microservice Integrations with Contract Testing
- Agile Testing: It's about time - Atlassian Summit 2011
- Finding bugs before writing code - Sigge Birgisson
- Code Reviews vs. Pull Requests - Atlassian Summit 2016
- Mock Servers - Fake all the things! - Peggy Kuo
- Quality at Speed
- Improving Software Quality with Effective Feedback - Atlassian Summit 2012
BBC
Blogs & Articles
- 10x your collaboration on writing tests - Part 1: Understanding
- 10 guidelines on readability and consistency when writing Integration Tests
- Maintaining Mobile: Update on Mobile Compatibility Programme
- Improving test automation with PUMA
- Test Automation with PhantomJS, Grunt and Friends
- Behaviour Driven Development: Tips for writing better feature files
- Testing BBC Connected Red Button
- Automating testing for BBC iPlayer mobile part one: 3 Amigos
- Automating testing for BBC iPlayer mobile part two: automation
- Automating BBC iPlayer mobile testing part three: legacy vs new features
- Testing BBC iPlayer Mobile App
- New BBC iPlayer: internal testing of user journeys
- Testing mobiles: Mobile Compatibility Program
- Testing BBC iPlayer: Krispy Gherkin feature file aggregator
- Testing BBC iPlayer: reading XML, JSON response into Java
- Testing BBC iPlayer Release For Android Downloads
- Testing for BBC Online: The creation of the POD Test Group
Videos
Box
Videos
Canva
Blogs & Articles
Videos
CapitalOne
Blogs and Articles
- Improvement and Parallelization of Android Automation Testing
- 9 Things I Wish I Knew About Automated Software Testing Before I Started My Career
- Improve Java Code Coverage and Quality with Unit Tests and JaCoCo
- Learning To Love Automated Testing
- Robot Pattern Testing for XCUITest
- Unit Testing Behavior of React Components with Test-Driven Development
- No Testing Strategy, No DevOps
- Spec to Gherkin to Code
- Blackbox API Testing With Kotlin for RESTful Microservices
- Performance Testing of Event-Driven Microservices
Videos
Carousell
Blogs & Articles
- Prabhagharan DK : Building our inhouse virtual device lab "caroufarm"
- Martin Schneider : Automated end-to-end tests and how they fit into our testing culture
- Carousell scales app automation with Browserstack
Videos
Dollar Shave Club
Blogs
Dropbox
Blogs & Articles
- Offensive testing to make Dropbox (and the world) a safer place
- Accelerating Iteration Velocity on Dropbox’s Desktop Client, Part 1
- Accelerating Iteration Velocity on Dropbox’s Desktop Client, Part 2
- Two years at Dropbox
- Creating a culter of accessibility
- Finding Kafka’s throughput limit in Dropbox infrastructure
- Athena: Our automated build health management system
- The Programmer Mindset: Main Debug Loop
- Testing sync at Dropbox
Videos
eBay
Blogs & Articles
- Tiered Test Automation
- Becoming a World-Class Tester
- Early Detection of Frontend Single Points of Failure
- eBay Motors: Screenshot Testing with Flutter
- Discovering Continuous Automation With Request Mirroring
- Failbot—Improving Visibility on End-to-end Tests
- Your Own Spring Test Context
- GUI Testing Powered by Deep Learning
- Integration Testing with React and Enzyme
- Introducing Regressr - An Open Source Command Line Tool to Regression Test HTTP Services
- eBay Releases Dynamic Application Security Testing Proxy as Open Source
- Scalable and Nimble Continuous Integration for Hadoop Projects
- Functional iOS Testing in Swift
- Next-generation BDD: From Automated Web Tests to Automated Web Acceptance Testing
- Validating Hadoop Platform Releases
- Finding Usability Bugs with Automated Tests
- 100% test coverage is not enough
- Ensuring Software Quality at eBay Denmark
- Software Quality: Elevating the Game
Videos
- Lightning talk at Selenium Conf about their test automation framework Breeze
- Applied Testing Heuristics in the Context of eBay
- Test automation for Android testing
- Implementing Test Automation in Agile Projects with Open Source Tools
- Test Automation - 10 (sometimes painful) Lessons Learned
- Parallizing UI tests on iOS
- Bingsheng Ru — How to build Automation Test as a Service for large-scale e-commerce website
- How we build & test software at eBay DE
Etsy
Handbook
Blogs & Articles
- Culture of Quality: Measuring Code Coverage at Etsy
- Quality Matters: The Benefits of QA-Focused Retros
- Divide and Concur
- Etsy’s Journey to Continuous Integration for Mobile Apps
- Etsy’s Device Lab
- What’s the value of transparency in testing and deployment?, Increment Magazine
- Mutation Testing: A Tale of Two Suites
Expedia
Blogs & Articles
- The Cost of 100% Reliability
- Getting started with a Dockerized Selenium Grid in your On-Prem Data Centers
- DA Kube — Selenium Grid using Kubernetes, Docker, Helm and Traefik
- Distributed Automation: How to run 1000 UI Automation Tests in 5mins
- Integration Testing in React
- How to Earn a Black Belt in GraphQL Testing
- How to: Emulate Amazon Simple Queue Service (SQS) for Testing Purposes
- Fixing Flaky Time Based Unit Tests
- Conversational Integration Tests for your Alexa Skills (Node/JS)
- Better Test Code Coverage Through Data
- Do you want to speed up your integration tests by 10x?
- Testing Elasticsearch Applications
Videos
Blogs & Articles
- Open-sourcing Python Test Runner for multiple tests in parallel
- Predictive test selection: A more efficient way to ensure reliability of code changes
- MobileLab: Highly accurate testing to prevent mobile performance regressions
- Sapienz: Intelligent automated software testing at scale
- Preventing performance regressions with Health Compass and Incident Tracker
- Finding and fixing software bugs automatically with SapFix and Sapienz
- Open-sourcing Flipper: a new extensible debugging tool
- BrowserLab: Automated regression detection for the web
- Getafix: How Facebook tools learn to fix bugs automatically
- Stetho: A new debugging platform for Android
- Open-sourcing Facebook Infer: Identify bugs before you ship
- Building and testing at Facebook
- Watir to WebDriver: Unit Test Frameworks
- Rapid release at massive scale
- Finding inter-procedural bugs at scale with Infer static analyzer
- Zoncolan: How Facebook uses static analysis to detect and prevent security issues
Videos
- Systems @Scale 2019 - Continuous Deployment at Facebook Scale
- Mobile End to End Testing at Scale - SeleniumConf 2015
- Turning iOS End to End Testing up to 11 - Lawrence Lomax
- Readable. Stable. Maintainable. E2E Testing @ Facebook - Archit Pal Singh Sachdeva
- Automated Fault-Finding and Fixing at Facebook
- Damien Sereni, Testing and moving fast making Selenium work for Facebook
- Taqelah - Facebook Meetup - Raj Kamal - Shreya Bhat
- GTAC 2013: Simon Stewart - How Facebook Tests Facebook on Android
- GTAC 2014: Never Send a Human to do a Machine’s Job: How Facebook uses bots to manage tests
- Facebook Infer: Advanced Static Analysis at Speed and Scale - Facebook - DevTools@Scale2017
- Building Mobile Apps with Open Source Tools - Simon Stewart - Mobile @ Scale
GitHub
Blogs & Articles
- How GitHub uses automated testing to empower developers to write less
- MySQL infrastructure testing automation at GitHub
- octocatalog-diff: GitHub’s Puppet development and testing tool
- Browser Monitoring for GitHub.com
Videos
GitLab
Handbook
Blogs and Articles
- An inside look at software testing at GitLab
- Introducing Accessibility Testing in GitLab
- How our QA team leverages GitLab’s performance testing tool (and you can too)
- How GitLab switched to Headless Chrome for testing
- How recent acquisitions introduce fuzz testing to GitLab
- How the Search Team at GitLab Implemented a Risk Map to Direct Automated Testing Efforts
GoDaddy
Blogs & Articles
Goibibo
Blogs & Articles
GoJek
Blogs
- Why you should automate tests early
- Screenshot Testing our Design System on Android
- Advice from a Test Journeyman
- How I Gave My First Conference Talk
- 7 Myths About Software Testing
- The Untold Story of Golang Testing
- The first principles of ‘Testing’
- How to Run Grouped TestNG Tests Using Gradle
- Golang Integration Testing Made Easy - How we at Gojek implement integration testing in our development processes.
Books
- James Whittaker, Jason Arbon and Jeff Carolllo provide insights into testing at Google in their (slighlty outdated) 2012 book How Google Tests Software. There is a seven part summary on the Google Testing Blog as well:
- How Google Tests Software - Part One
- How Google Tests Software - Part Two
- How Google Tests Software - Part Three
- How Google Tests Software - Part Four
- How Google Tests Software - Part Five
- How Google Tests Software - Part Six
- How Google Tests Software - Part Seven
Blogs
Videos
- Meet Test Engineers at Google
- GTAC 2016: Evolution of Business and Engineering Productivity, Slides
- GTAC 2016: The Quirkier Side of Testings, Slides
- GTAC 2016: How Flaky Tests in Continuous Integration: Current Practice at Google and Future Directions, Slides
- GTAC 2016: OpenHTF - The Open-Source Hardware Testing Framework, Slides
- GTAC 2016: Integration Testing with Multiple Mobile Devices and Services, Slides
- GTAC 2016: Finding bugs in C++ libraries using LibFuzzer, Slides
- GTAC 2015: Juggling Chainsaws for Fun and Profit: Lessons Learned from Mobile Cross-Platform Integration Testing, Slides
- GTAC 2015: How to Component Test Soup Dumplings, Slides
- GTAC 2015: Chromecast Test Automation, Slides
- GTAC 2015: Automated Accessibility Testing for Android Applications, Slides
- GTAC 2015: Statistical Data Sampling, Slides
- GTAC 2015: Nest Automation Infrastructure, Slides
- GTAC 2015: Fake Backends with RpcReplay, Slides
- GTAC 2014: Chrome OS Test Automation Lab, Slides
- GTAC 2014: Make Chrome the best mobile browser, Slides
- GTAC 2014: Test coverage at Google, Slides
- GTAC 2014: The Importance of Automated Testing on Real and Virtual Mobile Devices, Slides
- GTAC 2014: Free Tests Are Better Than Free Bananas: Using Data Mining and Machine Learning To Automate Real-Time Production Monitoring, Slides
- GTAC 2014: Google BigQuery Analytics, Slides
- GTAC 2014: The Testing User Experience, Slides
- GTAC 2014: Going Green: Cleaning up the Toxic Mobile Environment, Slides
- GTAC 2013: Opening Keynote - Evolution from Quality Assurance to Test Engineering, Slides
- GTAC 2013: Webdriver for Chrome, Slides
- GTAC 2013: Karma - Test Runner for JavaScript
- GTAC 2013: Automated Video Quality Measurements, Slides
- GTAC 2013: Testable JavaScript - Architecting Your Application for Testability, Slides
- GTAC 2013: Breaking the Matrix - Android Testing at Scale, Slides
- GTAC 2013: Android UI Automation, Slides
- GTAC 2013: Building Scalable Mobile Test Infrastructure for Google+ Mobile, Slides
- GTAC 2013: Espresso: Fresh Start to Android UI Testing, Slides
- GTAC 2013: Web Performance Testing with WebDriver, Slides
- GTAC 2013: Continuous Maps Data Testing, Slides
- GTAC 2013: Finding Culprits Automatically in Failing Builds - i.e. Who Broke the Build?, Slides
Helpshift
Videos
Blogs & Articles
Instacart
Blogs & Articles
Videos
Intel
Videos
Blogs & Articles
- LinkedIn’s approach to automated accessibility (A11y) testing
- Quality Control - LinkedIn's Testing Methodology
- iOS: Test Pyramid
- Writing Maintainable Integration Tests
- "Testscaling" with Test Engineering
- UI Automation: Keep it Functional – and Stable!
- Test Stability - How We Make UI Tests Stable
- Getting Code to Production With Less Friction and High Quality
- Eliminating toil with fully automated load testing
- The testing lifecycle at LinkedIn
- Testing Frontier, LinkedIn's Web Framework
Videos
Mercari
Blogs & Articles
Microsoft
Books
- How We Test Software at Microsoft by and Alan C. Page, Ken Johnston and BJ Rollison is a good starting point.
Articles
Miro
Blogs & Articles
Monese
Blogs & Articles
Monzo
Blogs & Articles
Mozilla
Handbook
Blogs & Articles
Videos
NASA
Articles
- Intelligent test automation gives orion spacecraft boos
- How did NASA make reliable software if they didn't invent unit tests?
- Software Development & Test
- Automated testing of nasa software
- NASA releases updated open source software bundle including testing tools
- Jon McBride Software Testing and Research (JSTAR)
Netflix
Blogs & Articles
- Edgar: Solving Mysteries Faster with Observability
- Automation as a Service — Introducing Scriptflask
- Automated testing on devices
- Product Integration Testing at the Speed of Netflix
- Automated Failure Testing
- Announcing Electric Eye
- From Chaos to Control — Testing the resiliency of Netflix’s Content Discovery Platform
- NTS: Real-time Streaming for Test Automation
- FIT: Failure Injection Testing
- Testing Netflix on Android
- Lerner — using RL agents for test case scheduling
- Announcing Sleepy Puppy — Cross-Site Scripting Payload Management for Web Application Security Testing
Videos
- GTAC 2015: Enabling Streaming Experiments at Netflix, Slides
- GTAC 2014: I Don't Test Often ... But When I Do, I Test in Production, Slides
- GTAC 2013: When Bad Things Happen to Good Applications..., Slides
- Design Testing at Netflix
- Testing In Production, The Netflix Way
- Test Automation at Scale
- Evolution of Payments Testing at Netflix
- Test Selection: Heuristics and Machine Learning
- The Automation before pressing Play
- Applying Failure Testing Research @Netflix
- Getting Netflix on a Variety of Devices
- Electric Eye
- Measuring Perceptual Video Quality at Scale
PayPal
Shazam
Blogs & Articles
Videos
Shopify
Blogs & Articles
- Webhook testing made easy
- The Unreasonable Effectiveness of Test Retries: An Android Monorepo Case Study
- iOS Application Testing Strategies at Shopify
- Building and Testing Resilient Ruby on Rails Applications
- Four Steps to Creating Effective Game Day Tests
- Mobile Tophatting at Shopify
- Great Code Reviews—The Superpower Your Team Needs
- Mobile Release Engineering at Scale with Shipit Mobile
- Software Release Culture at Shopify
Videos
Slack
Blogs & Articles
- Everything You Need to Know About How Slack Approaches Accessibility Testing
- Android UI Automation: Part 1, Building Trust
- Android UI Automation: Part 2, Making It Easy
- Surf’s Up! Preparing for Huge Waves of Traffic via Load Testing
- How About Code Reviews?
- On Empathy & Pull Requests
- Disasterpiece Theater: Slack’s process for approachable Chaos Engineering
- Scaling End-to-End User Interface Tests
Videos
Soundcloud
Blogs & Articles
- Release Quality and Mobile Trains
- Integration Testing for Memory Leaks
- Hands-Off Deployment with Canary
- Running Android UI Test Suites on Firebase Test Lab
- Automatic Stubbing of Network Requests to Deflakify Automation Testing
- Alerting on SLOs like Pros
- Periskop: Exception Monitoring Service
Videos
Spotify
Blogs & Articles
- Test Flakiness – Methods for identifying and dealing with flaky tests
- Testing of Microservices
- Generating test cases so you don’t have to
- Test Strategies and Spotify – Kristian Karl
- How to Build a QA Strategy Like Spotify
- How Spotify does test automation
- Model Based Testing at Spotify
- How We Improved Developer Productivity for Our DevOps Teams
Videos
- Spotify Model Based Testing - Kristian Karl
- Experiences Of Test Automation At Spotify - Kristian Karl
- Scaling Android Application Testing at Spotify - Sean Kenny
- How Spotify Does Testing for Mobile - Sean Kenny
- Test flakiness - Methods for dealing with flaky tests - Jason Palmer
- Applying Testing Mindset to Software Development - Andrew Dzynia. | Slides
Squarespace
Blogs & Articles
- Engineering a Better Working Group
- Turbocharging Our UI Tests
- UI Testing at Squarespace: Part I
- UI Testing at Squarespace: Part II
- Six Attributes of Beautiful Systems
- Under the Hood: Ensuring Site Reliability
- Continuous Localization at Squarespace
- Creating a Code Review Culture, Part 1: Organizations and Authors
- Creating a Code Review Culture, Part 2: Code Reviewers
Stripe
Blogs & Articles
The Guardian
Blogs & Articles
- Software quality in a world of microservices and continuous deployment
- Testing in Production: rethinking the conventional deployment pipeline
- Data-Driven Quality
- Testing in the right places
- The software testing identity crisis
- Device wall of awesome: how Lego helped our digital development
- QA testing for the Guardian's Content API
- Real-time QA: staying confident with code
- Testing, testing: the Guardian attends a QA community meet-up
Trivago
Blogs
Vidoes
Uber
Videos
- GTAC 2015: The Uber Challenge of Cross-Application/Cross-Device Testing, Slides
- Using the Octopus Framework - Jeff Hu
- Inside Uber's secret self-driving car testing facility
- Ludwig: A Toolbox for Training and Testing Deep Learning Models without Writing Code
- Chaos Engineering @ Uber -- Dianne Laguerta
- Observability infra @ Uber
Blogs & Articles
- Octopus to the Rescue: the Fascinating World of Inter-App Communications at Uber Engineering
- Introducing Hypothesis GU Funcs, an Open Source Python Package for Unit Testing
- Best Practices for Unit Testing in React Version 16
- Monitoring Data Quality at Scale with Statistical Modeling
- Introducing Piranha: An Open Source Tool to Automatically Delete Stale Code
Veracode
Handbook
Videos
WalmartLabs
Blogs & Articles
Videos
Wix
Blogs & Articles
- Software Developers, Communicate Your Intentions (TLDR: do it using tests)
- Meet Raven TestKit: Wix Engineering's Open Source Tool to Test Sentry Reports
- How to Test Your Redux App
- Testing Asynchronous Code
- How to integrate 3rd Party APIs without Fear
- Keep Your Tests Clean and Isolated with a Context
- Asynchronous Sad Path Testing in React Using Jest and The Driver Pattern
- Keep your tests clean, by refactoring
- Functional Testing with Tagless-Final
- Overestimated value of test documentation
- Testing Metrics: Are You Really Sure about the Quality of Your Product?
- Integration Tests in JavaScript That Don’t Suck
- Building and Open-Sourcing Wix’s A/B Testing System
Videos
- The Price of TDD is Eternal Vigilance
- Clean Tests
- Testing with Protractor
- To Test or Not To Test?
- Screenshots in Automated Testing: When? How? Why?
- Driving to UI Test Heaven
- Detox: tackling the flakiness of mobile automation
- Testing For Performance
- Detox: Graybox End to End Tests and Automation Library for Mobile Apps
- TDD for the Curious
- E2E testing with Javascript
- Intro to TDD
- Meaningful Code Reviews
- E2E testing with Javascript
- TDD: Discover Implementation by Stepping Small
git
Contributors
Contribution
Want to contribute? Just fork and raise a PR!
Credits
- Images in banner used from https://undraw.co/
- Idea inspired from the awesome lists.
- Static site powered by docsify.
- Contributors list generated using contributors-img
- All the authors and the companies they represented in the resources.
Code of Conduct
Please refer to the code of conduct here.
