Awesome Visual Regression Testing

Curated list of awesome visual regression testing resources.

Regression testing is a type of software testing which verifies that software which was previously developed and tested still performs the same way after it was changed or interfaced with other software. The purpose of regression testing is to ensure that changes to the software have not introduced new faults.

Foreword

This is intended to be an incomplete list of resources about visual regression testing. It is not tailored to a specific area or role (Developer/QA/UX-Designer). Note that this is for all areas of regression software testing after the code in question is written. For a awesome list on general software testing see e.g. awesome-testing(link is external).

Finally, I'm sure everyone who reads this list has one thing they want to add. Please read the How to Contribute page and Feel free to add to the list!!. If you think this is helpful Please give a Star ★.

Contents

• General information
• Browser automation
• Tools and frameworks
• Online services
• Blog posts
• Slideshows, talks and videos
• Deprecated
• Miscellaneous
  
  • Contributing
  • Code of Conduct
  • License

General information

• Wikipedia: Regression testing(link is external)
• Survey of screenshot-based CSS testing tools(link is external)

Browser automation

• Selenium(link is external) - Browser automation framework and ecosystem.
• SlimerJS(link is external) - Scriptable browser like PhantomJS, based on Firefox.
• trifleJS(link is external) - Headless automation for Internet Explorer.
• CasperJS(link is external) - Navigation scripting and testing utility for PhantomJS and SlimerJS.
• Webdriver.io(link is external) - Node.js bindings implementation for the W3C WebDriver protocol.
• Navalia(link is external) - Browser Automation based on headless Chrome and GraphQL.
• Chromeless(link is external) - Chrome automation made simple. Runs locally or headless on AWS Lambda.
• Cypress.io(link is external) - An automation framework that runs in-browser.

Tools and frameworks

• OSnap(link is external) - The speedy and easy to use Snapshot Testing tool for your project (1200 snapshots will run in under 3 minutes).
• basset(link is external) - Open source platform for generating and reviewing visual differences. Supports multiple browsers, integrations for github and slack.
• AyeSpy(link is external) - 44 image comparisons in 90 seconds.
• Wraith(link is external) - Easy to use ruby tool with docker support.
• BackstopJS(link is external) - Config-driven automated screenshot test framework.
• Galen(link is external) - Java framework based on Selenium(link is external).
• Creevey(link is external) - Cross-browser visual testing with magic. Feature-rich tool with UI Runner, Tests Hot Reloading, Docker and Storybook integration.
• CSSCritic(link is external) - Lightweight CSS regression testing.
• Spectre(link is external) - Provides image comparison capabilities and an admin interface for managing screenshots.
• Shoov(link is external) - UI regression and functional testing focused on Drupal 7 sites.
• qd_screenshottests(link is external) - CasperJS-based UI regression and functional testing focused on Drupal 8 sites.
• Look-alike(link is external) - Chrome Extension for taking and comparing screenshots.
• Hardy(link is external) - Selenium-driven, cucumber-powered CSS testing.
• TestCafe(link is external) - Automated browser testing for the modern web development stack.
• Needle(link is external) - Needle is a tool for testing visuals with Selenium and nose (Python).
• gatling(link is external) - Integrated visual RSpec matcher which makes real visual testing easy (Ruby).
• grunt-photobox(link is external) - Plugin to prevent your project of broken layout via screenshot photo sessions of your site.
• vrtest(link is external) - JavaScript library for running visual regression tests on your components cross browser via selenium.
• Happo(link is external) - Visual diffing in CI for user interfaces.
• reg-cli(link is external) - Visual regression test tool which output easy-to-read single file html report.
• Nightmare(link is external) - High-level browser automation library based on Electron.
• Puppeteer(link is external) - Headless Google Chrome Node API.
• Playwright(link is external) - Node library to automate Chromium, Firefox and WebKit with a single API.
• reg-suit(link is external) - Visual regression testing suite which compares images, stores snapshots, and notifies the difference to your GitHub repo.
• Chimp(link is external) - Develop acceptance tests & end-to-end tests with realtime feedback.
• Differencify(link is external) - A library for visual regression testing using Puppeteer(link is external).
• ResembleJS(link is external) - Analyse and compare images with Javascript and HTML5.
• Muppeteer(link is external) - Visual regression testing framework for Chrome using Mocha(link is external) and Puppeteer(link is external).
• ember-visual-test(link is external) - Simple visual regression testing for Ember(link is external).
• AET(link is external) - Scalable testing tool providing visual regression testing, accessibility and performance validation, markup analysis and more.
• Wendigo(link is external) - Test-oriented browser automation library based on Puppeteer.
• Loki(link is external) - Visual regression testing for Storybook using Chrome in docker et al.
• Zombie.js(link is external) - Insanely fast, headless full-stack testing using Node.js.
• CodeceptJS(link is external) - Modern Era Acceptance Testing Framework for NodeJS.
• FuncUnit(link is external) - A functional test suite based on jQuery
• Nightwatch(link is external) - Automated testing and continuous integration framework based on Node.js and using the Webdriver protocol.
• Protractor(link is external) - E2E test framework for Angular apps.
• jest-puppeteer-react(link is external) - Visual regression testing with Jest and puppeteer for React components
• jest-image-snapshot(link is external) - Jest matcher that performs image comparisons using pixelmatch(link is external)
• test-crawler(link is external) - Visual regression testing, by crawling a website and providing snapshot comparison reports.
• wdio-visual-regression(link is external) - Visual regression tool for webdriver.io
• Selenide(link is external) - Framework powered by Selenium WebDriver for writing easy-to-read and easy-to-maintain automated tests in Java.
• Karma(link is external) - A test runner by the AngularJS team, that fits all our needs.
• Touca(link is external) - Open source continuous regression testing without the hassle of managing snapshot files.

Online services

• BrowserStack(link is external) - Free for Open Source. Supports Selenium Webdriver(link is external).
• BugBug.io(link is external) - Lightweight test automation tool for web applications. Easy to learn and doesn't require coding. It's free, with unlimited tests. For an additional monthly fee, you also get cloud monitoring and CI/CD integration.
• HeadSpin(link is external) - HeadSpin's Regression testing gives you a powerful comparison tool for analysing degradation across new app builds, OS releases, feature additions, locations, and more.
• LambdaTest(link is external) - Perform Automated and Live Interactive Cross Browser Testing on 2000+ Real Browsers and Operating Systems Online.
• screener.io(link is external) - For React, looks open source.
• applitools(link is external) - Cloud base visual tests.
• percy.io(link is external) - Continuous visual reviews for web apps.
• screenster.io(link is external) - Cloud based automation testing platform for web and mobile UI.
• browserling(link is external) - LIVE interactive cross-browser testing.
• Browser Shots(link is external) - Screenshots only.
• Ghost Inspector(link is external) - See introduction video(link is external).
• CrossBrowserTesting(link is external) - Manual & exploratory testing on 1500+ real browsers and mobile devices.
• Argos-CI(link is external) - Automate visual regression testing.
• Diffy(link is external) - Cloud based visual regression tool. Uses puppeteer and proprietary comparison algorithm (detect layout shifts). Great scalability with AWS Lambda.
• Chromatic(link is external) - Visual testing and UI review for component libraries. Cloud-based. Video(link is external)
• VisWiz.io(link is external) - Flexible visual regression testing service.
• Happo(link is external) - Cloud-based screenshot testing service with support for multiple browsers.
• Visual Knight(link is external) - Cloud-based visual testing platform with realtime results for testing tools.
• Axcept(link is external) - Testing for the whole team. Up to 100 tests in parallel. Endpoint Mocking. Code Coverage.
• Fluxguard(link is external) - Screenshot pixel and DOM change comparisons and regressions.
• Vidiff(link is external) - Cloud-based visual regression testing across stages.
• Reflect(link is external) - Visual regression testing and test automation tool.
• Visual Regression Tracker(link is external) - Open Source selfhosted service for visual regression testing
• Micoo(link is external) - Open source service for all UI application visual regression solution
• TestingBot(link is external) - Provides +3600 browsers to run automated visual tests. Free for Open Source.
• Preflight(link is external) - Easiest Visual regression testing and Automated Web Testing tool. (Limited) free use.
• Preflight: Cypress Recorder(link is external) - Create AI-powered Cypress Tests/POM models in your browser and automate Email & Visual testing for Cypress.
• Meticulous.ai(link is external) - Easily create frontend tests without writing code. Use Meticulous to record workflows on your web app. You can then replay those flows on new frontend code, and create a test by diffing two replays.

Blog posts

• Kevin Lamping: The 5 best visual regression testing tools(link is external) - Compares: Wraith, PhantomCSS, Gemini, WebdriverCSS and Spectre.
• Garris Shipon: Visual Regression Testing For Angular Applications(link is external) - Tutorial using BackstopJS.
• Angela Riggs: Visual Regression Testing with BackstopJS(link is external) - Tutorial using BackstopJS.
• Garris Shipon: Automating CSS Regression Testing(link is external) - Tutorial using BackstopJS.
• Phillip Gourley: Making visual regression useful(link is external) - Why you should use BackstopJS.
• Pavels Jelisejevs: Visual Regression Testing with PhantomCSS(link is external) - Introduction to PhantomCSS.
• Chromeless, Chrominator, Chromy, Navalia, Lambdium, GhostJS, AutoGCD(link is external) - Headless Chrome is shaking up traditional approaches to test automation.
• Visual regression testing using Jest, Chromeless and AWS Lambda(link is external) - Tutorial using Chromeless and jest-image-snapshot.
• Make visual regression testing easier(link is external) - Introduction to Differencify(link is external) and how to use it.
• Visual Regression Testing with Puppeteer & Jest(link is external) - Tutorial to setup visual testing with Puppeteer, Jest and VisWiz.io.
• Keeping a React Design System consistent: using visual regression testing to save time and headaches(link is external) - Using percy, and jest puppeteer to visually test a React component library.
• Visual Regression Test with WebdriverIO & WebdriverCSS(link is external) - Tutorial using WebdriverIO and WebdriverCSS with Spec Reporter
• Automated screenshot comparison tests with headless Chrome, Puppeteer and Pixelmatch, in Bitbucket pipeline(link is external)
• Automatic visual diffing with Puppeteer(link is external)
• theheadless.dev(link is external) - Blog with practical guides and runnable examples on Playwright and Puppeteer.
• Visual regression testing for Hugo with Github-CI and BackstopJS(link is external) - How to automate regression testing for Hugo with BackstopJS
• UI Visual Regression Testing with Micoo(link is external) - Introduction about how to do visual regression testing with Micoo service
• Poor man's visual regression testing(link is external) - Improved manual visual regression testing with the PerfectPixel chrome plugin.

Slideshows, talks and videos

• CSS Regression Testing with Wraith(link is external) - Screencast: Basic introduction to wraith, a screenshot comparison tool.
• Visual Regression Testing with Shoov(link is external) - How to setup shoov and get your first test written.
• Visual Regression Testing with PhantomCSS(link is external) - Talk by Jon Bellah on how to use PhantomCSS during wordpress development.
• Visual Regression Testing: Sanity Checks With BackstopJS(link is external) - Screencast with code demo and best practices.
• Screenster Tutorial(link is external) - Tutorial on how to create visual automated tests with Screenster.
• Look-alike - visual regression testing tool(link is external) - Demo what the Look-alike Chrome extension is, how it works and how and why it was build.
• Screencast on CSS critic - a lightweight testing framework for CSS(link is external) - How to write your first CSS test with CSS critic, make it pass, break it, and make it pass again.
• Visual Regression Testing - from a tool to a process(link is external) by Nikhil Verma - How the Mobile Web team in Badoo converted and integrated PhantomCSS into their workflow and connected it to their CI process.
• Cypress in 100 Seconds(link is external) - Introduction video by Fireship.

Deprecated

The following projects are no longer maintained actively but are still worth mentioning because of their user base.

• PhantomJS(link is external) - Scriptable Headless WebKit. No longer maintained since 2 June 2018.
• PhantomCSS(link is external) - Visual/CSS regression testing with PhantomJS or SlimerJS. No longer maintained since 22 Dec 2017.
• PhantomFlow(link is external) - Experimental approach to UI testing, based on Decision Trees.
• DalekJS(link is external) - Automated cross browser testing with JavaScript. No longer maintained since 4 Jun 2017.
• dpxdt(link is external) - End-to-end testing with Python.
• Visual Review(link is external) - A human-friendly tool for testing and reviewing visual regressions.
• Huxley(link is external) - Python framework based on Selenium Webdriver(link is external).
• WebdriverCSS(link is external) - WebdriverCSS sits on top of Webdriver.io(link is external) and hooks into Selenium(link is external).
• Gemini(link is external) - Feature rich framework with support for Selenium(link is external) and CasperJS(link is external). Gemini is deprecated, use hermione instead.
• OcularJS(link is external) - uses PhantomJS(link is external).

Miscellaneous

Contributing

See the Contribution Guide for details on how to contribute.

Code of Conduct

See the Code of Conduct for details. Basically it comes down to:

In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to making participation in our project and
our community a harassment-free experience for everyone, regardless of age, body
size, disability, ethnicity, gender identity and expression, level of experience,
nationality, personal appearance, race, religion, or sexual identity and orientation.

License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License(link is external).
License holders are all contributors(link is external).