A testing system for catching visual regressions in Web applications.
Watches you browse, takes screenshots, tells you when they change
This is an archived project and is no longer supported or updated by Facebook or Instagram. Please do not file issues or pull-requests against this repo. If you wish to continue to develop this code yourself, we recommend you fork it.
Huxley runs in two modes:
Using Selenium WebDriver, Huxley opens a page and records your actions. When you press enter in the Huxley terminal, Huxley will save a screenshot.
Testing a new flow is as simple as manually testing it once. Huxley will remember and re-run your "manual" test plan for you automatically.
You should run Huxley in playback mode before submitting code for review and in continuous integration. Huxley will open the page and re-run your actions with WebDriver. It will take screenshots and compare them with the original screenshots. If they have changed, it will save the new screenshots and warn you that they have changed.
When screenshots have changed, those screenshot changes will show up in your commit. A designer can review them to be sure they're OK. And your continuous integration system can alert you if you forgot to run Huxley.
By default, Huxley will overwrite the old screenshots with new ones. That means you don't have to rewrite anything when your UI changes like you would with a traditional WebDriver test -- Huxley will just take a new screenshot for you and when it's checked in your test is updated!
pip install huxley
examples/you'll find two simple completed Huxley tests. To start from scratch, simply remove
For our example, simply
python -m SimpleHTTPServerto start a basic server for our demo. In your app you may need to start up whatever framework you're using.
A Huxleyfile describes your test. Create one that looks like this:
This creates a test named
togglethat tests the URL
Huxley makes writing tests easy because it simply records your browser session -- specifically mouse clicks and key presses on a single page -- and can replay them in an automated way. To do this you need to install Selenium Server and start it. It's as easy as
java -jar selenium-server-standalone-XXX.jar.
Then, run Huxley in record mode:
huxley --record. Huxley will bring up a browser using Selenium. Press enter in the Huxley console to take a screen shot of the initial page load. Then toggle the button in the browser a few times. After every click, switch back to the Huxley console to take a screen shot. When you've tested all of the functionality you want to test, simply type
qand then enter in the Huxley console to exit.
After confirming, Huxley will automatically record the test for you and save it to disk as
toggle.huxley. Be sure to commit the
Huxleyfileas well as
toggle.huxleyinto your repository so you can track changes to them.
Simply run the
huxleycommand in the same directory as the
Huxleyfileto be sure that your app still works.
You'll likely update the UI of the component a lot without changing its core functionality. Huxley can take new screen shots for you when this happens. Tweak the UI of the component in
toggle.htmlsomehow (maybe change the button color or something) and re-run
huxley. It will warn you that the UI has changed and will automatically write new screen shots for you. If you run
huxleyagain, the test will pass since the screen shots were updated.
If you're using a continuous integration solution like Jenkins you probably don't want to automatically rerecord screen shots on failure. Simply run
huxley --playback-onlyto do this.
Additionally, you may find that you're dissatisfied with Huxley replaying your browsing session in real-time. You can speed it up (or slow it down) by editing your
[toggle] url=http://localhost:8000/toggle.html sleepfactor=0.5
This edit should cut the execution time in half.
Integration tests sometimes get a bad rap for testing too much at once. We've found that if you use integration tests correctly they can be just as effective and accurate as unit tests. Simply follow a few best practices:
It's usually best if you use an image comparison tool like Kaleidoscope to tell what changed. But Huxley includes a simple image diff tool; simply run
--save-diffoption to output a
diff.pngwhich will show you the pixels that changed.
You can set the
HUXLEY_WEBDRIVER_LOCALenvironment variable to tell Huxley which webdriver URL to use for
--recordmode. You can set the
HUXLEY_WEBDRIVER_REMOTEenvironment variable to tell Huxley which webdriver URL to use for screenshots and playback. Usually you only need to use this when working in a team setting such that everyone's screenshots are taken on the same machine configuration (otherwise they'll change depending on who ran them last).
Of course! Simply add a
screensizesetting to your
Huxleyfile. The default is
First of all, if you sufficiently componentize your UI, Huxley can be used as a unit testing tool.
With that said, unit tests have two shortcomings today.
Use it however you want! But we generally shell out to it from within an existing test runner (i.e. Django or Rails). This lets us programmatically start a test server for Huxley to hit.
If you're using Python, you can use Huxley directly in a test (see
huxley.integration.HuxleyTestCase) or browse the source to see its core APIs.
If you're on a team I recommend setting up webdriver on a shared server and changing the
HUXLEY_WEBDRIVER_REMOTEenvironment variable such that everyone's screenshots are pixel perfect (see the technical FAQ above).
Lots of test frameworks and methodologies are very opinionated about how your code should be structured or how you should write tests. Some tools are so opinionated that they're almost religious about their view of testing! We wanted a tool that got out of our way and let us fight regressions as quickly and easily as possible without being opinionated about it. So we named it after the guy who coined the term "agnostic", Thomas Henry Huxley.