Reordering Test Run Columns

Today we introduced an oft-requested feature to provide for rearranging the order of test run columns. Work is continuing with the bigger ticket items of folders and permissions, but the re-ordering of test run columns has been requested by so many customers that we took a short break to push this smaller update.

CTRL-LEFT and CTRL-RIGHT

Typing Ctrl-Left or Ctrl-Right will move the currently selected test run one column to the left or right.

Where a test run is a retest of older test runs, the older test runs (when shown) will always stay with their latest retest. Remember you can show old test runs by right-clicking on the test run header and selecting Show Old Test Runs.

ALT-LEFT/RIGHT and CMD-LEFT/RIGHT

For some users, Ctrl-Left/Right will have other window-related actions, so Alt-Left/Right and Cmd-Left/Right are also provided as alternatives. Have a try and see which key combination works for you.


Infrastructure Updates

And in the background, we also took the chance to upgrade the versions of Django, MongoDB, and Linux that Testpad runs on to pick up the latest security updates. Thanks to the flexibility of Amazon's Web Services, together with MongoDB's seamless clustering, these updates were performed without any break in service.


Any questions on test run reordering, service updates or new feature requests, please email stef@ontestpad.com

Coming Soon: Folders, Permissions and Docs

The blog has been quiet this Summer, so I thought I'd post an update as three big improvements are afoot, but not quite ready for release.

First up will be folders to offer more flexibility in how scripts are organised within projects. Many customers, and indeed Testpad's own testing, are using Projects to model Releases; a new Project is started per Release, with the set of Scripts copied (by drag'n'drop with CTRL held down) from the previous Project/Release. This would work better if Testpad had folders within Projects so that, say, each Folder could look after a Release instead of the whole Project.

Next up will be a big extension to the permission model. Admins will be able to assign Users to one or more Groups, and then Projects can be restricted to be only visible/accessible to members of defined Groups. This will be of most use to the many consultancies using Testpad who wish to keep visibility of client-confidential material limited to just the teams working on those projects.

And in parallel with the above, Testpad's much needed documentation is in the works, covering all the features of Testpad and giving examples of good ways to deploy the testing process.

Timescale? I'm going with a non-committal "soon" :)

By all means email me at stef@ontestpad.com if you want to contribute any ideas in the folders, permissions, documentations areas - this would be an ideal opportunity to incorporate suggestions from the Testpad community.

Improved Filter Controls

This weekend saw lots of small improvements for Testpad, the biggest of which is an upgrade to how tagging and filtering works.

Filter with OR logic

Test Runs (columns) have a Tag Filter field that defaults to "ALL" which means that run will include all the tests. If you change this to e.g. "WIN", then that run will only include tests that are tagged "WIN", and the rest will be drawn greyed out.

Multiple tags can be supplied, and as before, if separated by a comma or space, will be applied using logical AND. For example, a run with Tag Filter set to "WIN HI" will only include test rows that are tagged with both WIN AND HI. If the desire was to include all rows with either tag WIN or tag HI, then that didn't used to be possible. However, with this release, you can now use the '+' character to achieve this kind of additive filtering. E.g. a run with the Tag Filter "WIN + HI" will include all tests tagged WIN OR tagged HI.

A note on precedence: there's no support for brackets or parentheses for complex expressions, instead the space/comma (for AND logic) takes precedence over the + for OR logic. So a more complicated Tag Filter of "A B + C D" will be including all tests tagged A AND B along with all tests tagged C AND D.

The minus prefix '-' still works to negate the match, e.g. the Tag Filter "A + -B" will include all tests tagged A along with all tests NOT tagged B. Note again that because parentheses aren't supported, there's no way to negate a group; the minus prefix can only be used on a single tag at a time.

Setting and Viewing Tags

To help with the new tag filtering, tags for test rows are now displayed as labels to the right of each row. As space is limited, tags are shown in fixed width labels that show the first 5 or 6 letters of the tag, and Testpad will only attempt to show the first 5 tags for each row.

Tags are set using the Test Details dialog, which can be opened in a number of ways: double-clicking on the row ID, clicking on the small corner triangle on the right-hand end of the row, or typing Alt-T.

Tags applied to outdented (parent, or title) rows will affect all their indented child rows. This is very convenient when you want to include/exclude whole sections of a script for a particular run; it saves on having to set a tag on each and every test.

User Interface Tweaks

In addition to the tag filtering improvements, a few UI improvements were also included in this release:

  • blocks of test rows can be selected and copied even if the Script has been locked for editing (remember Copy/Paste works between pages/tabs)
  • test run emails no longer CC the sender, instead they set the Reply-To field; this is to help with testers hitting Reply on these emails and inadvertently emailing Testpad Support instead of their test manager!
  • new option for the Auto Advance setting when running a Test: OFF simply turns off auto-advance so that the focus stays with the current test


As always, feedback on these changes is very welcome; please email me at stef@ontestpad.com

Growing Customer Base means Bigger Servers

This Spring has seen a signficant increase in traffic due to a growing customer base, to the point of prompting server upgrades that were just completed this weekend. The new servers have increased Testpad's capacity and reduced page loading times; with the upgrade process itself proving the convenience of technologies like MongoDB for seamless database replication and Amazon EC2 for command-line control of servers archicture.

Whilst Testpad's customers have been busy, Testpad has been quieter on the development front. However, with these latest upgrades, and further improvements around the corner, it's set to be a busy Summer. In the short term, look for improvements to tag filtering, followed later on with the publication of much needed user guides and documentation.

The rest of this post has a strong engineering bias. Read on if you're curious about Testpad's hosting, or get back to your testing, which should now be snappier than ever!


MongoDB + AmazonEC2 = Upgrades with Zero Downtime

The upgrade saw the main application servers moving from Amazon's older generation of servers (m1) to their latest (m3) servers. As well as newer CPU architecture, the m3 servers come with SSD storage, providing a big increase in database IO performance.

Thanks to the convenience of the Amazon EC2 management interfaces, and robustness of MongoDB's replication technology, these upgrades were performed with zero downtime.

Just to be safe, the upgrade was timed for the weekend when traffic is quieter than during the week, but nevertheless, the logs showed continuous usage before, during and after the upgrade, with no interruption to service.

Broadly, the upgrades were achieved by modifying one database (MongoDB) replica and application server at a time. Once a database replica is shutdown, the live service is maintained by the surviving replicas, thus freeing that server for maintenance. As all Testpad instances have their Linux root partition on an EBS volume, the upgrade itself is as simple as using AmazonEC2 controls to Stop the instance, edit the instance type, and Start it again. When the instance boots up, it is now running on bigger and better hardware! The database folder is then moved from its previous mount point on an EBS volume to its new home on the SSD drive. Once restarted, the MongoDB replica contacts the other replicas and catches up with all the changes that happened during that server's upgrade. Seamless.


Hourly Database Snapshots

Readers familiar with AWS might spot that m3 instance SSD drives are 'ephemeral' devices that do not persist if the instance is stopped or otherwise lost. Testpad therefore still runs replicas with the data volume stored on EBS drives, which continues to enable Testpad's backup strategy of hourly snapshots and archive to AmazonS3.


More Info and Feedback

Please get in touch if you have any feedback or would like to know more about the details of this upgrade - just email stef@ontestpad.com.


Keyboard Shortcuts for Even Faster Testing

Testpad now supports keyboard shortcuts to set the results of tests during testing.
When running a test, with the Test Run dialog open (pictured right), you can press:

  • 'p' or 'space' for pass
  • 'f' for fail
  • 'b' for blocked
  • 'q' for query
  • 'x' for exclude

You'll need focus on the result buttons (also as pictured right) for these new shortcuts to work.

If you want to record comments and issue numbers, hit 'tab' to cycle through the input boxes and finish a test by setting its pass/fail status. For example, the sequence to fail a test with a comment and issue number would be:

  1. Press 'tab' to move focus to the Comments field and type a comment
  2. Press 'tab' to move focus to the Issue field and type an issue number
  3. Press 'tab' to move focus back onto the result buttons
  4. Press 'f' to fail the test and auto-advance to the next test

Really Fast Checklists

If your tests are just simple checks that mostly pass without comment, then the spacebar shortcut (as an alternative to 'p') makes for a very fast way to zap through a checklist, ticking off items as they're checked.


Edit Tests during Testing

As before, you can continue to edit test decriptions during a test run. This is useful as you're most likely to discover a problem with the test description during a test run, and being able to edit the test right there and then saves on having to remember to come back later

Using the mouse, you can click on the test text, make the edits, then click back on the Test Run details dialog to continue testing.

Or if you prefer to stay on the keyboard, hit 'Esc' to close the Test Run dialog, hit 'Enter' to start editing the currently selected test row, make the edits, hit 'Esc' to finish editing the test row and type 'Alt-X' to resume testing.


As usual, if you've got any comments or feedback on the new shortcuts, please send me an email at stef@ontestpad.com.

Copy/Paste Between Testpad Scripts

The Copy/Paste feature in Testpad's script editor has been upgraded to work between scripts. This feature now uses javascript's HTML5 localstorage feature to save the clipboard to a place that survives page loads.

Between Page Loads

This means that tests copied on one script can be pasted into another script even after navigating to another script in the same or a different project.

Between Tabs

It also means that tests copied in a script viewed on one tab can be pasted into a script open on a different tab.

The only restrictions are that it will only work within the same account and on the same browser.

Paste Above or Below

As before, you can paste copied tests above or below the current selection.

By default, Ctrl-V (or Cmd-V on a Mac) will paste after the current selection, or at the end of the script if there is no selection (press Escape to clear the selection).

Shift-Ctrl-V (or Shift-Cmd-V) will paste before the current selection, or at the top of the script if there is no selection.

Copying a row will automatically include all the child rows (indented rows) of the current selection, making it very convenient to select and copy significant portions of a script in just a few clicks.

DIY Cut Support

Ctrl-X remains unsupported, but is simple to achieve by first copying with Ctrl-C then deleting with Del or Backspace. The copied (now cut) rows remain in the clipboard after the deletion and can be pasted back to the same script or a different script after navigation or change of tabs

Browser Compatibility

HTML5 localstorage is well supported these days, so it should work in the major/modern browsers. We have tested the new copy/paste on IE9/IE10 on Windows7/8, and Safari/Firefox/Chrome on OSX, but if you have any issues, please get in touch - stef@ontestpad.com.

Bug tracking - multiple issues per result

Testpad has always let you insert bug numbers against results. Typically these come from a 3rd party issue tracking tool such as Jira or Bugzilla. Further, these numbers can be a clickable link to the relevant page in the 3rd party tool if the bug-link pattern has been configured (either in project settings or account settings).

But in an update today, Testpad now lets you enter more than one bug number against one result. Simply separate numbers with a space and Testpad will treat them as separate issues when generating links in both the script-edit view and the various report views.

There isn't much space for more than a couple of numbers to be seen in the script-edit view, so the printable report has been extended to list all the issues against results, again as clickable links, in the comments section.