March 18, 2021
No Comments

This week, marked the launch of TestProject 2.0, the next instalment of what has been a continuously evolving app for web & mobile test automation. Having attended the webinar on Tuesday and after having some time to play with the new features I thought I’d write a blog post outlining some of the key features and sharing my thoughts, so let’s jump straight in.

The headline feature is the ability to run web and mobile tests offline and outside of the TestProject Cloud, via a new command line interface (CLI). Having talked about TestProject at meetups before I know the ability to store and run tests outside of the cloud has been a much desired feature, especially for companies working in secure environments or with sensitive data.

TestProject users will know that they can already execute tests, stored in the cloud, using the web interface or via the TestProject API. The CLI, however, will provide the ability to run tests stored outside of the cloud. So, let’s see how it works.

If you have yet to sign up to TestProject, you can register here, it’s free. Once registered you’ll need to download and install the TestProject agent. This is how TestProject runs your tests. The agent is required regardless of if the test is stored in the cloud or locally. If you already have a TestProject account, be sure to upgrade your agent to v2, doing this will automatically install the CLI.

Creating a test

The process for creating a test hasn’t changed much. If you’re planning on recording a web test you will still need to do this via the web app. The recording process stays as is. One thing that has changed is that you’ll now have the option to save the test to File or cloud.

Choosing file means that you’ll have the option to save you test from the recorder.

A screenshot of a computer

Description automatically generated with medium confidence

Clicking save will download the test as a .yml file. Nothing will be stored in the cloud and you won’t be able to access this test via the TestProject application, but you can run this test via the TestProject CLI.

Graphical user interface, text, application

Description automatically generated

From the terminal/command line simply run ‘testproject-agent run’ followed by the full path to the .yml file. You should see that the browser opens (if not running heedlessly) just as it would when executing the test via the API or TestProject App.

Once run a html report will be generated and saved locally.

Graphical user interface, text, application

Description automatically generated

The YAML file

Now we’ve seen how the test runs, let’s explore the file. The .yaml file contains everything needed to run the test. If you open it you’ll see details including the endpoint used in the test, details of all the steps (including all the locators and input values), any defined parameters and of course the browser the test is going to executed on. For Mobile tests it details the device(s). The great thing is the file is completely editable. If you want to change the locator or input value, you can directly in the .yaml file. No need to login to the app.

Of course, if you do want to open the test in the app you can, you just need to use the Open file button. From here you can add steps via the recorder and resave the test as a .yaml file.

Test packages

So, we’ve seen how you can save a single file locally and execute it via the CLI but now let’s go one step further. Let’s look at Test Packages.

Test Packages are similar to the .yaml test files we’ve just looked at but they are much more powerful.  They allow us to add multiple browsers or devices to our test and even additional test data for a data driven approach. They’re kinda similar to TestProject jobs.

Ok, say we have a previously recorded test in the TestProject app. All we need to do is select the test and choose ‘Save as File’.

From here we can save the test as a .yaml file or we can select ‘Download as a Test Package’. This then gives us the option to pick our required browsers or devices.

Once selected click save and the download will start. This time it’s a .zip file which is downloaded. The zip file contains 4 files.

  1. packages.yaml – this is like the file we saw previously. It contains the test steps, locators and inputs
  2. project-parameters.yaml – contains all project level parameters defined for the test
  3. test-parameters.csv – contains all test level parameters defined for the test
  4.  settings.yaml – contains a list of browsers for the test to be executed on.

As before, these files are editable so if I want to add another browser to my execution then I just need to specify it in the settings.yaml. If I want to add another set of data I can just add another row in the test-paramaters.csv.

We can run the package, just as before using testproject-agent run. This time we specify the path to the zip file and the CLI will take care of the execution and produce the report.


So far we’ve only mentioned the run command but the CLI is capable of more. You can register and start agents and list local browsers and devices use the help command to see the full list of options and visit the docs here

And there you have it, a brief intro to the new TestProject CLI and the ability to store and run tests outside of the cloud. At present this feature is only available on single tests but I’m sure we’ll see further enhancements to encompass entire projects in the future.

Also announced was the ability to Tag tests and jobs. Tags are searchable so are a great way to locate all tests related to specific applications, Test phases or functionality.

Watch this space….I can see this being used to drive test execution in the future .

A quick note on some up and coming features

It’s clear that the TestProject team are not taking a break after release 2.0. There are some eagerly anticipated features due for release in the first half of this year. I thought I’d mention just a few.

JavaScript Open SDK

TestProject has already released Open SDKs for python, C# and Java but the addition of JavaScript will, no doubt, prove to be very popular. The SDK will support Mocha test framework and Chai assertions, both widely used by testers familiar with test automation in JavaScript.

Jira Integration

The announcement of the impending jira integration was met by cheers (albeit virtual) on the webinar. TestProject currently boasts qTest integration, but jira integration is clearly something the TestProject community were waiting for.

Git integration

We’ve already mentioned how tests can now be saved locally, rather than in the cloud following TestProject 2.0. Of course, this gives testers the ability to store their test files in the version control system of their choice, however, this is still somewhat of a manual process. Explicit Git integration will allow testers to run tests direct from git hub without the need to download the file beforehand, nice, huh?  

Test Parallelisation

Anyone who has been using TestProject will understand the frustration of only being able to run a single test at a time without configuring multiple agents and jobs. The introduction of test parallelisation via a single agent is, no doubt, going to make TestProject even more powerful.

So, that’s it, some of the key takeaways from the latest TestProject release.

Happy Testing!

Ryan Howard

Hi,I'm Ryan. I'm a Software Tester, Quality Advocate and founder of How QA.

Your Turn To Talk

Leave a reply:

Your email address will not be published.