Gaurav Singh

Principal SDET @ CRED, ex Meta, Gojek with 13+ years of hands-on solving deep testing problems and leading engineering teams

3 minute read

Python and requests and header

Logos in header image sources: Python, Requests, JSON, HTTP, Cerberus ReportPortal

This is the ninth and final 👋 post in our series on how to build an API framework using python.

You can read previous parts below:

This one is gonna be short and sweet 😉

As the size of your automated test suites grows from 10 to 100 to 1000 automated cases, the only way to scale your automated tests is to run them in parallel.

Often Test automation engineers start thinking about parallel runs too late in the game at which point modifying your automation is a very tedious task. Patterns that work for serial execution sometimes can be a blocker for parallel runs.

If you are completely new to this problem space and terms like concurrency, threads, processes and parallel testing sound like greek, don’t worry 😉, no one knows about it till they really encounter it and solve it for themselves. I promise to write a more detailed blog around this topic in the future. Keep watching this space out for it.

I recommend going through These amazing chapter in MIT Open courseware 6.031 course on Software Construction on:

Done reading? 😇

Let’s proceed.

How can we set up our current framework to run our tests in parallel?

Pytest provides a wonderful plugin called pytest-xdist just for this purpose that makes it quite easy to achieve this

Setup

Install below modules via pipenv

# Install pytest-xdist plugin
pipenv install pytest-xdist
pipenv install pytest-xdist"[psutil]"

# Alternatively
# After switching to example/09_running_tests_in_parallel branch
pipenv install

Running in parallel 🚀

Run below command to run your cases on multiple cores in your machine.

Here -n mentions the number of CPU’s to send the tests to. You can either give a fixed no like 2, 3 or auto in case you want pytest to determine the no of processes (as per no of CPU cores)

python -m pytest -n auto

Notice gw0 [8] / gw1 [8] / gw2 [8] / gw3 [8] / gw4 [8] / gw5 [8] in run logs, these indicate different worker processes that pytest-xdist uses to run your tests.

platform darwin -- Python 3.8.7, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /Users/gauravsingh/self/course-api-framework-python, configfile: pytest.ini
plugins: reportportal-5.0.8, xdist-2.2.1, forked-1.3.0
gw0 [8] / gw1 [8] / gw2 [8] / gw3 [8] / gw4 [8] / gw5 [8]
......F.
1 failed, 7 passed in 1.63s

For a small test suite like this, the cost of multiprocessing results in longer runtime however this is easily amortized for a large enough test suite.

Customization

You can tell pytest to run all cases within a single module or all methods in a Test class within a single process by adding flag --dist loadscope

python -m pytest -n auto ./tests --reportportal --dist loadscope

Pytest xdist plugin provides many useful features like:

  • running these tests within a boxed subprocess,
  • over different platforms,
  • getting unique id for the test run and many more.

If you want to learn more, feel free to checkout their github readme as a starting point.

pytest-dev/pytest-xdist: pytest plugin for distributed testing and loop-on-failures testing modes.

Conclusion

This was the final chapter in this long series, I hope you had as much fun 😏 as I had while putting this up.

Honestly, this helped me revisit a lot of concepts and clarified my understanding of things. Do not forget to checkout the full course on Test automation university (when it comes out).

I have discussed all these concepts with more depth in the videos.

If you found this post useful, Do share it with a friend or colleague and if you have thoughts, I’d be more than happy to chat over at twitter or comments. Until next time. Happy Testing and coding. 😄

Comments

You May Also Enjoy

Is there growth in testing?

9 minute read

Is testing a dead-end career? Should I continue in testing? If I’m still here, how do I grow? Is there a future? In this blog, I provide my take on this freq...