Test Automation - Tools versus Frameworks

Disclaimer: In the Tricentis Tosca space, I have/ had pretty much every certification except for Test Architect, where I've failed the entry exam, and in the SmartBear space I am certified in SOAP UI Pro, and I also do programmatic test automation with Java (mostly).

Every so often, I get asked about which tool is better, when it comes to test automation, and there were times when I would honestly nominate one of the tools on the market, but those days are gone.

So, the short answer is that the best tool on the market is your brain.

Do you want the tool to click on a button? In programming you can do just that, often times by something as simple as button.click(). Do you want to organize your tests in a certain way? Of course, in programming you can use just that, we have packages, classes, methods.

This is not to say that programming is a solution to all of the problems, it is just that often times the tools create/ have problems of their own, so basically you solve a problem by creating another.

But this is not a plea for programming in test automation, I am not to say which approach is better for every project out there, it's just the realization of the fact that every test automation that I've seen using tools was mediocre at best. I have seen, however, very elegant and intellectually appealing test automation implementations that I've fallen in love with (not you, Jasmine!).

I think, therefore, the conversation around tools versus frameworks should be around honesty:

• are your test automation engineers gifted enough to do some programming?
• are your test automation engineers willing to do some programming?
• are your test automation engineers professionally proficient enough to continue learning and re-learning?

If the answer to any of the above is maybe, then probably a combination of some tool and a mediocre outcome is to be expected, at best, from the project. I mean, it's not great and it should not be great, rare are the projects where test automation encompasses art and engineering, so it would just be good enough, maybe, so... fine.

But, if you ask me, and I got asked just today, the best test automation tool is a team with a mix of people that are knowledgeable, of people that are open to learning more, and of people that are committed to upskilling themselves.

In short, there is no magic, it's just hard work that gets easier and more palatable with time.