My Approach to Testing

Someone asked me, “do I have a method for generating testing ideas?”. My answer; “I follow an exploratory approach to simultaneously learn, generate and execute test ideas”. This might involve exploring the product a little to learn about how it works, this will feed into questions/ideas and then I might explore some other ideas further.

I might use sources like the Design Sketch as an oracle (source of truth) for inspiration/ideas, or my mobile testing experience (heuristics/rules of thumb like “accessibility always sucks and its easy to find bugs there” or “screen rotations often causes bugs”), or things like boundaries to explore. It depends on the context of what I’m testing to what ideas get generated.

Exploratory Testing

Exploratory testing is a fairly efficient approach to finding bugs. It’s not like we know where the bugs are in the product before hand. However there are always gaps in anyone’s testing approach. It would take an almost infinite amount of time to test all of the possible permutations. So testers will also use their risk radars to help hone in on testing high risk areas.

Mareet Pyhäjärvi has an interesting keynote for SeleniumConf on the intersection of exploratory testing and automation:

And she also has a Medium article explaining Exploratory Testing. Here is the sketch notes for her talk:

James Bach has a 10 page article on exploratory testing but it can be a little dry to read all in one go. I’ll also often reference Elisabeth Hendrickson’s book, “Explore it! Reduce Risk and Increase Confidence with Exploratory Testing“. Elisabeth has also put together a cheat sheet for testing heuristics but I feel like that is more suited to testing a web frontend context.

Mindmaps

I often use mindmaps to help generate ideas, e.g. when I was at Tyro I created this mind map for generating mobile testing ideas, the idea being I’d bring this along to planning meanings and it would prompt me to ask questions like, “what about performance testing?”.

Everyone does testing on some levels, some people are just more practiced at the skills involved than others. For me, testing is my craft.

Technical Presentations

A few years ago I was lucky enough to win a YOW! diversity scholarship for a day of speaker training with Damian Conway. I’ve never had the opportunity to learn so much about public speaking and giving technical presentations in as condensed timeframe before. These are my hints and tips picked up from that training and other public speaking activities.

Here are my slides, if you like the sketchnote style, Here is a guide on getting started with sketchnotes.

Inspiration for this content

These following talks are good inspirations for better tech presentations

Damian Conway’s YOW! Night talk on Instantly better tech presentations

Vicky Brasseur’s 10 step program for great tech talks

Brainstorming your talk

Damian goes through a brainstorming technique to help form your talk. DO NOT start with slides, this is a sure way to waste time with creating your story as you iterate through it. First off, start with a pen and paper (or equivalent digital note taking technology), brainstorm all of the possible things you know about a topic. Pick 5 sub-topics you know the most about. Ask yourself, do these create a narrative flow? Do they form a story? If not, chuck one idea and pull in another that helps with the story. Then for each sub-topic brainstorm all of the things related to it, pick 5 from that list that forms a mini-story. You now have a good, narrative flow through your presentation and your slides almost write themselves, you could now put one word/phrase on each slide that reminds you of the story you are telling at each point and you’d have a pretty good presentation.

Slide Design

Think of your slides as a user interface, too much information will overload your audience ensuring they switch off and start browsing the web. Your audience doesn’t have the insights you have, that’s why they are in your audience. They want to learn, what do you want them to learn? Most often they won’t care about you as a person, they care about your content. Structuring your content around stories will help make your information easier to digest. Big walls of texts are a huge no no. Slides are free, why does all of your content have to be squished on 1 slide? I avoid dot points like the plaque, I can’t see any value they add being on slides.

Your text is too small

Always assume your text is too small, just because you can easily read it on your high resolution retina display doesn’t mean the person in the back of your audience is able to read it through a shitty 800×600 projector with terrible contrast. Also never put anything important at the bottom of your slide, how often have you strained to look over someones shoulder to try and read what’s on a slide?

Ditch that Intro slide

We (the audience) do not need to see what you look like, you are standing in front of us. We also do not need to know why you feel the need to justify why you are on stage. You already sold us with your title and description in the conference program. So please ditch that intro slide, it may switch off your audience before you’ve even really started. The audience is there to learn from you, just jump into the good stuff.

Slow Reveal

Build up to showing complicated diagrams/graphs rather than showing it all at once, you want to walk people through you idea not hit them over the head with it.

1 thought per slide

This is called Takahashi method, it’s awfully minimalistic but it forces you to use your slides as a crutch for your story telling rather than you relying on your slides too much. It’s worth experimenting with as an idea.

Showing code

Screenshots of code work better than switching into an IDE/text editor, your code will always be too small with a text editor. If it’s a screenshot you also get syntax highlighting. Screenshots also help you to simplify the ideas you are trying to communicate.

Opaque boxes

Can’t avoid the complicated diagram? Fine, use opaque boxes and slide transitions to help guide your audience through the diagram. Take them on a journey. Here is an example using an XKCD diagram.

Demo’s are optional

I actually enjoy giving a demo first and then explaining how I got there, that way if it fails I still have content to continue with and I don’t end on a sour note. This way I always ensure I have enough time for the demo; which is often the fun part of technical presentations.

Practice

Nothing beats practice, if you can’t practice with colleagues, friends or meetup groups set up a TV with a picture of an audience and present to that. Or present to a mirror, or to your cat.

Other Resources

Want to take your presentations to the next level? Have a read of TED talks by Chris Anderson, audio version here.

Agile Australia 2018

On Tuesday the 19th of June I spoke at Agile Australia( access the recording here) on how to get more people involved with testing. You can access my slides; The bug hunt is on. I proposed 5 activities to help get more people involved with quality:

  1. Bug Bashes
  2. Bug Bounties
  3. Dog Fooding
  4. Knowledge sharing practices
  5. Soap Opera testing (you also watch the first three minutes of this video)

Sketch notes

Here are all of my sketch notes on the presentations I attended;

My personal highlights where Nigel’s talk on “Agile is the last thing you need”:

And Martin Fowler’s talk on “the state of Agile in 2018”:

Steven gave an interesting talk on visual strategy maps:

Getting up on stage

I enjoy performing. Don’t ask me why. I can’t explain it. You could say it’s something to do with the rush, or the perception of adding value or entertainment for other people. I want to tell you a few stories about my adventures in performing. Do you want to improve your performances? I’m available for free consultations on improving technical presentations.

During High School

I was involved with nearly every extra curricular activity I could sign up for. I was in the school concert band; I played trombone. I can legitimately say, “this one time in band camp …”. I was the fat kid in school, there weren’t many other kids fatter than I. I once got up in front of my whole school dressed up in a Santa suit and played Jingle Bells on the trombone. Talk about a nerve racking, getting out of my comfort zone experience. I got a laugh at least. I was in an Auslan signing choir (Australian Sign Language) and a singing choir too. In the signing choir we would often perform to retirement homes in the area and our signature song was, “I believe I can fly” by R Kelly. I could still sign to that song. What does a signing choir performance look like? Check out this example on YouTube;

Watching that makes me want to sign up to an auslan class and pursue deaf poetry.

I was also in a musical. It was called Wolfstock, it was a 1950’s themed musical about a 16 year old boy called Jay, his parents had sold his soul to the devil and had to get to wolfstock (aka woodstock) before the next full moon or else he would remain a warewolf. I played Wolfman Jack in act 2; a character based on the DJ host by the same name, I even had my own song. I’m sure the musical was terrible. My mum has it on tape somewhere. I’m sorry mum for putting you through all of my horrible performances in school.

During Uni

I ran my own radio show on a community radio station called, “chat with an engineer”. I would interview engineers in our community and chat about the work they did. It was to help raise the profile of Engineering. I didn’t have the budget for the training course so I asked Engineers Australia if they’d paid for me to do the course. They did and I’m forever grateful for that. My biggest success was interviewing 2012’s Young Australian of the Year; Marita Cheng. She was visiting a high school as part of a Robogals visit and we were able to organise an interview.

I also started the Robogals Chapter in Tasmania. Robogals is a student run group who promote engineering and technology to young kids through lego robotics workshops with the goal of increasing female engineers. I taught robotics to over 1000 kids in tasmania in the 1.5 years I was involved with Robogals with next to no funding and while going through my first bout of chronic depression. I can’t understand how I was functioning, I wasn’t passing uni so let’s just say I wasn’t functioning very well. Teaching is another type of performance that I enjoy.

Professional Presentations

During my professional career the main performances I’ve been involved with are presentations. My most nerve wracking experience was getting up in front of the whole company during an all hands and talking about my struggles with depression. Getting that venerable in front of such a large crowd is another one of those big, “getting out of my comfort zone” experiences. It’s definitely made giving technical presentations easier. Interviews are another performance. A lot of people hate interviews, in a weird way I enjoy them. Having that opportunity to talk about my passions in software testing is what I enjoy. I am narcissistic. I remember doing a first year psychology 101 personality test during uni, I scored very highly on the narcissistic scale and I’m ok with that. It’s only an issue when it’s combined with a lack of empathy.

I’ve been involved with a few community bands since moving to Sydney. The Sydney homotones and Sunday Assembly being the main ones. I’m not actively involved with any now but I would love to join a community swing band. Or do some taiko drum classes. Or learn how to play the double bass. Garhhh, I can’t decide.

My favourite presentation has been my talk at YOW! Connected last year on using robots for mobile testing;

I was able to combine my passion for music, robots and mobile testing. #Winning at life.

I’ve collected a bunch of hints and tips on giving presentations. Reach out to me at sam[AT]thebughunter.com.au if you’d like a free consultation.

Evolution of my CV

My CV has evolved a bit over the years. But there have also been a few constants too. I’ve always tried to keep my CV under 2 pages, my most recent one is 1 page with embedded links, it’s more like a portfolio.

Sam Connelly Tester Profile 2014

I used this CV to apply for my role at Tyro, it was created using Word and it’s interesting to read over how I presented myself 4 years ago. This was my first attempt at putting the 5 c’s of testing on my profile.

Sam Connelly Tester Profile 2016

I started experimenting with the 1 page CV, using Canva (an online designer’s tool). I was told somewhere that dates didn’t matter as much as duration. I used this CV to get my role at EPAM Systems. I still have the 5 ‘c of testing but I’ve replaced crazy with coding.

Sam Connelly Tester Profile 2017

This is a 2 page CV that I used to help me get my role at Campaign Monitor. It went through a few iterations and I had a few other ideas in how to tweak/and experiment with it. I also wanted to experiment with a timeline representation;

timeline view of my CV

I never submitted any CV’s with the timeline view but it was an interesting idea.

Bug Hunter Sam Connelly 2018

This is my latest iteration, it’s a 1 pager. Created using Google Docs. I’ve dropped the 5c’s of testing idea. If I was to keep it, I would mention it but link out to a blog post about it. I’ve also adjusted the date I graduated from Uni. Because of my exchange year, changing course and bout of chronic depression my uni was all over the place. I had done most of my course by 2012 and was only doing 1 subject a semester in 2013, I was doing a full time testing contract at the time and had done a 6 month part time testing job before that. I recently had someone reject my 2017 CV because the recruiter did this quick check; 2017 minus graduation year of 2013 only equals 4 years of experience = rejection, we are looking for 5+ years experience. So instead of trying to explain this I’ve just put 2012 as my graduation year.

The whole point of a CV is to make you seem like an interesting enough person to invite into for an interview. Think of it as a user interface with that whole purpose in mind. How can you simplify the data and layout to make it easier to read? How has your CV evolved over time?

Back on the job market

I find myself back on the job market after a break up with Campaign Monitor. I didn’t successfully pass probation. It was a mutual thing and both sides of the discussion were adult about it. These aren’t easy conversations to have and it doesn’t serve any purpose to get angry and rage quit. I am a little sad to leave because I enjoyed the company and people but I wasn’t able to advocate for quality in a way that added the business value they needed from the role.

Depression and Job Hunting

When I was job hunting around 8-9 months ago, it took me well over 2 months to find a job and interviews with over 13 companies (blog). However in that situation I wasn’t in a rush and was willing to wait for something that looked like it would fit me well. The constant rejections were hard to deal with; especially when I had been experiencing a spell of imposter phenomenon and feeling like I was not good enough for anything. I also broke my ankle during these job hunting efforts which had a huge impact on my mental well being (blog).

My broken ankle contributed to a relapse of depression at the start of the year. Because of this I wasn’t able to give my 100% to the new job at Campaign Monitor and this negatively impacted the engineering’s team view of the Quality Coach role. Once your perception of value is seen in a non favorable light, it is very challenging to recover. You only get one shot at leaving a first impression and your reputation is built up on that. I didn’t do a great job when I started, then I tried a new team and a new process and saw some improvements. However there were still some doubts if this role was what the company needed and if it was the right fit for my skills. I went to a third team for the last 3 weeks but I feel like the decision had already been made by that point.

Keeping Track of Job Hunting

I used a spreadsheet in my previous job hunting efforts to help me keep track of where I was up to with every company;

spreadsheet of previous job hunting efforts

With this spreadsheet I noted the source of  the lead; I was relying on mostly LinkedIn and a technical recruiter from Opus. I noted down where I was up to in the interview process, excitement for the role (out of 5) and any follow up notes. I also noted the few companies who contact me after I had received a successful job offer.

Will I do the same thing this time? I’m not too sure. I’ve got the luxury of around 2-3 weeks for job hunting before the personal budget starts getting a little tight but it would be worth experimenting with the spreadsheet again this time.

What am I looking for in a new job?

This time round I have more confidence in my skills as a tester. Last time I wanted to quit testing and try something different (either Android Development or Product Manager). However now I know I love growing my reputation for being known as a passionate tester. In a few years time I’d love to be running my own company focusing on running workshops for technical testing and mobile apps (e.g. TDD and kotlin, Continuous Integration and iOS). I’m not there yet so I’m looking for a mobile app testing role while I work on workshops in my spare time.

I’d love to have a role with support for speaking at conferences. I’m speaking at Agile Australia on how to get more people involved with testing in a few weeks, Selenium Conf in India at the end of June on using robots for mobile app testing, CAST in Florida in August on stories in becoming a quality coach. I now have an anecdote where that didn’t work out for me so that should be an interesting talk.

Why bother testing?

As a tester, I don’t practice selling my craft very often. This blog post is an attempt to put together my thoughts when asked the question, “why bother testing?”

Let’s talk about bugs

When was the last time you used some buggy software? How did it make you feel? Did it cause you to swear at your computer in frustration? We are surrounded by software full of bugs, some bugs are minor but some cause us to pull out our hair in frustration. Some bugs when discovered cause nightmare headlines to spread like wild fire. Buggy low quality software is hard to sell, maintain and keep people using. Your customers won’t rave about your products to their friends if they think it’s shit.

Your customers won't rave about your products to their friends if they think it's shit

Testing doesn’t improve quality

Testing itself doesn’t improve the quality of the product, but it can help highlight issues that could impact the perception of quality. Bugs will always exist in software, it’s impossible to test for every possible scenario every time before releasing to production (especially in a world of continuous integration/deployment) but if you knew about some the bugs in you products before your customers find them then you’d be in a better place to make informed decisions about what to release and when.

Testing is active discovery

Everyone does testing on some level, most of the time we are unaware of what we are doing until we discover a frustrating situation. Testing is a skill where you practice looking for quirks in software. Testers are constantly experimenting and observing the product and are well practiced in talking through their thought processes.

There are generally 2 activities people say they are doing when they talk about testing;

1 – Verify the product works as intended

2 – Actively go hunting for bugs

Activity 1 is usually where people talk about automation, when your product can be codified as “working as intended” you might be in a place to build some automation checking to help facilitate faster feedback. However you could build all of the automation checking in the world into your products but people can still think it’s shit. You users don’t care that your unit tests are less than 0.01% flakey.

Activity 2 can be easily practiced, it’s not like anyone knows where the bugs are in software until they are discovered. You can build things into your products that make it easier for people to report the bugs they find or you could do a chaos monkey approach to find out where your product crashes unexpectedly.

There are many other activities involved with testing but I want to keep this high level.

People say testing is risk mitigation

This is not true, testers may use their own internal risk radars to help guide their testing efforts but testing itself doesn’t mitigate any risks. I like the analogy; testing is like an x-ray, an x-ray on it’s own can’t tell you how healthy you are or prevent issues from happening but it’s a tool that can give you a snapshot in time so your doctor can make informed decisions about some element of your health.

Testing can help people answer questions like, “are we comfortable shipping this code to customers?”. Bugs are a fact of life but we can’t fix what we don’t know. Testing helps us discover these potential issues.

Running a Bug Bash

For any solo tester out there I recommend leading a regular bug bash/mob testing activity. It’s an activity you can run at the end of a sprint/feature dev cycle. You invite the team, get some snacks/beverages together and get everyone testing for around an hour.

Setup

You might want to make sure you have good device/browser coverage by ensuring everyone is set up with data/devices before hand. You might also want to prepare some help docs to help people get started. I like to have a mindmap on coverage ideas prepared before hand so people have a visual indicator of what they are testing. I might also create a bunch of test accounts before hand and distribute them to the team.

I use this whiteboard for running bugbashes at Insight Timer; regression on the left, what’s changed recently in the middle and “other” things to consider on the right

Light Bug reporting

You should encourage light weight bug reporting;

  • maybe people write bugs on sticky notes
  • add bugs to a shared spreadsheet
  • or raise Jira bugs directly from Slack using a /jirio create bug shortcut

The focus should be on finding bugs, not getting caught up in how to report them. You can always clarify issues further after the bug bash if you need more information.

What to test in a bug bash

I have a 3 step heuristic for deciding what to test (think; guide or rule of thumb):

  1. what’s changed recently? Change introduces new risk. what features have been built? What code has been refactored? What libraries have been updated? Focus the team to test these areas first
  2. Regression testing. Have a checklist of at most a dozen scenarios of core functionality (maybe around registration, payments and the main stuff people do with your product). Ask people to consider these cases while doing 1), maybe ask people to put there name next to a case when they test it so you can get an idea of coverage
  3. What other testing should/could we do? Sometimes it might make sense to do a pen test bug bash session or a performance testing bash or a cross browser test bash but it doesn’t have to happen every time

Test in Production

If you can; test in production. There’s no environment quite like it.

Put on your ruby red slippers and repeat after me, "there's no place like prod"
Toto, I’ve a feeling we’re not in Kansas anymore – Wizard of Oz

But if you can’t test in production, make sure your test environment is stable, your data is set up and you’ve given your team the “Do Not Touch except for bug bash testing” request.

Post Bug Bash

Make sure to thank people for their time, count up the bugs and give kudos to your best bug hunter. Testing/Quality is a team responsibility after all. Send out a thank you email with results and award trophies. I’ve handed out this trophy of a rhino bug in resin on a trophy base to our number one bug hunter before:

rhino bug in resin on a trophy base
Bug Bash Trophy

Good morning World, what offends you today?

Ah the internet… is it just me or a things getting angrier lately?

Sometimes I can see how the internet makes people angry. E.G. there is a bit of a Uncle Bob vs Sarah Mei thing happening on twitter over how the word craftsman isn’t inclusive. I think it’s a reasonable point to raise, I’ve intentionally tried to stop using the word “guys” in my vocabulary here is an example tweet that demonstrates why I might try to avoid using those words;

A fond memory from my university days: I was seeking to understand why we can't use "mankind" or "men" for all cases. A wise woman asked me to name 10 famous men. I named 10 famous males, and no females. Her point exactly. That's the problem with using craftsmen for all cases.
https://twitter.com/srogalsky/status/991431460681854976

So I’ve come to realize words like guys and men aren’t really all that gender neutral/inclusive after all. I personally didn’t think about the word craftsman being noninclusive before this point. I do talk about the craft of testing and will continue to talk about it. But I’ll make sure to try to catch myself if I’m about to use the word craftsman. Huzzah, the power of language; it changes and adopts to new cultural expectations. Original tweet for reference;

Uncle bob says; "Rude. Craftsman is a gender neutral term"
https://twitter.com/unclebobmartin/status/990411971106426880?s=19

My takeaway here; it’s hard to be polite on twitter. Both sides are being rude (but their online personas are built around this, I imagine these people are too busy to give a shit over how the world interprets their tweets and rudeness/bluntness can help drive conversation, it motivated this blog after all and many people are now re-assessing the word craftsman). But I’d have to side with Sarah on this one and I think Uncle Bob was just a little ruder than Sarah here. Do you disagree with me on this one?

Other things I see are just riding the “anger” train to get people riled up. People on both sides of the equation are guilty of over reacting though. Then there’s this teenager who found a Chinese dress in a thrift store and wore it to prom;

How did this become news? I was given a Japanese Yukata from a mayor of a Japanese town, do I have to be concerned when I wear that now? I think that aj+ piece is just asking for that knee jerk emotional reaction, I’m actually a little disappointed in aj+ about this. It reminds me of an article I saw come up in my facebook feed from a pop culture media outlet about wonder woman’s hairless armpits; about how angry feminists were critiquing the trailer and asking, how does wonder woman shave? This was clearly click baity because 1 “okay the Wonder Woman movie looks rad but why does Diana have clean shaven armpits” is hardly angry (or did I miss something?) and the article was clearly shared on facebook for the knee jerk facebook reaction, “dam special snowflake feminazi’s ruining a good movie like wonder woman”. I stopped following the pop culture media outlet because of it. Does anyone remember how angry Australia got over Yassmin’s, “Lest. We. Forget. (Manus, Nauru, Syria, Palestine…)” tweet? I definitely wasn’t proud to call myself Australian after the hate she received after that.

However anger is pretty effective at engaging people, it’s a common marketing tactic now (get people angry and talking about it). BUT I think it causes people to get into a “defensive” thinking mode and it can be really hard to reason through that knee jerk emotional reaction.

Now I’m not a christian but when I feel that anger bubble up and inspire me to get all keyboard warrior I take a moment to reflect and think, “how would Jesus react with love and compassion in this situation?”. Jesus is my role model in these kind of situations. How do you catch yourself before you respond in anger? Or maybe I should harness my anger to drive discussion? Like James Bach does in this blog about Machine Learning tools in software testing are bullshit. It definitely has a lot of people talking about it.

Is it just me or are there more angry people out there?

What is software testing?

A quick google of ‘What is software testing’ will give you 75,300,000 results but not much of an answer. On the surface level, we can surmise that it is about testing software but in reality; it’s more than that.

Software testing is an activity that everyone does on some level. E.g. when you interact with a product like Facebook you are helping to test it. You have certain expectations of how the scrolling behaves and what type of content comes up in your feed. You are probably a part of several on going experiments that Facebook conducts and you’re scrolling habits are also helping shape their performance testing approach.

Q. Hang on, if everyone does it, why do we need dedicated software testers?

A. Justifying the value I add to a project can be challenging sometimes but let me try to go into more detail. There are some very successful companies out there that don’t hire dedicated testers but their context and software development practices can help mitigate some of the risks associated with their products.

Software testing is about reducing the risks associated with software development. Software development is, at its core, a business venture. The software is created to solve an issue that the end user is willing to pay money for. Each element within the development lifecycle is working towards providing a marketable solution to that original problem. Software testing serves to highlight any risks that might impact on the value of the product.

The software development cycle is a continuous iteration of identifying the customer’s problems, developing the solution, ensuring the solution works, and selling the solution. Software testing ensures that our apps, websites and software does what it’s supposed to do whilst not doing what it shouldn’t. It is an integral part of any successful software project.

Planning

Good testing can be completed with very little “formal” planning procedures if a skilled tester is involved. You can see an example test report generated from 1 day of testing from James Bach for testing a medical device.

Often the waterfall model of software development is associated with exhaustive requirement gathering and testing planning even before the software is designed. On the contrary, the agile manifesto focuses on providing working software over comprehensive documentation. The whole Agile vs Waterfall is a conversation for another blog.

When working within the agile methodology we can move away from explicit and highly specific planning and into the rapid software testing realm. James Bach and Michael Bolton are pioneers in the Rapid Software Testing space. This is a context-driven software testing methodology that works on the premise that the testing team will have an understanding of the business problem that the product is solving.

This Rapid Software Testing method allows the testing team to conduct a more informal, risk-based approach to planning the testing of software. Sam explains how to take a risked-based approach to planning UI tests in her /dev/world presentation.

Exploratory Testing

Exploratory testing* is where a tester interacts with software to compare it with their mental models of expected outcomes.

Exploratory testing is a part of all testing regimes because it focuses on human-based feedback. Having people run through the software with intention of finding brokenness helps to highlight any design, UI or experience elements that are confusing or hard to use. This is something that automated testing scripts aren’t able to handle.

Exploratory testers will explore how well the software solves the end-users problem. They will highlight any high-risk issues that will impact the value of the product. Skilled exploratory testers will work on a continuously iterative testing approach where the exploration can help form a plan that guides the direction of future testing efforts. This continuous feedback loop allows testers to improve the quality of their testing rather than just the quantity.

Exploratory testing is based on human observations. This is incredibly useful for user experience focussed testing but less practical for testing that needs to be repeated over time or scaled up to assess performance.

Whilst the benefits of having human based feedback far outweigh the cons it is important to note that human error will always occur. Hopefully using a risked-based approach with an experienced exploratory tester will negate some of these issues before your users find them.

Automated testing

Automated testing (sometimes called checking) involves using software tools or scripts to compare the expected results for how a program should run with actual outcomes. It leverages the computational power available today to automate testing that would have to have previously been conducted manually.

definition of automated testing; involves using tools or scripts to help asses expected behaviour of software

For example, an automated testing suite may be used to check that ‘a + b = c’. To test this we could run a sample of known numbers through the function and ensure that the outcomes were as expected.sudo code for 3 unit tests

This is an extremely simplified example of automated testing but this idea can be expanded to include all of the predefined actions within a software.

Automated testing has numerous benefits including increasing the amount of test coverage available within a budget or timeframe, automating basic or low-level tasks, reducing human error when checking outcomes, reusable tests and leveraging the computational power of machines.

If automated testing suites aren’t maintained in line with the development cycle it can lead to outdated tests that return incorrect outcomes. This can lead to less secure testing coverage and a loss of team morale and trust in the testing cycle.  Additionally, automating poorly designed tests only increases the rate that these misaligned tests are run.

Automated testing has been hailed as the holy grail of the software development world of late. It fits in well with the continuous integration methodology and allows for quick iterations of products. However, it’s important to keep in mind that automated testing isn’t a magic bullet. Instead, automated testing should be used in conjunction with other tools to gain more time for your testers to focus on high-value tasks that are outside the scope of automation.

Unit Testing

Unit testing is about isolating the components in your software and testing them individually.

definition of unit testing, isolating components to test individually

Unit testing is a powerful tool when used early in the development cycle. This is a tool often employed by developers to ensure that the code they’re writing works.

Like all of the tools that we have explored today unit testing in only one option that a development team will employ during as part of their testing process.

Conclusion

Anyone doing testing has a plethora of tools available to them when conducting a thorough testing cycle. Often we get caught up in the discussion of Exploratory Testing vs Automated Testing but we need to bring this back to what is the tool that will best fit this testing scenario. I’ve only scratched the surface here of what can be involved with software testing.

Software testing is a continuous feedback loop. Everyone does testing on some level, and one of the goals of testing is to highlight any issues or risks that might impact the perceived value of the product. Ultimately we want to help deliver quality products on time and on budget.


*The intricacies about whether exploratory testing should be referred to as ‘testing’ has been discussed extensively within the software testing industry. Please note: for ease of understanding Sam will be referring to all testing without scripting as ‘exploratory testing’ within this article.

**Sam commissioned Stefanie C. from upwork to help write this article, there might be a follow up blog on Sam’s experiences using freelancers to help outsource work later and why she would do this