Sydney Testers and Money

We have a Sydney Testers evening with James Bach coming up soon and you might be curious to know why we are charging $5 for it when it seems we have a sponsor for the event (Campaign Monitor). There are actually a few reasons to try this model that I will explain in detail.

Ongoing costs

prices from meetup

Meetup charges US$14.99/month for any big meetup and the organizing committee often foots the bill for this. There were times we were buying name badges and accessories for events. There’s been a few occasions when the organizing committee have footed the bill for food (I paid for the pizza for Michael Bolton’s talk expecting to be able to claim it back later but couldn’t). Sometimes we have sponsorship for food scrape through at the last minute. For example our bug bounty discussion panel at Prospa, Prospa did an awesome job of hosting us but didn’t have the budget for food and drinks. We had Bug Crowd confirm the day before the event that they were able to sponsor food and drinks. It would be nice to have some buffer of money/sponsorship so these moments aren’t as stressful. Our average meetup generally costs our sponsors anywhere from $200 to $500.

Meetup’s view of the money portal

Turn out rates

The meetup average drop out rate is around 50%, plus or minus 10% depending on things like the weather and location of the event. This can have an impact on food budgeting and moral. With a paid event the drop out rate is expected to be around the 20-30%. It makes it easier on organizers who are catering food to organize if the RSVP numbers match up to reality. We also get a better vibe. Imagine expecting to turn up to an event where 100 people RSVP’d and only 40 people turned up. You’d feel a little disappointment with the turn out. As an organizer it would suck your motivation for running future events too.

Lost funds

The previous committee had a paypal account set up and linked to Meetup but when the organizer left, the paypal account left with them and that means any previous funds we had raised (like when Michael Bolton came to Sydney and we charged $10 per person for) we actually never saw that. Which is a sucky situation to come to terms with but it’s the current situation. That’s why we haven’t charged for anything since then because we wanted to try and sort it out. This time we’ve just started from scratch with a write off for the previous balance.

Paypal setup

Paypal takes a 4.4% cut out of every transaction so for every $5 ticket sold we get $4.57. Currently I re-jigged an old business account I had set up for my tutoring services. The account has been shared with the committee so if one person leaves existing members still have access. You might see Sam’s Tutoring on your receipts until we update the details.

please don't be concerned if you see my details on the receipt
Example PayPal receipt

Free events

We will continue to run free events that benefit members, if you want to help organize any or have ideas for events that you’d like to attend hit us up. We are always open for ideas. The CV clinics we’ve been running this month came about from discussions in the pub. So please, suggest your ideas, we’d also appreciate if you helped with an event or two.

Any profits will be donated to charities

For James Bach event, Campaign Monitor has selected the drought appeal where profits will be contributed. You don’t need to worry about me profiteering from this event.

Investing in your education

Isn’t it worth spending a few dollars to support the community, your education and the drought appeal? These are just our first steps towards managing this meetup just a little better. We are always open to suggestions.

What’s with the coach titles?

Agile, Quality, Life, Career … everyone wants to be some sort coach these days.

coach sign

I don’t like the term coach. I feel like it downplays the hard work that goes into teaching. I learn and I teach what I learn. I love teaching so much I often do it for free. During uni I volunteered with Robogals teaching kids programming through Lego Robotics workshops. I had my own community radio show where I interviewed Engineers to promote the field. I give presentations at conferences to teach other people my learnings. I volunteer with Sydney Testers to teach my community how to be better at testing.

Is Coaching just asking questions?

As I teach I’ll ask coaching like questions to help you to explore ideas further. I often do this when I play the dice gamed with other testers.

Roleplay dice

If you have an idea while playing the dice game with me I’ll prob you with questions to try to get you to explain your thinking. The dice game is a fun way to explore black box testing and critical thinking, I’ll be running these on a fortnightly basis. Keep an eye out on the Sydney Testers meetup page for more events like this.

The Coaching habit book

The coaching habit” book would have us believe that coaching is all about asking questions. Just 7 questions in total. Where as I believe the main function of a sports coach is to provide the fast feedback in the moment to help an athlete improve and perform at their best. 

Coaching is having the knowledge to know when to ask probing questions, knowing what questions to ask and how to follow up on particular questions to gain more insights. I use these techniques to learn not to coach.

Mentoring vs Coaching vs Teaching

There might be different contexts where one might be applied over another but I use these terms almost interchangeable. On twitter recently I said I mentored a colleague in giving technical presentations:

And I thoroughly enjoyed the experience. I love helping other people improve. I love this stuff so much, I do it for free. The mentoring wasn’t ongoing though, it was just a 1.5 hour session with the clear goal of improving 1 thing. So maybe I shouldn’t call it mentoring? Well I view it as a mentoring session, but I also taught content and asked coaching questions. All three; Mentoring, Coaching and Teaching were all involved in this situation.

Here’s the review of that session:

Review from Tania Dastres for a technical presentations workshops

Teaching is Learning

But putting together a blog post, workshop or lecture I consolidate my own learning into key take aways. It helps me practice my communication and I learn to exchange ideas more effectively. The best teachers are constant learners.

Maybe I’m jaded because I had a bad experience where a job with the title “Quality Coach” didn’t quite work out for me but I don’t think there is much point in distinguishing the differences between teaching, coaching and mentoring. I’ll use it all to help you improve.

Personal Branding

Warning: This blog contains promotional content. Personal branding is all about that, what else did you expect?

We had a pretty awesome CV clinic for Sydney Testers yesterday held at Opus Recruitment Solutions. Barry Smith and Mike Hollowbread specialise in recruitment for software testers and went through their pointers for CVs. These guys see a lot of CVs and had built up years of experience in this area. They had tons of pointers and have put together some awesome resources with this advice:

Opus CV Pointers

This event completely booked out and the waiting list filled out to double the event capacity within 24 hours of the announcement. If you missed out don’t worry. We are doing a month of these events. Next week will be at Four Corners Recruitment. This event is already full but please join the waitlist. We had a dozen people change their RSVP at the last minute with the last one which opened up the event for half the people on the wait list. If you want some inspiration for your CV I’ve blogged about the evolution of mine before.

Have a good LinkedIn Photo

My LinkedIn Photo

My LinkedIn profile came up in Barry and Mike’s talk as an example of a good LinkedIn photo. I’d like to dig into more detail the planning that went behind that photo in this blog post. I used a professional photography service called Snappr. I paid a photographer $59 for a 2 hour photoshoot with the soul purpose of getting a good LinkedIn photo. They now charge $75 for this service. Snappr is a Sydney based startup that’s making the news for disrupting the photography industry and being successful in funding rounds. I quite enjoyed the photoshoot and I had a bit of fun with it too:

Some of my favourite photos from the photoshoot

For the photoshoot we walked around Pyrmont and the Star Casino looking for good backdrops. Maybe you can recognise a few of the locations? Some of the photos didn’t come out that great but it was still fun. You should check out Snappr’s LinkedIn Photo analyser, it’s a fun way to analyse your photo using an algorithm. What score did you get?

Hair and Makeup

The thing is, the photographer was not the only thing I paid for leading up to this shoot. It was my birthday, I decided to treat myself to a pampering. So I also got my hair and makeup professionally done. I think that cost me around $150 at CatWalk hair and beauty in Gladesville. And then there’s the outfit:

I bought the suit from the Oxford Factory Outlet in Alexandria for roughly $200. The shoes were from Paul Dane; $90. The tie was an impulse purchase from an antique suit shop in Melbourne. I think it cost me $140 but I love that tie. It’s so groovy. I didn’t purchase this outfit just for this shoot, I will wear my suit on the odd occasion. You don’t have to spend what I did on this shoot but I’m trying to highlight the planning that went into that one smiling photo.

Tailored Suits

If I was to get another photoshoot, I’d probably order a tailored suit from Luxury Plus Suits, they do a basic suit for $349. They will measure you here in Sydney and it’ll get made in China. I actually did get one of their suits but it was just before my weight loss surgery. I’ve now lost 50kg since I bought that suit so it’s safe to say it doesn’t fit me any more.

Corporate branding

There’s two testing companies that I love the branding for; Ministry of Testing and House of Test. I love the sense of Community that Ministry of Testing is working on building. They even inspire community members to get tattoos of their logo:

Rockstar Testers

House of Test literally market themselves as rock star testers. If I ever move to Europe, this is the company that I’d be begging to join. I’ll tell you a secret. There’s a small community of context driven testers out there (House of test) are one such group. Mentioning context driven testing is like secret handshake material that gets you on top of interview list with these people.

Do you want to improve your Context Driven Testing? Do the Rapid Software Testing course by James Bach at the end of August. I consider James Bach to be the original rockstar tester. If I ever saw a CV with his course on it, it would be at the top of my interview pile, no questions asked. It is some of the best training any tester worth their salt can do.

You will occasional see this secret handshake material mentioned in Job ads too; like this one for a Quality Coach role at Campaign Monitor. Campaign Monitor is an awesome company to work for; awesome people, awesome perks (free breakfast and lunch) and awesome views of Sydney.

Taking Personal Branding to the next level

Edward Zia is an example of someone in my network who has taken personal branding to the next level. He even has a caricature made up with his facial birthmark as part of his branding:

If you don’t have a personal brand, people won’t want to do business with you. What have you done with your own personal branding before? Have you ever made a bad investment?

So you want to become more technical?

Here is my 5 step guide for becoming more technical without learning to code. No matter what your background, if you follow these steps you will be more hands on technical than I was when I first graduated from Computer Science at Uni.

NOTE: If you are a software tester looking to get more technical DO NOT start with automation. Automation testing tools change as often as the seasons. These 5 steps have served me better than any automation framework has. If you are looking to get into automation, starting with some basic web developer skills could help if you want to learn UI automation via Selenium (but Javascript is hotter right now).

1. Get comfortable with the command line

This is so fundamental. I use this skill nearly every single day without even thinking about it. E.g. my morning starts out with checking out the latest code for our android app using a git command from the command line. Every other step proposed in this guide can all be done from the command line. You can even do all of your emailing from the command line, this blew my mind when I discovered this.

I’m biased towards using a linux/unix based system (Ubuntu, Android and macOS are the main ones I use). You can learn this on windows but the transition from windows to linux/unix is hard, for example the slashes for file directories are the WRONG WAY in window.

screenshot of a terminal session on macOS
You can access the command line with an application called “Terminal” on macOS. Here is an example where I list the files on my Desktop with the command LS. I can see all of the screenshots that I’ve captured for testing. I have my terminal set up in bright green font because it makes me feel more like a hardcore hacker. Want to feel like a hacker? go to http://hackertyper.com/ and start typing gibberish.

e.g. To change the directory linux/unix

cd /usr/home/some/folder

that just feels nice, natural, it’s also the same direction as the slashes in your browser address bar. Just look up ^ ^.

For windows its

CD C:\user\some\folder

GARRH I feel like pulling out my hair, why do you do this to me windows?

If you want to explore linux/unix commands on windows you can:

Let me tell you a secret, I didn’t learn this skill until I was 2 years into my testing career when I started working at Tyro and every developer computer was running Ubuntu. The first thing I learned was how to CAT logs for a test environment. I studied computer science, you’d think I’d learn this skill there but nope I breezed through uni without ever learning how to change the directory from the command line.

You don’t need to memorize all of the linux commands but learning how to change a directory, make a file, move that file, edit a file and search a file (probably by using a grep pipeline command) is all you need to get started. Learning how to read a help file to understand all of those pesky characters will also help. I found help files intimidating. Heck I still find them intimidating and I prefer to google an example than to try to make sense of a help file.

2. Learn Git

Now you don’t need to know how to code or have a coding project idea to learn this. For Example, you can just put together a wiki of your favorite resources. Like my amazing colleague Paul Maxwell Walters who put together this list of testing learning resources. You could easily learn how to add a bunch of Testers twitter handles to that project, Paul would appreciate the help. Here are some resources for learning git. You just need to learn a few basic git commands (PULL, CHECKOUT, COMMIT, BRANCH are the ones I use 80% of the time),

Here is another secret; I don’t really understand git, I just know enough in how to use it. If I get stuck I always fall back onto a git hard reset to head but I always have to google the structure of that command. Stack overflow is your friend.

3. Learn a query language

Learning how to construct SQL queries was one of the first technical skills I picked up on the job and I have used it in every single role since. Again I didn’t learn this at uni. It was hands on experience that taught me this. MySQL is a common query language (it’s open source too). You can also construct MySQL statements from the command line. E.G. I’m working on a poo diary app to teach myself mobile development and google cloud. I have a MySQL test database set up and when I query the database from the command line this is what I get back:

mysql query example

Now you don’t need to learn everything, just learning how to do a simple SELECT WHERE {Conditions} will get you far. I use this type of query 80% of the time. Maybe learning how to update an entry and how to do a query across multiple tables will also help. You don’t need to be a database master. Here is a follow up blog post on this topic.

4. Learn how the internet works

I’m talking about learning about API’s, REST requests and JSON payloads. You should learn how network requests work; how servers talk to each other using the internet. Again you can learn this by using the command line and googleing a few resources. If you are exploring a new API, learning how to construct a simple GET request using a CURL command will get you far. Learning how to do a GET, POST or a DELETE request will be 80% of any API testing tool out there (e.g. Swagger, restlet or postman).

Learning about how requests work will naturally open up the door to using performance testing tools or security testing tools. Learning this is a good foundation for more technical testing practices. You can also use developer debugging tools for browsers; Firefox & Chrome to monitor network traffic or to explore API’s. If you can mostly understand what this stack overflow question is about then you already have a good foundation here. Joe Colantonio has an awesome resource which has a section for learning about API’s right up the top.

5. Start a learning project

I often start new projects with the focus on learning. I wanted to learn some arduino skills, so I researched open source robots and built a tapster robot which can be used for mobile testing:

Now, it’s nearly impossible to find a practical use case for Tappy McTapFace, but building this robot has helped me learn a great deal and I’ve been able to demo this robot all over the world. That’s pretty cool.

My next project is a poo diary app. I want to learn how to do mobile app development, how to integrate mobile apps with google cloud/firebase and how to build an API in Java that talks to a MySQL database backend. The app itself is a little silly (some would say it’s crap – pun intended) but I’m using it as a learning tool. I’m mostly just following tutorials on the internet and it’s glorified copy paste work with a little bit of thinking. However with the 4 foundations listed here and studying Java at uni it doesn’t seem all that hard.

Do you consider yourself to be technical? What’s your next step in improving your technical skills?

My Approach to Testing

Someone asked me, “do I have a method for generating testing ideas?”. My answer; “I follow an exploratory approach to simultaneously learn, generate and execute test ideas”. This might involve exploring the product a little to learn about how it works, this will feed into questions/ideas and then I might explore some other ideas further.

I might use sources like the Design Sketch as an oracle (source of truth) for inspiration/ideas, or my mobile testing experience (heuristics/rules of thumb like “accessibility always sucks and its easy to find bugs there” or “screen rotations often causes bugs”), or things like boundaries to explore. It depends on the context of what I’m testing to what ideas get generated.

Exploratory Testing

Exploratory testing is a fairly efficient approach to finding bugs. It’s not like we know where the bugs are in the product before hand. However there are always gaps in anyone’s testing approach. It would take an almost infinite amount of time to test all of the possible permutations. So testers will also use their risk radars to help hone in on testing high risk areas.

Mareet Pyhäjärvi has an interesting keynote for SeleniumConf on the intersection of exploratory testing and automation:

And she also has a Medium article explaining Exploratory Testing. Here is the sketch notes for her talk:

James Bach has a 10 page article on exploratory testing but it can be a little dry to read all in one go. I’ll also often reference Elisabeth Hendrickson’s book, “Explore it! Reduce Risk and Increase Confidence with Exploratory Testing“. Elisabeth has also put together a cheat sheet for testing heuristics but I feel like that is more suited to testing a web frontend context.

Mindmaps

I often use mindmaps to help generate ideas, e.g. when I was at Tyro I created this mind map for generating mobile testing ideas, the idea being I’d bring this along to planning meanings and it would prompt me to ask questions like, “what about performance testing?”.

Everyone does testing on some levels, some people are just more practiced at the skills involved than others. For me, testing is my craft.

Getting up on stage

I enjoy performing. Don’t ask me why. I can’t explain it. You could say it’s something to do with the rush, or the perception of adding value or entertainment for other people. I want to tell you a few stories about my adventures in performing. Do you want to improve your performances? I’m available for free consultations on improving technical presentations.

During High School

I was involved with nearly every extra curricular activity I could sign up for. I was in the school concert band; I played trombone. I can legitimately say, “this one time in band camp …”. I was the fat kid in school, there weren’t many other kids fatter than I. I once got up in front of my whole school dressed up in a Santa suit and played Jingle Bells on the trombone. Talk about a nerve racking, getting out of my comfort zone experience. I got a laugh at least. I was in an Auslan signing choir (Australian Sign Language) and a singing choir too. In the signing choir we would often perform to retirement homes in the area and our signature song was, “I believe I can fly” by R Kelly. I could still sign to that song. What does a signing choir performance look like? Check out this example on YouTube;

Watching that makes me want to sign up to an auslan class and pursue deaf poetry.

I was also in a musical. It was called Wolfstock, it was a 1950’s themed musical about a 16 year old boy called Jay, his parents had sold his soul to the devil and had to get to wolfstock (aka woodstock) before the next full moon or else he would remain a warewolf. I played Wolfman Jack in act 2; a character based on the DJ host by the same name, I even had my own song. I’m sure the musical was terrible. My mum has it on tape somewhere. I’m sorry mum for putting you through all of my horrible performances in school.

During Uni

I ran my own radio show on a community radio station called, “chat with an engineer”. I would interview engineers in our community and chat about the work they did. It was to help raise the profile of Engineering. I didn’t have the budget for the training course so I asked Engineers Australia if they’d paid for me to do the course. They did and I’m forever grateful for that. My biggest success was interviewing 2012’s Young Australian of the Year; Marita Cheng. She was visiting a high school as part of a Robogals visit and we were able to organise an interview.

I also started the Robogals Chapter in Tasmania. Robogals is a student run group who promote engineering and technology to young kids through lego robotics workshops with the goal of increasing female engineers. I taught robotics to over 1000 kids in tasmania in the 1.5 years I was involved with Robogals with next to no funding and while going through my first bout of chronic depression. I can’t understand how I was functioning, I wasn’t passing uni so let’s just say I wasn’t functioning very well. Teaching is another type of performance that I enjoy.

Professional Presentations

During my professional career the main performances I’ve been involved with are presentations. My most nerve wracking experience was getting up in front of the whole company during an all hands and talking about my struggles with depression. Getting that venerable in front of such a large crowd is another one of those big, “getting out of my comfort zone” experiences. It’s definitely made giving technical presentations easier. Interviews are another performance. A lot of people hate interviews, in a weird way I enjoy them. Having that opportunity to talk about my passions in software testing is what I enjoy. I am narcissistic. I remember doing a first year psychology 101 personality test during uni, I scored very highly on the narcissistic scale and I’m ok with that. It’s only an issue when it’s combined with a lack of empathy.

I’ve been involved with a few community bands since moving to Sydney. The Sydney homotones and Sunday Assembly being the main ones. I’m not actively involved with any now but I would love to join a community swing band. Or do some taiko drum classes. Or learn how to play the double bass. Garhhh, I can’t decide.

My favourite presentation has been my talk at YOW! Connected last year on using robots for mobile testing;

I was able to combine my passion for music, robots and mobile testing. #Winning at life.

I’ve collected a bunch of hints and tips on giving presentations. Reach out to me at sam[AT]thebughunter.com.au if you’d like a free consultation.

Evolution of my CV

My CV has evolved a bit over the years. But there have also been a few constants too. I’ve always tried to keep my CV under 2 pages, my most recent one is 1 page with embedded links, it’s more like a portfolio.

Sam Connelly Tester Profile 2014

I used this CV to apply for my role at Tyro, it was created using Word and it’s interesting to read over how I presented myself 4 years ago. This was my first attempt at putting the 5 c’s of testing on my profile.

Sam Connelly Tester Profile 2016

I started experimenting with the 1 page CV, using Canva (an online designer’s tool). I was told somewhere that dates didn’t matter as much as duration. I used this CV to get my role at EPAM Systems. I still have the 5 ‘c of testing but I’ve replaced crazy with coding.

Sam Connelly Tester Profile 2017

This is a 2 page CV that I used to help me get my role at Campaign Monitor. It went through a few iterations and I had a few other ideas in how to tweak/and experiment with it. I also wanted to experiment with a timeline representation;

timeline view of my CV

I never submitted any CV’s with the timeline view but it was an interesting idea.

Bug Hunter Sam Connelly 2018

This is my latest iteration, it’s a 1 pager. Created using Google Docs. I’ve dropped the 5c’s of testing idea. If I was to keep it, I would mention it but link out to a blog post about it. I’ve also adjusted the date I graduated from Uni. Because of my exchange year, changing course and bout of chronic depression my uni was all over the place. I had done most of my course by 2012 and was only doing 1 subject a semester in 2013, I was doing a full time testing contract at the time and had done a 6 month part time testing job before that. I recently had someone reject my 2017 CV because the recruiter did this quick check; 2017 minus graduation year of 2013 only equals 4 years of experience = rejection, we are looking for 5+ years experience. So instead of trying to explain this I’ve just put 2012 as my graduation year.

The whole point of a CV is to make you seem like an interesting enough person to invite into for an interview. Think of it as a user interface with that whole purpose in mind. How can you simplify the data and layout to make it easier to read? How has your CV evolved over time?

Back on the job market

I find myself back on the job market after a break up with Campaign Monitor. I didn’t successfully pass probation. It was a mutual thing and both sides of the discussion were adult about it. These aren’t easy conversations to have and it doesn’t serve any purpose to get angry and rage quit. I am a little sad to leave because I enjoyed the company and people but I wasn’t able to advocate for quality in a way that added the business value they needed from the role.

Depression and Job Hunting

When I was job hunting around 8-9 months ago, it took me well over 2 months to find a job and interviews with over 13 companies (blog). However in that situation I wasn’t in a rush and was willing to wait for something that looked like it would fit me well. The constant rejections were hard to deal with; especially when I had been experiencing a spell of imposter phenomenon and feeling like I was not good enough for anything. I also broke my ankle during these job hunting efforts which had a huge impact on my mental well being (blog).

My broken ankle contributed to a relapse of depression at the start of the year. Because of this I wasn’t able to give my 100% to the new job at Campaign Monitor and this negatively impacted the engineering’s team view of the Quality Coach role. Once your perception of value is seen in a non favorable light, it is very challenging to recover. You only get one shot at leaving a first impression and your reputation is built up on that. I didn’t do a great job when I started, then I tried a new team and a new process and saw some improvements. However there were still some doubts if this role was what the company needed and if it was the right fit for my skills. I went to a third team for the last 3 weeks but I feel like the decision had already been made by that point.

Keeping Track of Job Hunting

I used a spreadsheet in my previous job hunting efforts to help me keep track of where I was up to with every company;

spreadsheet of previous job hunting efforts

With this spreadsheet I noted the source of  the lead; I was relying on mostly LinkedIn and a technical recruiter from Opus. I noted down where I was up to in the interview process, excitement for the role (out of 5) and any follow up notes. I also noted the few companies who contact me after I had received a successful job offer.

Will I do the same thing this time? I’m not too sure. I’ve got the luxury of around 2-3 weeks for job hunting before the personal budget starts getting a little tight but it would be worth experimenting with the spreadsheet again this time.

What am I looking for in a new job?

This time round I have more confidence in my skills as a tester. Last time I wanted to quit testing and try something different (either Android Development or Product Manager). However now I know I love growing my reputation for being known as a passionate tester. In a few years time I’d love to be running my own company focusing on running workshops for technical testing and mobile apps (e.g. TDD and kotlin, Continuous Integration and iOS). I’m not there yet so I’m looking for a mobile app testing role while I work on workshops in my spare time.

I’d love to have a role with support for speaking at conferences. I’m speaking at Agile Australia on how to get more people involved with testing in a few weeks, Selenium Conf in India at the end of June on using robots for mobile app testing, CAST in Florida in August on stories in becoming a quality coach. I now have an anecdote where that didn’t work out for me so that should be an interesting talk.

Why bother testing?

As a tester, I don’t practice selling my craft very often. This blog post is an attempt to put together my thoughts when asked the question, “why bother testing?”

Let’s talk about bugs

When was the last time you used some buggy software? How did it make you feel? Did it cause you to swear at your computer in frustration? We are surrounded by software full of bugs, some bugs are minor but some cause us to pull out our hair in frustration. Some bugs when discovered cause nightmare headlines to spread like wild fire. Buggy low quality software is hard to sell, maintain and keep people using. Your customers won’t rave about your products to their friends if they think it’s shit.

Your customers won't rave about your products to their friends if they think it's shit

Testing doesn’t improve quality

Testing itself doesn’t improve the quality of the product, but it can help highlight issues that could impact the perception of quality. Bugs will always exist in software, it’s impossible to test for every possible scenario every time before releasing to production (especially in a world of continuous integration/deployment) but if you knew about some the bugs in you products before your customers find them then you’d be in a better place to make informed decisions about what to release and when.

Testing is active discovery

Everyone does testing on some level, most of the time we are unaware of what we are doing until we discover a frustrating situation. Testing is a skill where you practice looking for quirks in software. Testers are constantly experimenting and observing the product and are well practiced in talking through their thought processes.

There are generally 2 activities people say they are doing when they talk about testing;

1 – Verify the product works as intended

2 – Actively go hunting for bugs

Activity 1 is usually where people talk about automation, when your product can be codified as “working as intended” you might be in a place to build some automation checking to help facilitate faster feedback. However you could build all of the automation checking in the world into your products but people can still think it’s shit. You users don’t care that your unit tests are less than 0.01% flakey.

Activity 2 can be easily practiced, it’s not like anyone knows where the bugs are in software until they are discovered. You can build things into your products that make it easier for people to report the bugs they find or you could do a chaos monkey approach to find out where your product crashes unexpectedly.

There are many other activities involved with testing but I want to keep this high level.

People say testing is risk mitigation

This is not true, testers may use their own internal risk radars to help guide their testing efforts but testing itself doesn’t mitigate any risks. I like the analogy; testing is like an x-ray, an x-ray on it’s own can’t tell you how healthy you are or prevent issues from happening but it’s a tool that can give you a snapshot in time so your doctor can make informed decisions about some element of your health.

Testing can help people answer questions like, “are we comfortable shipping this code to customers?”. Bugs are a fact of life but we can’t fix what we don’t know. Testing helps us discover these potential issues.

What is software testing?

A quick google of ‘What is software testing’ will give you 75,300,000 results but not much of an answer. On the surface level, we can surmise that it is about testing software but in reality; it’s more than that.

Software testing is an activity that everyone does on some level. E.g. when you interact with a product like Facebook you are helping to test it. You have certain expectations of how the scrolling behaves and what type of content comes up in your feed. You are probably a part of several on going experiments that Facebook conducts and you’re scrolling habits are also helping shape their performance testing approach.

Q. Hang on, if everyone does it, why do we need dedicated software testers?

A. Justifying the value I add to a project can be challenging sometimes but let me try to go into more detail. There are some very successful companies out there that don’t hire dedicated testers but their context and software development practices can help mitigate some of the risks associated with their products.

Software testing is about reducing the risks associated with software development. Software development is, at its core, a business venture. The software is created to solve an issue that the end user is willing to pay money for. Each element within the development lifecycle is working towards providing a marketable solution to that original problem. Software testing serves to highlight any risks that might impact on the value of the product.

The software development cycle is a continuous iteration of identifying the customer’s problems, developing the solution, ensuring the solution works, and selling the solution. Software testing ensures that our apps, websites and software does what it’s supposed to do whilst not doing what it shouldn’t. It is an integral part of any successful software project.

Planning

Good testing can be completed with very little “formal” planning procedures if a skilled tester is involved. You can see an example test report generated from 1 day of testing from James Bach for testing a medical device.

Often the waterfall model of software development is associated with exhaustive requirement gathering and testing planning even before the software is designed. On the contrary, the agile manifesto focuses on providing working software over comprehensive documentation. The whole Agile vs Waterfall is a conversation for another blog.

When working within the agile methodology we can move away from explicit and highly specific planning and into the rapid software testing realm. James Bach and Michael Bolton are pioneers in the Rapid Software Testing space. This is a context-driven software testing methodology that works on the premise that the testing team will have an understanding of the business problem that the product is solving.

This Rapid Software Testing method allows the testing team to conduct a more informal, risk-based approach to planning the testing of software. Sam explains how to take a risked-based approach to planning UI tests in her /dev/world presentation.

Exploratory Testing

Exploratory testing* is where a tester interacts with software to compare it with their mental models of expected outcomes.

Exploratory testing is a part of all testing regimes because it focuses on human-based feedback. Having people run through the software with intention of finding brokenness helps to highlight any design, UI or experience elements that are confusing or hard to use. This is something that automated testing scripts aren’t able to handle.

Exploratory testers will explore how well the software solves the end-users problem. They will highlight any high-risk issues that will impact the value of the product. Skilled exploratory testers will work on a continuously iterative testing approach where the exploration can help form a plan that guides the direction of future testing efforts. This continuous feedback loop allows testers to improve the quality of their testing rather than just the quantity.

Exploratory testing is based on human observations. This is incredibly useful for user experience focussed testing but less practical for testing that needs to be repeated over time or scaled up to assess performance.

Whilst the benefits of having human based feedback far outweigh the cons it is important to note that human error will always occur. Hopefully using a risked-based approach with an experienced exploratory tester will negate some of these issues before your users find them.

Automated testing

Automated testing (sometimes called checking) involves using software tools or scripts to compare the expected results for how a program should run with actual outcomes. It leverages the computational power available today to automate testing that would have to have previously been conducted manually.

definition of automated testing; involves using tools or scripts to help asses expected behaviour of software

For example, an automated testing suite may be used to check that ‘a + b = c’. To test this we could run a sample of known numbers through the function and ensure that the outcomes were as expected.sudo code for 3 unit tests

This is an extremely simplified example of automated testing but this idea can be expanded to include all of the predefined actions within a software.

Automated testing has numerous benefits including increasing the amount of test coverage available within a budget or timeframe, automating basic or low-level tasks, reducing human error when checking outcomes, reusable tests and leveraging the computational power of machines.

If automated testing suites aren’t maintained in line with the development cycle it can lead to outdated tests that return incorrect outcomes. This can lead to less secure testing coverage and a loss of team morale and trust in the testing cycle.  Additionally, automating poorly designed tests only increases the rate that these misaligned tests are run.

Automated testing has been hailed as the holy grail of the software development world of late. It fits in well with the continuous integration methodology and allows for quick iterations of products. However, it’s important to keep in mind that automated testing isn’t a magic bullet. Instead, automated testing should be used in conjunction with other tools to gain more time for your testers to focus on high-value tasks that are outside the scope of automation.

Unit Testing

Unit testing is about isolating the components in your software and testing them individually.

definition of unit testing, isolating components to test individually

Unit testing is a powerful tool when used early in the development cycle. This is a tool often employed by developers to ensure that the code they’re writing works.

Like all of the tools that we have explored today unit testing in only one option that a development team will employ during as part of their testing process.

Conclusion

Anyone doing testing has a plethora of tools available to them when conducting a thorough testing cycle. Often we get caught up in the discussion of Exploratory Testing vs Automated Testing but we need to bring this back to what is the tool that will best fit this testing scenario. I’ve only scratched the surface here of what can be involved with software testing.

Software testing is a continuous feedback loop. Everyone does testing on some level, and one of the goals of testing is to highlight any issues or risks that might impact the perceived value of the product. Ultimately we want to help deliver quality products on time and on budget.


*The intricacies about whether exploratory testing should be referred to as ‘testing’ has been discussed extensively within the software testing industry. Please note: for ease of understanding Sam will be referring to all testing without scripting as ‘exploratory testing’ within this article.

**Sam commissioned Stefanie C. from upwork to help write this article, there might be a follow up blog on Sam’s experiences using freelancers to help outsource work later and why she would do this