In software development, we are hired to deliver value to the customers. Usually that value comes in the form of running software that does something useful. The end user really doesn't care about tests.
So why write Automated Tests then?
Well, we want to make sure we don't break something for our users. All of us already do some form of testing, and that's in the form of clicking around and playing with the UI.
Sometimes that works, but there are two important problems with it.
The first one is consistency. When you're clicking around, are you always going to check the stuff that was working before? As the application gets more complex, there are more and more things that can break, and you're not going to want to spend the time to click through them thoroughly every time you add a feature.
The second problem is that developers are way too nice to their own system! When we play around with the application, we're subconsciously not giving it bad or malicious input, we're waiting for asychronous processes to finish, we don't pull the power plug in the middle of a save operation. All those things will happen in production!
By writing automated tests, we are having a robot diligently do all the tedious work for us, and we can consciously code in edge cases to avoid having our developer "blinders" on.
Tests as Communication
There are other benefits to automated testing that manual testing can't even approximate. Some of these may surprise you.
Tests demonstrate features. High level tests can actually show customers what has been built, and demonstrate that it works.
Tests help other developers understand your architecture. Tests are a form of documentation - they provide working examples that others can refer to, and show how the pieces fit together. In fact, tests can be better documentation than comments, because comments might be out-of-date or just flat out wrong!
Tests help you come up with, and validate, good code design. If your code is hard to use in tests, it's probably hard to use outside of tests too. As you write your code, you can get feedback on how your libraries and modules fit together.
As you can see, it's not all about preventing bugs. Tests are a way of evaluating your work - it helps avoid bugs, but also tests your internal code quality by running it through different scenarios.
What kinds of tests can we write?
There are three common categories that automated tests fit into:
- Unit Tests: tiny, fast, and precise
- Integration Tests: medium sized, less precise and slower
- End-to-End tests: large and slow, but cover a lot
Here's a diagram, showing all three and their relative scopes.
Unit Tests tiny, fast, precise tests
Unit tests are the most fine-grained type of test there is, it's where you test a single "unit". A "unit" in terms of a unit test is the smallest chunk of code that has some behavior, usually a single function or class.
Have lots of them
You want to have a lot of unit tests compared to the other types.
In any given codebase, it's not unrealistic to expect about 50% of your code to be unit test code. And it makes sense, because you have a roughly 1:1 correspondence between a function and the test for it.
Run them Frequently
Unit tests are also the ones you want to run all the time. Some people set a hook in their IDE so when they hit Ctrl+S or ⌘+S it runs all the unit tests. That way they can get instant feedback if something is broken.
Because you have a lot of them, and you want to run them all the time, they need to be really fast.
Quick iteration cycle
When you do this, unit tests are great for instant feedback. If you broke a unit test, you'll know right away. Bugs are easier to fix when you find out about them a couple seconds after creation, because you're already looking right at the code that has the problem, and reasons why it might be wrong are already in your head.
Integration Tests medium sized, less precise and slower tests
Integration tests where you take several chunks of code that talk to each other and test them together. One example might be testing that your Database queries work, or that your billing system can talk to the shipping system.
Not quite so fast
Integration tests have a lot more code executing and interacting, so they will necessarily run slower than unit tests. So what's the upside to running more code?
Better verification
Integration tests verify that the interactions between classes work. Unit tests, by definition, only test a certain class in isolation. They do nothing to verify that the classes were called correctly to begin with. Integration tests allow for multiple classes to interact with each other, so it will catch issues that are in the communication between classes.
End to End Tests large and slow tests, but cover a lot
The End-to-End tests are at the highest level - it's the kind of test that starts up the whole system and does something that a user might do. A good example might be logging into the website and making a test bank transaction, then verifying it went through properly.
Slow...
Usually end-to-end tests are very slow... they have to bring up the whole application, including usually a database, a server, and another server to interact with the application in an automated way. Usually the database is a brand new one, just for this test, because we don't want to pollute our real database with test data.
Because of this, we are only going to have a few of these tests. If we had thousands of end to end tests, it would take days or weeks to run! And tests that go a long time without running tend to break more frequently, and are more annoying to fix, because the developer who broke it has already moved on to the next task.
User Like
However, the major positive of end to end tests is that it behaves just like a user. If the end to end test doesn't fail, it's likely that the user is going to have a good experience too. Unit tests, and even integration tests don't verify whether a button is visible on the screen - the end to end test can catch this sort of thing.
Regressions
End to end tests are also great at catching regressions in the code. Any time we make a change or add a feature, we risk breaking some existing feature accidentally. Since end to end tests run through so much code all at once, they are very likely to trip over any bugs that were introduced by new features.
Fin
That's it! Now you know why we write automated tests, and what kinds of tests there are to write.
In the upcoming posts, you'll actually apply this knowledge to write some Unit Tests.