Creating test scenarios is as essential as developing code in the software development process. Test scenarios running in the pipeline allow us to detect potential bugs that may arise during development before they are delivered to the merchant. The test methods we apply vary depending on the structures of the projects and the flows they contain.
Manual Testing vs. Automated Testing
Manual testing is a method where test scenarios are run manually. In contrast, automated testing is a method that allows scenarios to be run on the test code as soon as the project is completed. Both approaches have their advantages and disadvantages. For example, with manual testing, we run the same scenarios manually for each development, while with automated testing, we eliminate this workload. At Craftgate, we prefer that our test processes run as automated as possible. This way, we feel confident in new feature developments and refactoring processes for existing structures.
What is End to End (E2E) Testing?
At Craftgate, we offer a high level of technology in the scope of the services throughout our product. Within this system, we apply practices such as unit testing and integration testing for developments. However, in addition to the individual behaviors of applications or services, we also find it essential to test the responses we direct to our customers due to the interaction between all of this orchestration. At this point, we test the features we provide to our customers from an end-to-end perspective.
These tests ensure that successful and unsuccessful scenarios flow throughout the system via input points from our integration points. Ultimately, we verify whether the resulting values align with our expectations. In these tests, we prioritize not only the responses that will be returned as a result of successful scenarios but also the contents of the errors that may occur due to unsuccessful scenarios.
Our goal in these tests is to automate the scenarios that end-users may encounter to be run in the pipeline. In this way:
- We minimize the risk of human error in the testing process.
- We reduce the time and cost of the testing and development processes.
- We ensure that the system works coherently from end to end, providing the correct results and data integrity.
How Did We Use It?
End-to-end testing methods can be used in many flows for payment domains. For example, it is possible to scenarioize almost every process, including services such as saving a card, making a payment, and installing inquiries that we offer our merchants with the end-to-end test method. There are many tools that can be used for end-to-end test development, and our choice was Cucumber.
How Should Scenarios Be Designed?
- Test scenarios should be designed from the end-user’s perspective.
- Both successful and unsuccessful scenarios should be included in the test scope.
- This way, our chances of catching errors that users may encounter before they experience them will increase.
Benefits of End-to-End Testing
- End-to-end tests can create scenarios that read like conversations due to their syntax. When writing these tests, it is crucial to focus on creating test scenarios and determining their scope rather than algorithm knowledge.
- Some tests are manually repeated after each development. With end-to-end tests, the standard test processes that QA will perform as a result of each development are automated, and significant speed is gained in the testing processes.
- End-to-end tests run continuously in the pipeline, enabling the possibility to detect a bug that may arise due to development early when defined. Work periods can be determined before each commit, before each PR, or in daily-weekly-hourly intervals.
- With a single test, we include a wide scope in the testing process. While unit or integration tests verify the output of a single application or service, end-to-end tests verify the result to be delivered to the customer as a chain of interconnected applications.
- It reduces development time and cost.
