Every piece of software is unique: different tech stacks, varying coding styles, wide arrays of use cases, and distinct ways in which the software itself will be exercised. Due to this, testing methodologies, be it manual or automated, are also unique.

While this certainly keeps our jobs fresh and exciting, it presents a problem that is left to the expertise of the developers and QA engineers. A plethora of questions need to be answered before we begin designing an automation approach; including:

• What approach should we follow in selecting an automation framework?

• Which CI/CD tools should be used for accelerating automation testing?

• What should be the frequency at which automated tests need to run?

Automation engineers must maintain a robust tool belt in order to execute the respective automation tasks effectively. However, given the myriad of CI/CD tools and frameworks to choose from, it can be challenging to keep our skills up-to-date on top of our day-to-day activities at work.

After working on a variety of applications in completely different industries (online gaming, financial services, advertising, travel, video streaming, eCommerce, and currently commercial insurance), I’ve identified working patterns and necessary components to succeed in QA and test automation in these domains.

If you work in the eCommerce industry, I suggest bookmarking this article as the tips mentioned below can help you in your day-to-day work.

There are intricacies in every domain that, if known, can drastically improve an automation engineer’s efficacy on a team. Foundational knowledge of QA and test automation best practices paired with a specialized, industry-specific toolset will have a significant positive impact on a team.

US eCommerce sales grew more than 30% in 2020, which is projecting a strong shift in buying patterns of consumers. As the world shifts from in-person shopping to online shopping, eCommerce giants like Target, Amazon, Walmart, and Best Buy are increasing their engineering efforts to enhance their platforms further.

With this meteoric shift comes an increased need for strong QA practices since these businesses are now seeing a larger share of their revenue come from online fulfillment.

Here are QA & test automation best practices that are derived from my QA experience with companies like Best Buy.

Utilize data-driven testing

When you hear the term data-driven in the context of testing, it commonly means setting up test scenarios so that the same scenario is run ‘X’ number of times, where X is the amount of data permutations that exist for the given scenario. For example: Verify that a customer can add X products to the shopping cart.

As an automation engineer, I may opt to implement automation testing using BDD. TDD vs BDD is a quest for some developers. The choice of an appropriate BDD framework can depend on the team’s prior experience with BDD tools. I personally prefer the Cucumber test framework since it is a popular, easy to use framework. For the test scenario mentioned above, I could do something like this (simplified example):

Scenario Outline: Add to Cart
  Given a logged in user
  When I add <items> to my cart
  Then I should see <items> in my cart

  Examples:
    | items |
    |     1 |
    |     5 | 
    |    25 |

This allows me to easily add thorough test coverage for common workflows in the application. For example, I would likely use an API endpoint to add items to my cart, then call another endpoint to get all items in my cart once they’ve been added. These tests could also be run through the UI, resulting in lengthy runtimes and more test flakiness.

Let’s look at my second definition of data-driven testing: obtaining real, production user data and statistics to drive test scenario design. In the example above, I arbitrarily chose three different test scenarios: adding a single item to the shopping cart, adding 5 items, and adding 25 items. Test scenarios should, ideally, never be arbitrarily chosen!

When I join a team, I always ask for production reports — this, in my opinion, is some of the most valuable data you can leverage for your test suite. Based on this information, you can set up test scenarios and have greater confidence that your most common use cases are working as expected.

This data is a goldmine that can be extensively used for setting up the baseline test cases. Once the data is available to the entire team, they have visibility into the most critical pieces of the application’s functionality. The type of production data you use depends on the system under test, but here are a few examples of the most prominent data used in the testing of e-commerce websites:

• Common types of order fulfillment — delivery, in-store pickup, curbside pickup, locker pickup

• Common size of orders — single item, 2–10 items, 10+ items, bulk orders

• Insights into the most commonly ordered items

Depending on the user metrics derived from the above data, we can create our baseline test scenarios, accounting for a large amount of our application’s real use cases.

*Read *—12 Mistakes In An E-commerce Website That Affect Online Sales

Leverage Cloud-based testing solutions

A majority of eCommerce sites can be broken into the following core functionalities:

• Product search

• Addition of product to shopping cart

• Order summary and modification

• Check out and payments

It is common for separate development teams to own these features, which unfortunately creates silos within the organization. Intertwining different pieces of the system with each other is one of the major uphill tasks when it comes to eCommerce development and testing.

If you are working on product pages, you’re reliant on the search development teams and the inventory development teams. In distributed eCommerce platforms, visibility into the state of applications in the test system will save teams a significant amount of time during testing activities.

With cloud-based automated cross browser testing solutions like LambdaTest, teams have increased visibility into the state of upstream and downstream apps in the test system. It enables testing of the different site functionalities across browsers and platforms at a much faster pace! It leverages the cloud to its maximum potential.

LambdaTest Automation Dashboard

Another great benefit of cloud-based tools is the transparency into other teams’ test suites. eCommerce organizations are large and require cross-team collaboration to ensure company-wide success. Tools like LambdaTest enable teams to have transparent testing efforts, enabling teams to understand the rules and requirements of applications outside of their own.

You can also refer to the below video tutorial on how to perform real-time browser testing on LambdaTest.

Check this out: How To Debug Websites Using Developer Tools for safari

Think like a user, but test like An Engineer

It’s likely that you are working with highly distributed and complex systems when working in the eCommerce industry. As a result, if you were to automate your tests to mimic a real user’s path through the system, your tests would be huge, slow, and prone to failures caused by upstream dependencies.

If your system allows it, utilize writing directly to a database or posting to a web service endpoint to eliminate time-consuming test data setup.

An example of where this can be useful:

Testing Cart functionality: My high-level suggestion here would be to do thorough API automation testing. Here are some of the relevant endpoints:

• POST: /addToCart/{sku} → {sku} is associated to an item and this call adds the item to the user’s cart

• GET: /cart → All items currently in the user’s cart are returned

• DELETE: /removeFromCart/{sku} → {sku} is removed from the cart

Your test cases can quickly add items to the cart with the POST call, retrieve all items in the cart by calling the /cart endpoint, then use the DELETE call to clean up the test. Using the earlier example, a test like this would probably run in a matter of seconds. However, the test could take several minutes and be vulnerable to upstream dependencies in the Search application and the Item Details application. This is especially true if you handled the data setup and test execution at the UI level.

This approach to test data setup can be used within your UI tests and functional areas in an eCommerce platform.

Check this out: Cross Browser Testing Cloud- Browser & app testing cloud to perform both exploratory and automated testing web across 3000+ different browsers, real devices and operating systems.

Leverage feature flags and isolate new features in production

eCommerce platforms require 24/7 uptime and availability. In Agile software development, frequent releases are recommended to reduce the overall risk introduced by not allowing changes to pile up over long periods of time. I would recommend embracing shift left testing for eCommerce website testing so that development and testing can move hand-in-hand.

Simon

@simon360

That was the most effortless Apple pre-order I’ve ever done - the site launched on time and didn’t crash. 30 seconds from opening the app to checking out with Apple Pay. Enormous kudos to Apple - ecommerce is hard, but seems like they’ve gotten their heads around it.

07:05 AM - 14 Sep 2018

While it’s important to have either a roll-back or fall-forward strategy in case an issue is introduced in production, there are ways to isolate features so that a new piece of functionality can also be tested in production before rolling it out to the entire system of users.

Here are some of the ways in which you can leverage flags in testing:

• Enabled/disabled feature flags: Akin to turning a lightswitch on and off, these flags are useful for enabling and disabling features for short periods of time. This can be used for production testing during a low-traffic time period.

• User-based flags: You can really narrow down the visibility of a feature in production using these flags.

In general, don’t be afraid to test in production (safely!), if it is possible to do so. Features can be easily isolated to your own production user account (and/or others on your team) to give you higher confidence that the application is functioning as expected in a true production environment. This is particularly useful in eCommerce, as it is hard to end-to-end test the full user flow in test environments considering the expansive size of the e-commerce platform.

Read — Top Automation Testing Trends To Look Out In 2021

Run Selenium, Cypress, and Appium tests on LambdaTest to scale with the demand of your website and web apps.

Check this out: How To Debug Websites Using Dev tools in Safari

Wrapping It Up

While major eCommerce platforms — Target, Walmart, Amazon, Best Buy, and more — are competitors trying to differentiate from one another, remember that their systems are generally the same at a high level. A customer searches for a product, adds it to their cart, checks out, and can view/modify their order.

Of course, underlying systems that power this simple-sounding user flow are large and significantly complex inventory systems, supply chain systems, integrations with retail POS systems, and integrations with marketplaces like eBay. Thankfully, the testing we can do at the database, web service, and UI application layers cover the user-facing functionality that customers interact with daily.