Reverse Engineering - understanding Spies in Testing

Chris Noring - Jul 29 '19 - - Dev Community

Reverse Engineering - understanding Spies in Testing

Follow me on Twitter, happy to take your suggestions on topics or improvements /Chris

We use a spy to not only mock a response from a dependency but to ensure that our dependency has been correctly called. With correct we mean we mean the number of times, the correct type and number of arguments. There is a lot we can verify to ensure our code behaves correctly. This exercise is about understanding Spies in Jasmine, what goes on under the hood

In this article we are looking to explain:

  • WHY, Understand WHY we use Spies and what they are good far
  • WHAT, Explain what Spies can do for us
  • HOW, uncover how they must be working under the hood but attempt to reverse engineering their public API

TLDR If you just want to see the implementation and don't care for reading how we got there then scroll to the bottom where the full code is. :)

 Why Spies

Let's set the scene. We have a business-critical function in which we want to ship an order to a user. The application is written in Node.js, that is JavaScript on the backend.

It's imperative that we get paid before shipping the order. Any changes to this code should be caught by our spy that we are about to implement.

The code looks like this:

async function makeOrder(
  paymentService, 
  shippingService, 
  address, 
  amount, 
  creditCard
) {
  const paymentRef = await paymentService.charge(creditCard, amount)

  if (paymentService.isPaid(paymentRef)) {
    shippingService.shipTo(address);
  }
}
Enter fullscreen mode Exit fullscreen mode

We have the function makeOrder(). makeOrder() gets help from two different dependencies a shippingService and a paymentService. It's critical that the paymentService is being invoked to check that we have gotten paid before we ship the merchandise, otherwise it's just bad for business.

It's also important that we at some point call the shippingService to ensure the items gets delivered. Now, it's very seldom the code is this clear so you see exactly what it does and the consequences of removing any of the below code. The point is we need to write tests for the below code and we need spies to verify that our code is being called directly.

In short:

Spies are about asserting behavior over asserting on results

 What

Ok so we've mentioned in the first few lines of this article that Spies can help us check how many times a dependency is called, with what arguments and so on but let's try to list all the features that we know of in Jasmine Spies:

  • Called, verify it has been called
  • Args, verify it has been called with a certain argument
  • Times called, verify the number of times it has been called
  • Times called and args, verify all the number of times it was called and all the arguments used
  • Mocking, return with a mocked value
  • Restore, because spies replace the original functionality we will need to restore our dependency to its original implementation at some point

That's quite a list of features and it should be able to help us assert the behavior on the above makeOrder().

The HOW

This is where we start looking at Jasmine Spies and what the public API looks like. From there we will start to sketch out what an implementation could look like.

Ok then. In Jasmine we create a Spy by calling code like this:

const apiService = {
  fetchData() {}
}
Enter fullscreen mode Exit fullscreen mode

Then we use it inside of a test like this:

it('test', () => {
  // arrange
  spyOn(apiService, 'fetchData')

  // act
  doSomething(apiService.fetchData)

  // assert
  expect(apiService.fetchData).toHaveBeenCalled();
})
Enter fullscreen mode Exit fullscreen mode

As you can see above we have three different steps that we need to care about.

  1. Creating the spy with spyOn()
  2. Invoking the spy
  3. Asserting that the spy has been called

Let's start implementing

Creating the Spy

By looking at how it's used you realize that what you are replacing is one real function for a mocked function. Which means WHAT we end up assigning to apiService.fetchData must be a function.

The other part of the puzzle is how we assert that it has been called. We have the following line to consider:

expect(apiService.fetchData).toHaveBeenCalled()
Enter fullscreen mode Exit fullscreen mode

At this point we need to start implementing that line, like so:

function expect(spy) {
  return {
    toHaveBeenCalled() {
      spy.calledTimes()
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

WAIT. You just said that apiService.fetchData is a function. Yet in expect() you send it in and call calledTimes() on it like it was an object. I'm lost :(

Ah, I see. You probably have a background from an OO language like C# or Java right?

How did you know?

In those languages you are either an object or a function, never both. But we are in JavaScript and JavaScript state that:

Functions are function objects. In JavaScript, anything that is not a primitive type ( undefined , null , boolean , number , or string ) is an object.

Which means our spy, is a function but it has methods and properties on it like it was an object..

Niiice. and weird..

Ok then. With that knowledge, we can start implementing.

// spy.js

function spy(obj, key) {
  times = 0;
  old = obj[key];

  function spy() {
    times++;
  }

  spy.calledTimes = () => times;

  obj[key] = spy;
}


function spyOn(obj, key) {
  spy(obj, key);
}

module.exports = {
  spyOn
}
Enter fullscreen mode Exit fullscreen mode

spyOn() calls spy() that internally creates the function _spy() that has knowledge of the variable times and expose the public method calledTime(). Then we end up assigning _spy to the object whose function we want to replace.

Adding matcher toHaveBeenCalled()

Let's create the file util.js and have it look like so:

// util.js

function it(testName, fn) {
  console.log(testName);
  fn();
}

function expect(spy) {
  return {
    toHaveBeenCalled() {
      let result = spy.calledTimes() > 0;
      if (result) {
        console.log('spy was called');
      } else {
        console.error('spy was NOT called');
      }
    }
  }
}

module.exports = {
  it, 
  expect
}
Enter fullscreen mode Exit fullscreen mode

As you can see it just contains a very light implementation of expect() and it() method. Let's also create a demo.js file that tests our implementation:

// demo.js

const { spyOn } = require('./spy');
const { it, expect } = require('./util');

function impl(obj) {
  obj.calc();
}

it('test spy', () => {
  // arrange
  const obj = {
    calc() {}
  }

  spyOn(obj, 'calc');

  // act
  impl(obj);

  // assert
  expect(obj.calc).toHaveBeenCalled();
})
Enter fullscreen mode Exit fullscreen mode

We have great progress already but let's look at how we can improve things.

Adding matcher toHaveBeenCalledTimes()

This matcher have pretty much written itself already as we are keeping track of the number of times we call something. Simply add the following code to our it() function, in util.js like so:

toHaveBeenCalledTimes(times) {
  let result = spy.calledTimes();
  if(result == times) {
    console.log(`success, spy was called ${times}`)
  } else {
    console.error(`fail, expected spy to be called: ${times} but was: ${result}`)
  }
}
Enter fullscreen mode Exit fullscreen mode

Adding matcher toHaveBeenCalledWith()

Now this matcher wants us to verify that we can tell what our spy has been called with and is used like this:

expect(obj.someMethod).toHaveBeenCalledWith('param', 'param2');
Enter fullscreen mode Exit fullscreen mode

Let's revisit our implementation of the spy():

// excerpt from spy.js

function spy(obj, key) {
  times = 0;
  old = obj[key];

  function spy() {
    times++;
  }

  spy.calledTimes = () => times;

  obj[key] = spy;
}
Enter fullscreen mode Exit fullscreen mode

We can see that we capture the number of times something is called through the variable times but we want to change that slightly. Instead of using a variable that stores a number let's instead replace that with an array like so:

// spy-with-args.js

function spy(obj, key) {
  let calls = []

  function _spy(...params) {
    calls.push({
      args: params
    });
  }

  _spy.calledTimes = () => calls.length;
  _spy._calls = calls;

  obj[key] = _spy;
}
Enter fullscreen mode Exit fullscreen mode

As you can see in thee _spy() method we collect all the input parameters and adds them to an array calls. calls will remember not only the number of invocations but what argument each invocation was done with.

Creating the matcher

To test that it stores all invocation and their argument lets create another matcher function in our expect() method and call it toHaveBeenCalledWith(). Now the requirements for it is that our spy should have been called with these args at some point. It doesn't say what iteration so that means we can loop through our calls array until we find a match.

Let's add our matcher to the method it() in our utils.js, like so:

// excerpt from util.js
toHaveBeenCalledWith(...params) {
  for(var i =0; i < spy._calls.length; i++) {
    const callArgs = spy._calls[i].args;
    const equal = params.length === callArgs.length && callArgs.every((value, index) => { 
      const res = value === params[index];
      return res;
    });
    if(equal) {
      console.log(`success, spy was called with ${params.join(',')} `)
      return;
    }
  }
  console.error(`fail, spy was NOT called with ${params} spy was invoked with:`);
  console.error(spy.getInvocations());

}
Enter fullscreen mode Exit fullscreen mode

Above you can see how we compare params, which is what we call it with to each of the arguments in our invocations on the spy.

Now, let's add some code to demo.js and our test method invocation, so we try out our new matcher, like so:


// excerpt from demo.js

it('test spy', () => {
  // arrange
  const obj = {
    calc() {}
  }

  spyOn(obj, 'calc');

  // act
  impl(obj);

  // assert
  expect(obj.calc).toHaveBeenCalled();
  expect(obj.calc).toHaveBeenCalledWith('one', 'two');
  expect(obj.calc).toHaveBeenCalledWith('three');
  expect(obj.calc).toHaveBeenCalledWith('one', 'two', 'three');
})
Enter fullscreen mode Exit fullscreen mode

Running this in the terminal we get:

We can see that it works like a charm. It succeeds on the two first ones and fails on the last one, as it should.

Reset, the final piece

We got one more piece of functionality we would like to add, namely the ability to reset our implementation. Now, this is probably the easiest thing we do. Let's visit our spy-with-args.js file. We need to do the following:

  1. Add a reference to the old implementation
  2. Add a method reset() that points us back to our original implementation

Add a reference

Inside of our spy() function add this line:

let old = obj[key];
Enter fullscreen mode Exit fullscreen mode

This will save the implementation to the variable old

Add reset() method

Just add the following line:

_spy.reset = () => obj[key] = old;
Enter fullscreen mode Exit fullscreen mode

The spy() method should now look like so:

function spy(obj, key) {
  let calls = []
  let old = obj[key];

  function _spy(...params) {
    calls.push({
      args: params
    });
  }

  _spy.reset = () => obj[key] = old;
  _spy.calledTimes = () => calls.length;
  _spy.getInvocations = () => {
    let str = '';
    calls.forEach((call, index) => {
      str+= `Invocation ${index + 1}, args: ${call.args} \n`;
    });

    return str;
  }

  _spy._calls = calls;

  obj[key] = _spy;
}
Enter fullscreen mode Exit fullscreen mode

Summary

We've come to the end of the line.
We've implemented a spy from the beginning. Additionally, we've explained how almost everything is an object which made it possible to implement it the way we did.

The end result is a spy that stores all the invocations and the parameters it was called with. We've also managed to create three different matchers that test whether our spy was called, how many times it was called and with what arguments.

All in all a successful adventure into understanding the nature of a spy.

We do realize that this is just a starter for something and taking it to production means we should probably support things like comparing whether something was called with an object, supporting, mocking and so on. I leave that up to you as an exercise.

As another take-home exercise, see if you can write tests for the function makeOrder() that we mentioned in the beginning.

Full code

Here is the full code in case I lost you during the way:

util.js, containing our matcher functions

Our file containing our functions it() and expect() and its matchers.

// util.js

function it(testName, fn) {
  console.log(testName);
  fn();
}

function expect(spy) {
  return {
    toHaveBeenCalled() {
      let result = spy.calledTimes() > 0;
      if (result) {
        console.log('success,spy was called');
      } else {
        console.error('fail, spy was NOT called');
      }
    },
    toHaveBeenCalledTimes(times) {
      let result = spy.calledTimes();
      if(result == times) {
        console.log(`success, spy was called ${times}`)
      } else {
        console.error(`fail, expected spy to be called: ${times} but was: ${result}`)
      }
    },
    toHaveBeenCalledWith(...params) {
      for(var i =0; i < spy._calls.length; i++) {
        const callArgs = spy._calls[i].args;
        const equal = params.length === callArgs.length && callArgs.every((value, index) => { 
          const res = value === params[index];
          return res;
        });
        if(equal) {
          console.log(`success, spy was called with ${params.join(',')} `)
          return;
        }
      }
      console.error(`fail, spy was NOT called with ${params} spy was invoked with:`);
      console.error(spy.getInvocations());

    }
  }
}

module.exports = {
  it, 
  expect
}
Enter fullscreen mode Exit fullscreen mode

spy implementation

Our spy implementation spy-with-args.js:

function spyOn(obj, key) {
  return spy(obj, key);
}

function spy(obj, key) {
  let calls = []
  let old = obj[key];

  function _spy(...params) {
    calls.push({
      args: params
    });
  }

  _spy.reset = () => obj[key] = old;
  _spy.calledTimes = () => calls.length;
  _spy.getInvocations = () => {
    let str = '';
    calls.forEach((call, index) => {
      str+= `Invocation ${index + 1}, args: ${call.args} \n`;
    });

    return str;
  }

  _spy._calls = calls;

  obj[key] = _spy;
}

module.exports = {
  spyOn
};
Enter fullscreen mode Exit fullscreen mode

demo.js, for testing it out

and lastly our demo.js file:

const { spyOn } = require('./spy-with-args');
const { it, expect } = require('./util');

function impl(obj) {
  obj.calc('one', 'two');

  obj.calc('three');
}

it('test spy', () => {
  // arrange
  const obj = {
    calc() {}
  }

  spyOn(obj, 'calc');

  // act
  impl(obj);

  // assert
  expect(obj.calc).toHaveBeenCalled();
  expect(obj.calc).toHaveBeenCalledWith('one', 'two');
  expect(obj.calc).toHaveBeenCalledWith('three');
  expect(obj.calc).toHaveBeenCalledWith('one', 'two', 'three');
})
Enter fullscreen mode Exit fullscreen mode
Image
Are You Sick Of Asbestos Lawsuit Lawyers? 10 Inspirational Sources That Will Bring Back Your Passion
Image
How to Fix Kubernetes Node Disk Pressure

devops, sre, kubernetes, help
Image
Why People Are Talking About Fabric Corner Suites Today
Image
7 Things You've Never Knew About Corner Sofa Grey Fabric
Image
How To Become A Prosperous Lost Key Car If You're Not Business-Savvy
Image
How Saleberate Tables Added Elegance and Functionality to My Small Apartment"
Image
Exploring the Benefits of Daily Prayer for Mental and Emotional Well-Being"
Image
How I Cloned Myself with LLMs to Work 24/7 – Even While I Sleep

ai, llm, tutorial, productivity
Image
Replacing Lost Car Key Explained In Fewer Than 140 Characters
Image
How do we manage to redirect users to the App Store from our download page?
Image
Luxury Cabo Yacht Charter | Private Yachts for Unforgettable Ocean Adventures
Image
Level Up Your GitHub Workflow with AI-Powered Analytics: devActivity
Image
5 costly mistakes when deploying Docker containers (and how to dodge them like a pro 😎)

docker, webdev, devops, beginners
Image
What Is Unit Testing: A Complete Guide With Examples

webdev, testing, software, ai
Image
Easily Build an AI-Powered Smart Scheduler with Blazor

blazor, webdev, scheduler, productivity
Image
122
Image
"Embracing the Arabic Language: A Gateway to Culture and Understanding"
Image
Top 8 Web development projects for beginners

webdev, javascript, beginners, programming
Image
Cat Flap Installers Near Me
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Terabox Video Player