Few days ago I filled a ticket with a proposal to use Sparrow6 as a tool for cross-platform testing of community Raku modules. While the discussion is being held I'd like to share a few examples of how one can use Sparrow6 to test Raku modules.

Install Sparrowdo

Name: Multi-platform testing of Raku modules using Sparrow6
Rating: 4.5 (3279 reviews)
Author: melezhik

Sparrowdo is a command line utility to run Sparrow6 scenarios on docker containers.

zef install --/test Sparrowdo

Sparrowdo has just a few dependencies so it won't take long for it to install:

===> Searching for: Sparrowdo
===> Updating cpan mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/cpan1.json
===> Updating p6c mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/p6c1.json
===> Updated p6c mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/p6c1.json
===> Updated cpan mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/cpan1.json
===> Searching for missing dependencies: Sparrow6
===> Searching for missing dependencies: File::Directory::Tree, Hash::Merge, YAMLish, JSON::Tiny
===> Updating cpan mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/cpan1.json
===> Searching for missing dependencies: MIME::Base64
===> Installing: File::Directory::Tree:auth<labster>
===> Installing: Hash::Merge:ver<1.0.0>:auth<github:scriptkitties>:api<1>
===> Installing: MIME::Base64:ver<1.2.1>:auth<github:retupmoca>
===> Installing: YAMLish:ver<0.0.5>
===> Installing: JSON::Tiny:ver<1.0>
===> Installing: Sparrow6:ver<0.0.11>
===> Installing: Sparrowdo:ver<0.1.2>

Get a source code of examples

git clone https://github.com/melezhik/RakuDist && cd RakuDist && ls -l

drwxr-xr-x. 4 user1 wheel    28 янв  2 17:27 modules
-rw-r--r--. 1 user1 wheel 10266 янв  2 17:27 README.md
drwxr-xr-x. 2 user1 wheel    50 янв  2 17:27 reports

Folder modules/ will contain examples of test scenarios written on Sparrow6 DSL that we run remotely on running docker instances.

You can use those examples as starting point when creating your own scenarios.

We will go back to one of them a little bit later.

Spin up a docker container

Choose a docker image with OS you want to run tests against and spin it up.

I am choosing an alpine OS image as it extremely light ( 5MB in size ) and does not take long to download:

docker pull alpine && container_name='alpine-rakudist' && docker run -d -t --rm --name $container_name alpine && docker ps

A literally few second and we can see our docker container up and running:

Using default tag: latest
latest: Pulling from library/alpine
e6b0cf9c0882: Pull complete 
Digest: sha256:2171658620155679240babee0a7714f6509fae66898db422ad803b951257db78
Status: Downloaded newer image for alpine:latest
docker.io/library/alpine:latest
14404faef5589e9228f9a75c2cfc900ed08b50b3e9f44c269a6741864809acff
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                  PORTS               NAMES
14404faef558        alpine              "/bin/sh"           1 second ago        Up Less than a second                       alpine-rakudist

Run tests

Let's change to module directory from RakuDist project and find some examples.

Say, I want to run tests for Red module:

cd modules/red/

We have a couple of files here:

config.pl6

config.pl6 - a file that contains all configuration data, it's just a easy way to hold configuration parameters and pass them as a Raku Hash to a test scenario.

cat config.pl6

%(
  user => "red",
  scm => "https://github.com/FCO/Red.git"
)

sparrowfile

sparrowfile - is a test scenario itself, written on Sparrow6 DSL and defining all testing logic:

check out module source code from a git repository
create a system user - we might want to reuse the same docker container to test other modules as well, so it's reasonable to have a separate user for every module to avoid overlaps between module Raku dependencies, because we install those dependencies under a dedicated user.
install module external dependencies (sqlite library), for simplicity assumption is made that sqlite is already included in other linux distros, so we only install it for alpine os. But see modules/Cro/sparrowfile how external libraries for a variety of OS could be installed through the same scenario, that to write true multi-platform tests.
install module dependencies through zef install --deps-only
run module unit tests through zef --test .

cat sparrowfile

my $user = config()<user>;

my $directory = "/data/test/{$user}";

my $scm = config()<scm>;

user $user;

directory $directory, %( 
  owner => $user,
  group => $user

);

git-scm $scm, %( 
  to => $directory, 
  user => $user,
);

bash "cd {$directory} && git log --name-status HEAD^..HEAD", %(
  description => "last commit"
);

bash "cd {$directory} && ls -l";

zef "Test::META", %( notest => True );

if os() eq 'alpine' {

  # this is needed for alpine rakudo installation
  unless "/bin/zef".IO ~~ :e {
    copy "/opt/rakudo-pkg/share/perl6/core/bin/zef", "/bin/zef"
  }

  # this is needed for alpine rakudo installation
  unless "/bin/perl6".IO ~~ :e {
    copy "/opt/rakudo-pkg/bin/perl6", "/bin/perl6"
  }

  package-install "sqlite-libs";

}


zef $directory, %( 
  force => False,
  depsonly => True, 
  notest => True, 
  user => $user 
);

bash "cd {$directory} && zef test .", %(
  description => "zef test",
  user => $user
);

Run tests

Sparrowdo is a command line client to run Sparrow6 remotely ( over ssh or docker ).

When I run sparrowdo I add --bootstrap flag that ensures that Rakudo and Sparrow6 are installed on a docker container.

Luckily when Sparrowdo was designed it had this --bootstrap option, which makes it so convenient tool to test Raku modules!

It worth to mention that Sparrow6 has a small dependency tree (File::Directory::Tree, Hash::Merge, YAMLish, JSON::Tiny) that means it gets installed really quick and almost also does not "pollute" a testing environment with dependencies installed system wide.

It also worth to say, that we only use --boostrap once and don't pass it every time we run tests.

We also need to change to a specific directory so that sparrowdo will pick up mentioned files ( sparrowfile and config.pl6 )

I also pass _no_sudo flag, as we already run all the commands under root user when using a docker.

Finally I pass a --repo argument pointing to a public Sparrow6 repository as Sparrow6 DSL needs to download some dependencies ( plugins ) when run.

So a full command like like this:

cd modules/red && sparrowdo --bootstrap --no_sudo --docker=$container_name --repo=http://repo.westus.cloudapp.azure.com

And we have a test report:

alpine
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
v3.11.2-29-ge1c437ff58 [http://dl-cdn.alpinelinux.org/alpine/v3.11/main]
v3.11.2-28-g72cc761b5b [http://dl-cdn.alpinelinux.org/alpine/v3.11/community]
OK: 11261 distinct packages available
(1/17) Installing ncurses-terminfo-base (6.1_p20191130-r0)
(2/17) Installing ncurses-terminfo (6.1_p20191130-r0)
(3/17) Installing ncurses-libs (6.1_p20191130-r0)
(4/17) Installing readline (8.0.1-r0)
(5/17) Installing bash (5.0.11-r1)
Executing bash-5.0.11-r1.post-install
(6/17) Installing ca-certificates (20191127-r0)
(7/17) Installing nghttp2-libs (1.40.0-r0)
(8/17) Installing libcurl (7.67.0-r0)
(9/17) Installing curl (7.67.0-r0)
(10/17) Installing expat (2.2.9-r1)
(11/17) Installing pcre2 (10.34-r1)
(12/17) Installing git (2.24.1-r0)
(13/17) Installing libbz2 (1.0.8-r1)
(14/17) Installing perl (5.30.1-r0)
(15/17) Installing perl-error (0.17028-r0)
(16/17) Installing perl-git (2.24.1-r0)
(17/17) Installing git-perl (2.24.1-r0)
Executing busybox-1.31.1-r8.trigger
Executing ca-certificates-20191127-r0.trigger
OK: 68 MiB in 31 packages
(1/1) Installing rakudo-pkg (2019.11-01)
OK: 68 MiB in 32 packages
===> Searching for missing dependencies: File::Directory::Tree, Hash::Merge, YAMLish, JSON::Tiny
===> Updating cpan mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/cpan1.json
===> Searching for missing dependencies: MIME::Base64
===> Installing: File::Directory::Tree:auth<labster>
===> Installing: Hash::Merge:ver<1.0.0>:auth<github:scriptkitties>:api<1>
===> Installing: MIME::Base64:ver<1.2.1>:auth<github:retupmoca>
===> Installing: YAMLish:ver<0.0.5>
===> Installing: JSON::Tiny:ver<1.0>
===> Installing: Sparrow6:ver<0.0.11>

1 bin/ script [s6] installed to:
/opt/rakudo-pkg/share/perl6/site/bin
===> Updating p6c mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/p6c1.json
===> Updated p6c mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/p6c1.json
===> Updated cpan mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/cpan1.json
22:03:46 01/05/2020 [repository] index updated from http://repo.westus.cloudapp.azure.com/api/v1/index
22:03:53 01/05/2020 [create user red] uid=1000(red) gid=1000(red) groups=1000(red)
22:03:53 01/05/2020 [create user red] user red created
[task check] stdout match <created> True
22:03:57 01/05/2020 [create directory /data/test/red] directory path: /data/test/red
22:03:57 01/05/2020 [create directory /data/test/red] directory owner: <red>
22:03:57 01/05/2020 [create directory /data/test/red] directory group: <red>
22:03:57 01/05/2020 [create directory /data/test/red] directory access rights: drwxr-xr-x
[task check] stdout match <owner: <red>> True
[task check] stdout match <group: <red>> True
22:04:01 01/05/2020 [bash: git checkout https://github.com/FCO/Red.git] /data/test/red
22:04:01 01/05/2020 [bash: git checkout https://github.com/FCO/Red.git] stderr: Cloning into '.'...
22:04:04 01/05/2020 [bash: last commit] commit e59b3146e2e2f7040aaa1d4af1b1924d79246a41
22:04:04 01/05/2020 [bash: last commit] Author: Fernando Correa de Oliveira <fernandocorrea@gmail.com>
22:04:04 01/05/2020 [bash: last commit] Date:   Fri Jan 3 00:12:12 2020 +0000
22:04:04 01/05/2020 [bash: last commit] 
22:04:04 01/05/2020 [bash: last commit]     0.1.4
22:04:04 01/05/2020 [bash: last commit] 
22:04:04 01/05/2020 [bash: last commit] M   Changes
22:04:04 01/05/2020 [bash: last commit] M   META6.json
22:04:04 01/05/2020 [bash: last commit] M   lib/Red.pm6
22:04:07 01/05/2020 [bash: cd /data/test/red && ls -l] total 68
22:04:07 01/05/2020 [bash: cd /data/test/red && ls -l] -rw-r--r--    1 red      red            592 Jan  5 22:04 CONTRIBUTING.md
22:04:07 01/05/2020 [bash: cd /data/test/red && ls -l] -rw-r--r--    1 red      red           2598 Jan  5 22:04 Changes
22:04:07 01/05/2020 [bash: cd /data/test/red && ls -l] -rw-r--r--    1 red      red            384 Jan  5 22:04 Dockerfile
22:04:07 01/05/2020 [bash: cd /data/test/red && ls -l] -rw-r--r--    1 red      red            332 Jan  5 22:04 Dockerfile-no-config
22:04:07 01/05/2020 [bash: cd /data/test/red && ls -l] -rw-r--r--    1 red      red            427 Jan  5 22:04 Dockerfile-no-run
22:04:07 01/05/2020 [bash: cd /data/test/red && ls -l] -rw-r--r--    1 red      red           8902 Jan  5 22:04 LICENSE
22:04:07 01/05/2020 [bash: cd /data/test/red && ls -l] -rw-r--r--    1 red      red           5955 Jan  5 22:04 META6.json
22:04:07 01/05/2020 [bash: cd /data/test/red && ls -l] -rw-r--r--    1 red      red            123 Jan  5 22:04 Makefile
22:04:07 01/05/2020 [bash: cd /data/test/red && ls -l] -rw-r--r--    1 red      red          10509 Jan  5 22:04 README.md
22:04:07 01/05/2020 [bash: cd /data/test/red && ls -l] -rw-r--r--    1 red      red            482 Jan  5 22:04 Red.iml
22:04:07 01/05/2020 [bash: cd /data/test/red && ls -l] drwxr-xr-x    2 red      red             17 Jan  5 22:04 bin
22:04:07 01/05/2020 [bash: cd /data/test/red && ls -l] -rw-r--r--    1 red      red            100 Jan  5 22:04 dist.ini
22:04:07 01/05/2020 [bash: cd /data/test/red && ls -l] drwxr-xr-x    4 red      red             83 Jan  5 22:04 docs
22:04:07 01/05/2020 [bash: cd /data/test/red && ls -l] drwxr-xr-x    8 red      red             82 Jan  5 22:04 examples
22:04:07 01/05/2020 [bash: cd /data/test/red && ls -l] drwxr-xr-x    6 red      red             69 Jan  5 22:04 lib
22:04:07 01/05/2020 [bash: cd /data/test/red && ls -l] drwxr-xr-x    3 red      red           4096 Jan  5 22:04 t
22:04:07 01/05/2020 [bash: cd /data/test/red && ls -l] drwxr-xr-x    2 red      red             27 Jan  5 22:04 tools
22:04:12 01/05/2020 [bash: zef install Test::META] ===> Searching for: Test::META
22:06:55 01/05/2020 [bash: zef install Test::META] ===> Searching for missing dependencies: META6, URI, License::SPDX
22:06:55 01/05/2020 [bash: zef install Test::META] ===> Searching for missing dependencies: JSON::Class:ver<0.0.14+>, JSON::Class:ver<0.0.5+>, JSON::Fast
22:06:55 01/05/2020 [bash: zef install Test::META] ===> Searching for missing dependencies: JSON::Marshal:ver<0.0.18+>, JSON::Unmarshal:ver<0.08+>
22:06:55 01/05/2020 [bash: zef install Test::META] ===> Searching for missing dependencies: JSON::Name
22:06:55 01/05/2020 [bash: zef install Test::META] ===> Installing: JSON::Fast:ver<0.10>
22:06:55 01/05/2020 [bash: zef install Test::META] ===> Installing: JSON::Name:ver<0.0.4>:auth<github:jonathanstowe>:api<1.0>
22:06:55 01/05/2020 [bash: zef install Test::META] ===> Installing: JSON::Marshal:ver<0.0.18>:auth<github:jonathanstowe>:api<1.0>
22:06:55 01/05/2020 [bash: zef install Test::META] ===> Installing: JSON::Unmarshal:ver<0.08>
22:06:55 01/05/2020 [bash: zef install Test::META] ===> Installing: JSON::Class:ver<0.0.14>:auth<github:jonathanstowe>:api<1.0>
22:06:55 01/05/2020 [bash: zef install Test::META] ===> Installing: META6:ver<0.0.23>:auth<github:jonathanstowe>:api<1.0>
22:06:55 01/05/2020 [bash: zef install Test::META] ===> Installing: URI:ver<0.3.0>
22:06:55 01/05/2020 [bash: zef install Test::META] ===> Installing: License::SPDX:ver<3.7.1>:auth<github:jonathanstowe>:api<1.0>
22:06:55 01/05/2020 [bash: zef install Test::META] ===> Installing: Test::META:ver<0.0.16>:auth<github:jonathanstowe>:api<1.0>
22:06:57 01/05/2020 [install package(s): sqlite-libs.perl] fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
22:06:57 01/05/2020 [install package(s): sqlite-libs.perl] fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
22:06:57 01/05/2020 [install package(s): sqlite-libs.perl] v3.11.2-29-ge1c437ff58 [http://dl-cdn.alpinelinux.org/alpine/v3.11/main]
22:06:57 01/05/2020 [install package(s): sqlite-libs.perl] v3.11.2-28-g72cc761b5b [http://dl-cdn.alpinelinux.org/alpine/v3.11/community]
22:06:57 01/05/2020 [install package(s): sqlite-libs.perl] OK: 11262 distinct packages available
22:06:58 01/05/2020 [install package(s): sqlite-libs.perl] trying to install sqlite-libs ...
22:06:58 01/05/2020 [install package(s): sqlite-libs.perl] installer - apk
22:06:58 01/05/2020 [install package(s): sqlite-libs.perl] (1/1) Installing sqlite-libs (3.30.1-r1)
22:06:59 01/05/2020 [install package(s): sqlite-libs.perl] OK: 69 MiB in 33 packages
22:06:59 01/05/2020 [install package(s): sqlite-libs.perl] Installed:                                Available:
22:06:59 01/05/2020 [install package(s): sqlite-libs.perl] sqlite-libs-3.30.1-r1                   = 3.30.1-r1 
22:06:59 01/05/2020 [install package(s): sqlite-libs.perl] sqlite-libs
22:07:04 01/05/2020 [bash: zef install /data/test/red] ===> Searching for missing dependencies: DBIish, DB::Pg, UUID
22:07:04 01/05/2020 [bash: zef install /data/test/red] stderr: ===> Updating cpan mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/cpan1.json
22:08:36 01/05/2020 [bash: zef install /data/test/red] ===> Searching for missing dependencies: LibUUID, NativeHelpers::Blob
22:08:36 01/05/2020 [bash: zef install /data/test/red] ===> Searching for missing dependencies: NativeLibs:auth<github:salortiz>
22:08:36 01/05/2020 [bash: zef install /data/test/red] ===> Installing: UUID:ver<1.0.0>:auth<github:retupmoca>
22:08:36 01/05/2020 [bash: zef install /data/test/red] ===> Installing: NativeLibs:ver<0.0.7>:auth<github:salortiz>
22:08:36 01/05/2020 [bash: zef install /data/test/red] ===> Installing: LibUUID:ver<0.5>:auth<github:CurtTilmes>
22:08:36 01/05/2020 [bash: zef install /data/test/red] ===> Installing: DB::Pg:ver<0.6>
22:08:36 01/05/2020 [bash: zef install /data/test/red] ===> Installing: NativeHelpers::Blob:ver<0.1.12>:auth<github:salortiz>
22:08:36 01/05/2020 [bash: zef install /data/test/red] ===> Installing: DBIish:ver<0.5.19>
22:08:36 01/05/2020 [bash: zef install /data/test/red] stderr: ===> Updating p6c mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/p6c1.json
===> Updated p6c mirror: ht
22:08:36 01/05/2020 [bash: zef install /data/test/red] stderr: tps://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/p6c1.json
===> Updated cpan mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/cpan1.json
22:08:40 01/05/2020 [bash: zef test] ===> Testing: Red:ver<0.1.4>:auth<Fernando Correa de Oliveira>:api<2>
22:12:23 01/05/2020 [bash: zef test] [Red] Use of Nil in numeric context
22:12:23 01/05/2020 [bash: zef test] [Red]   in block  at /data/test/red/.precomp/902863C6FF81B0B9901E5C42393B9B7181A4AE04/F2/F2E53992C6FFEDC5DC3B09E6E9D69BBEB965D56B line 1
22:12:23 01/05/2020 [bash: zef test] ===> Testing [OK] for Red:ver<0.1.4>:auth<Fernando Correa de Oliveira>:api<2>

Further thoughts

It's possible to test against various Rakudo versions once I'll add ability to pass a Rakudo version to bootstrap process ( now the latest version is installed ).

If one need to add some custom tasks that are not covered by Sparrow6 DSL, it's possible through Sparrow6 tasks API:

cat modules/red/data/tasks/hello/task.pl6

say "Hello Raku!", config()<var1>;

Then in sparrowfile just say this:

task-run "data/tasks/hello", %(
  var1 => "value" # parameter
)

You can even share your custom tasks across team converting them into Sparrow6 plugins, and make reusable across many test scenarios.

Say you create a plugin named "hello", based on "data/tasks/hello" task. This is how you run it in a test scenario:

task-run "my hello plugin", "hello", %(
  var1 => "value" # parameter
)

Conclusion

Sparrow6 is simple to use and flexible tool to automate test scenarios to check Raku modules against different OS and Rakudos, making multi-platform testing dead easy.

Follow to a proposal ticket and share your opinion.

Thank you for reading.