Enhance your .NET Testing #5: Integration tests with Docker.DotNet

In the vast realm of software development, integration testing stands as a pivotal phase, ensuring that various software modules work in harmony. But how can one streamline this process, especially when external services like databases are involved?

Enter Docker.DotNet, a gem supported by the esteemed .NET Foundation.

This article delves deep into the intricacies of Docker.DotNet, guiding you through the creation of integration tests using this remarkable library.

An In-Depth Overview

Docker.DotNet is a beacon for developers aiming to harness the power of Docker through .NET applications, encapsulating the principles of seamless integration, flexibility, and robustness.

It facilitates communication with the Docker Remote API, enabling developers to manage container lifecycles with ease. Its design ensures that developers can interact with Docker in a manner that feels native to the .NET environment, bridging the gap between containerization and application development.

Before diving into the code, it's essential to understand the 4 core methods of Docker.DotNet that we'll be leveraging:

DockerClientConfiguration: This is the starting point. It allows us to configure and create a Docker client, specifying the endpoint for Docker's Remote API.
CreateContainerAsync & RemoveContainerAsync: Create or remove a Docker container based on specified parameters, such as the Docker image and exposed ports.
StartContainerAsync & StopContainerAsync: These methods are your controls to start and stop a Docker container, ensuring you have full command over the container's lifecycle.

It seems pretty obvious, and all of this is necessary to manage the container lifecycle during an integration test.

Crafting Integration Tests: A Step-by-Step Approach

Imagine you're tasked with retrieving all goats from an API that communicates with a database housed within a container. How would you go about it?

Firstly, you'd need a Docker image of the external service, in this case, a database. This image is the cornerstone of our integration test, laying the groundwork for the subsequent steps.

With the Docker image ready, the next logical step is to set up a WebApplicationFactory. This factory is then enhanced by integrating the Docker image, ensuring our application has uninterrupted access to the database.

As we navigate through the WebApplicationFactory, the Docker image is ignited, marking the onset of our tests. This ensures our testing environment is primed and mirrors real-world scenarios.

With our environment set, we then run our test. This test employs the HttpClient of the WebApplicationFactory to fetch a list of goats, focusing on their id and name.

Once our tests conclude, it's essential to halt the Docker image. This ensures optimal resource management and prevents any potential overlaps or conflicts.

To set the stage, ensure Docker.DotNet is installed via NuGet: Install-Package Docker.DotNet

Now, let's bring this plan to life with some code:

public class Goat
{
    public int Id { get; set; }
    public string Name { get; set; }
}

public class IntegrationTests : IDisposable
{
    private readonly DockerClient _dockerClient;
    private readonly WebApplicationFactory<YourWebApp.Startup> _factory;
    private string _containerId;

    public IntegrationTests()
    {
        _dockerClient = new DockerClientConfiguration()).CreateClient();
        // Use new DockerClientConfiguration(new Uri("PATHTODOCKER")).CreateClient() if you need to specify Docker daemon
        _factory = new WebApplicationFactory<YourWebApp.Startup>();
    }

    [Fact]
    public async void TestGoatsEndpoint()
    {
        // Produce a docker container of an external service (e.g. a database)
        var createParameters = new CreateContainerParameters
        {
            Image = "your-database-image",
            // Configure container's ports
            HostConfig = new HostConfig
            {
                PortBindings = new Dictionary<string, IList<PortBinding>>
                {
                    { "1111", new List<PortBinding> { new PortBinding { HostPort = "1111" } } }
                }
            }
            // Configure exposed ports, usefull if you need to access from your tests
            ExposedPorts = new Dictionary<string, EmptyStruct>
            {
                { "1111", default(EmptyStruct) } // Set any port you want, it just needs to match the one from your configuration
            },
        };

        var container = await _dockerClient.Containers.CreateContainerAsync(createParameters);
        _containerId = container.ID;

        // Start the docker image at the beginning of the tests
        await _dockerClient.Containers.StartContainerAsync(_containerId, null);

        // Run a test using the HttpClient of WebApplicationFactory
        var client = _factory.CreateClient();
        var response = await client.GetAsync("/api/goats");
        response.EnsureSuccessStatusCode();

        var goats = JsonSerializer.Deserialize<List<Goat>>(response.Content.ReadAsStringAsync().Result);
        goats.Should().NotBeEmpty();
    }

    public void Dispose()
    {
        // Stop and remove the docker container at the end of the tests
        if (_containerId != null)
        {
            _dockerClient.Containers.StopContainerAsync(_containerId, new ContainerStopParameters()).Wait();
            _dockerClient.Containers.RemoveContainerAsync(_containerId, new ContainerRemoveParameters()).Wait();
        }
    }
}

Next steps to build the whole integration tests

The code above allows you to build the first integration test using real containers for external components such as the database, RabbitMq, Vault and others.

However, it is not possible to keep the test as it is and copy it to create new tests. It's only a base from which to add random generation of container ports.

Why do we need to do this?

Quite simply so that test execution can be parallelized. If several tests are run with several containers on the same port, the tests will explode. To avoid this, you need to retrieve a free port from all those available for each test. I personally use this technique:

int FreeTcpPort()
{
    var tcpListener = new TcpListener(IPAddress.Loopback, 0);
    tcpListener.Start();
    var port = ((IPEndPoint)tcpListener.LocalEndpoint).Port;
    tcpListener.Stop();
    return port;
}

The next step is to modify the application parameters to change the connection port to the one defined above, directly in the WebApplicationFactory.

If you've never used it before, I suggest you read our article on the subject:

⚠️

Don't forget to modify the _containerId with a concurrent dictionary to avoid access errors.

Enhancing Docker.DotNet

HTTPS Configuration

Securing your Docker API calls is paramount, especially when dealing with sensitive data or operations. Docker.DotNet provides a straightforward way to integrate HTTPS, ensuring encrypted communication and safeguarding data integrity.

To set up HTTPS:

First, ensure you have SSL certificates for your Docker server.
Configure Docker to use these certificates.
Use Docker.DotNet to communicate over HTTPS.

Here's a simple code snippet to illustrate how to configure Docker.DotNet to use HTTPS:

var credentials = new CertificateCredentials(new X509Certificate2("path_to_your_cert.pfx", "your_cert_password"));
var config = new DockerClientConfiguration(new Uri("https://your_docker_host:2376"), credentials);
var client = config.CreateClient();

To set the stage, ensure Docker.DotNet.X509 is installed via NuGet: Install-Package Docker.DotNet.X509

Stream responses

Docker.DotNet offers the capability to utilize stream responses, which can be particularly useful when you want to monitor the logs of a running container or track real-time events.

For instance, to fetch logs from a container:

var logsParameters = new ContainerLogsParameters
{
    ShowStdout = true,
    ShowStderr = true,
    Follow = true,
    Tail = "all"
};

using (var stream = await _dockerClient.Containers.GetContainerLogsAsync(_containerId, logsParameters, CancellationToken.None))
{
    using (var reader = new StreamReader(stream))
    {
        while (!reader.EndOfStream)
        {
            var line = await reader.ReadLineAsync();
            Console.WriteLine(line);
        }
    }
}

Summary

Integration testing, when powered by Docker.DotNet, is transformed into an efficient and seamless endeavor. By adhering to the guidelines above, developers can craft software modules that are not only robust but also harmonious.

To go further: