With the bloom of micro-service architectures in recent days, a demand in being able to test the integration of those micro-services has also become a lot relevant and a challenge as well. I was also facing a problem while doing some integration tests with my services and tried to come up with an automated testing strategy for my code base written in c#.
Background
A few months ago I released a open source library that contains a simple repository.
You can find the repository here – Atl.GenericRepository
One of the features I was adding in recent days was a test case that demonstrates the way to connect to a PostgreSQL database. Since this does not require any special changes to the repository, I wanted to add it as an integration test. I decided to go with docker. But I wanted the make it automated, so in short –
- I should be able to spawn a
postgres
docker when running the test - Run the test cases
- Stop the container when the test cases are finished
- Remove the container
Introducing Microsoft Docket.DotNet
While looking for a wrapper for docker that can spawn and remove container I came across this open source library from MS and well, this was exactly what I was looking for. I will be using this library to create, spawn and stop my container for my coded integration test. This gives me options to –
- Run the container from C# code
- Connect to the container
- Stop/Remove the container from C# code.
The library is available here –
https://github.com/Microsoft/Docker.DotNet
Start PostgreSQL Container with C# Code (Microsoft Docker.DotNet)
Ok, lets begin, we will down a postgres
image and spawn a container with the image.
Prerequisite – A Docker installation
Docker.DotNet connects to your local docker installation pretty easily. I am using a windows machine so I had docker for windows installed on my machine.
Install the nuget package
Install the nuget package Docker.Dotnet to get started –
install-package Docker.DotNet
or,
dotnet add package Docker.DotNet
Connect to docker
The following code base connects to the local docker (docker for window for me) and creates a client to interact with it –
var client = new DockerClientConfiguration(new Uri("npipe://./pipe/docker_engine")).CreateClient();
Run container for PostgreSQL
The strategy –
- I am going to map a random port (10000 – 12000) on host machine to port 5432 on the docker container. That way I will be able to connect to the docker container as if this postgresql database is running on the host machine. This makes it easier to connect.
- Since I am only running for a integration test that works on volatile data, I am not worried about mapping a volume or mounting a data file to pre-load data into the database. But if you want you can surely do that with appropriate commands.
Now, we have got our docker wrapper/client, lets spawn container. We will first look for a suitable image. Then we will create a container for it and finally will start the container. I have created a method to do all this task, lets call it GetContainer -
private async Task<(CreateContainerResponse, string)> GetContainer(DockerClient client, string image, string tag)
{
var hostPort = new Random((int) DateTime.UtcNow.Ticks).Next(10000, 12000);
//look for image
var images = await client.Images.ListImagesAsync(new ImagesListParameters()
{
MatchName = $"{image}:{tag}",
}, CancellationToken.None);
//check if container exists
var pgImage = images.FirstOrDefault();
if (pgImage == null)
throw new Exception($"Docker image for {image}:{tag} not found.");
//create container from image
var container = await client.Containers.CreateContainerAsync(new CreateContainerParameters()
{
User = "postgres",
Env = new List<string>()
{
"POSTGRES_PASSWORD=password",
"POSTGRES_DB=repotest",
"POSTGRES_USER=postgres"
},
ExposedPorts = new Dictionary<string, EmptyStruct>()
{
["5432"] = new EmptyStruct()
},
HostConfig = new HostConfig()
{
PortBindings = new Dictionary<string, IList<PortBinding>>()
{
["5432"] = new List<PortBinding>()
{new PortBinding() {HostIP = "0.0.0.0", HostPort = $"{hostPort}"}}
}
},
Image = $"{image}:{tag}",
}, CancellationToken.None);
if (!await client.Containers.StartContainerAsync(container.ID, new ContainerStartParameters()
{
DetachKeys = $"d={image}"
}, CancellationToken.None))
{
throw new Exception($"Could not start container: {container.ID}");
}
var count = 10;
Thread.Sleep(5000);
var containerStat = await client.Containers.InspectContainerAsync(container.ID, CancellationToken.None);
while (!containerStat.State.Running && count-- > 0)
{
Thread.Sleep(1000);
containerStat = await client.Containers.InspectContainerAsync(container.ID, CancellationToken.None);
}
return (container, $"{hostPort}");
}
Well, that is a long method. Lets try to understand what it does –
var hostPort = new Random((int) DateTime.UtcNow.Ticks).Next(10000, 12000);
The purpose of this block is randomizing a port to bind with the postgresql 5432 port. The reason for is to minimizing the probability of choosing a port which could already be blocked in the host machine.
Next, we look for a image, (I am using postgress:10.7-alpine
) , so I passed in image=postgres
and tag=10.7-alpine
–
var images = await client.Images.ListImagesAsync(new ImagesListParameters()
{
MatchName = $"{image}:{tag}",
}, CancellationToken.None);
//check if container exists
var pgImage = images.FirstOrDefault();
if (pgImage == null)
throw new Exception($"Docker image for {image}:{tag} not found.");
Assuming we got our image, we then crate the container –
//create container from image
var container = await client.Containers.CreateContainerAsync(new CreateContainerParameters()
{
User = "postgres",
Env = new List<string>()
{
"POSTGRES_PASSWORD=password",
"POSTGRES_DB=repotest",
"POSTGRES_USER=postgres"
},
ExposedPorts = new Dictionary<string, EmptyStruct>()
{
["5432"] = new EmptyStruct()
},
HostConfig = new HostConfig()
{
PortBindings = new Dictionary<string, IList<PortBinding>>()
{
["5432"] = new List<PortBinding>()
{new PortBinding() {HostIP = "0.0.0.0", HostPort = $"{hostPort}"}}
}
},
Image = $"{image}:{tag}",
}, CancellationToken.None);
Well, that is pretty straight forward. I am supplying all the details needed to run a postgresql server in a docker container and mapping a host port to its 5432 port.
Lets start the container –
if (!await client.Containers.StartContainerAsync(container.ID, new ContainerStartParameters()
{
DetachKeys = $"d={image}"
}, CancellationToken.None))
{
throw new Exception($"Could not start container: {container.ID}");
}
Using the container ID, that we got it from the previous step, we start the container. Now, this call is a bit tricky, for some reason it returns before even the container is fully initialized. So, in the next code block we poll the container to make sure it is initialized and running –
var count = 10;
Thread.Sleep(5000);
var containerStat = await client.Containers.InspectContainerAsync(container.ID, CancellationToken.None);
while (!containerStat.State.Running && count-- > 0)
{
Thread.Sleep(1000);
containerStat = await client.Containers.InspectContainerAsync(container.ID, CancellationToken.None);
}
Once initialized lets save the container ID to dispose when the test is completed.
Connect to the Database
Since I am proxying a host port to the postgresql database, connecting to it pretty straightforward. Here my connection string –
$"User ID=postgres;Password=password;Server=127.0.0.1;Port={_port};Database=repotest;Integrated Security=true;Pooling=false;CommandTimeout=3000"
See it in Action
Well, lets first up the test cases and see if it works or not. Here are some screenshots.
I put a break point just after the container is started, and voila, you can see a container running –
Cleanup
Then when integration tests are completed, you stop the container and remove it.
if (await _client.Containers.StopContainerAsync(_containerResponse.ID, new ContainerStopParameters(), CancellationToken.None))
{
//delete container
await _client.Containers.RemoveContainerAsync(_containerResponse.ID, new ContainerRemoveParameters(), CancellationToken.None);
}
At the end of the test, everything is success and our container is cleaned and removed –
The full test class is available at the repository of Atl.Repository.Standard, here –
Hi, works great when i run dotnet test locally but when I use a dockerfile to run the tests i get “The operation has timed out.” when it tries to CreateImageAsync. Maybe it’s unable to create a docker container inside the docker container?
LikeLiked by 1 person
hm.. I have never tried that. But this could be interesting. Normally, you are not supposed to run docker inside a docker. The usual and recommended one is connect to host docker and spawn a sibling. I have never tried that with this library. Will give it a try next time I have some free hours.
LikeLike
It s really interesting for integration test. Next week I ll use your concept for sql server test
LikeLike