Make (containers with) networks reusable

[Tillerino]

I am loving the reusable containers. They can speed up development so much when repeating individual tests locally or when tests are run in several JVMs successively in the CI.

Currently, I am working with containers that are additionally connected through a network. These cannot be reused since the name of the network will be a random UUID each time. The hash of the containers to be reused is based on the create container command. This includes the network. Hence the hash chances each time and a reusable container cannot be found if a network is used.

I guess an easy solution would be to make the name of the network customizable. I can imagine weird effects when a user doesn't know the exact consequences, however. A more robust solution would be copy the reuse mechanism from containers using a name prefix for the network to make sure that one can have multiple networks which are created with identical settings.

What do you think? I would like to work on this, if it matches the direction of the project.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this is a mistake, please reply to this comment to keep it open. If there isn't one already, a PR to fix or at least reproduce the problem in a test case will always help us get back on track to tackle this.

[Tillerino]

Thanks stale bot. There is already a PR which at least prevents erroneously reusing containers with Networks.

I was hoping for some discussion on the matter itself :)

[knutwannheden]

Allowing reuse of connected containers would be really great. As a workaround I now use the Docker client to check if the network already exists and then use Java reflection to modify the NetworkImpl object. #3084 (comment) mentions that "there are options to preserve the network IDs, which makes the reuse work". Can someone elaborate on those options? I assume they are better than my hack.

[bsideup]

@knutwannheden there is no need to use Reflection btw, just implement Network yourself.
We plan to work on improving the reusable mode eventually, but it requires some major changes, as compared to the current "alpha" version.

[knutwannheden]

Thanks for the tip. I guess I have grown too used to always declaring everything final 😄

I was a bit surprised that the containers declared for reuse always get stopped through the JUnit 5 lifecycle hooks of the @Container annotation. Is there also some workaround for that?

[knutwannheden]

I was a bit surprised that the containers declared for reuse always get stopped through the JUnit 5 lifecycle hooks of the @Container annotation. Is there also some workaround for that?

Would it possibly make sense with something like @Container(reuse = true) to indicate that a container should not be stopped at the end of the test scope and thus be available for reuse by another matching @Container annotation in another test?

[bsideup]

@knutwannheden the problem with this approach is that the behaviour will differ between reuse and regular mode (as it is per-environment)

It is recommended to use the manual lifecycle control:
https://www.testcontainers.org/test_framework_integration/manual_lifecycle_control/

You should really think about the reusable containers as a next step in the "Container per test (@Rule) -> per class (@ClassRule) -> per JVM (singleton)" chain

[knutwannheden]

Makes sense. Thanks for the explanation.

[loic-seguin]

Hi @knutwannheden, if you could share your hack or at least what you are planning to do, it would be great !!
I am also really interested by the ability to reuse containers that share the same network.

[knutwannheden]

@loic-seguin What I did was to use Java reflection to create a Network object and then set its name, id, and initialized fields. I set the name field, since I also use the name to find any corresponding network to reuse, if it already exists. But that could of course also be done using the id or some label:

    private Network createNetwork(String networkName) {
        Network network = Network.newNetwork();
        try {
            Field nameField = Network.NetworkImpl.class.getDeclaredField("name");
            nameField.setAccessible(true);
            nameField.set(network, networkName);
            List<com.github.dockerjava.api.model.Network> networks =
                    DockerClientFactory.instance().client().listNetworksCmd().withNameFilter(networkName).exec();
            if (!networks.isEmpty()) {
                Field idField = Network.NetworkImpl.class.getDeclaredField("id");
                idField.setAccessible(true);
                idField.set(network, networks.get(0).getId());
                Field initializedField = Network.NetworkImpl.class.getDeclaredField("initialized");
                initializedField.setAccessible(true);
                ((AtomicBoolean) initializedField.get(network)).set(true);
            }
            return network;
        } catch (NoSuchFieldException | IllegalAccessException e) {
            throw new RuntimeException(e);
        }
    }

Also, as discussed, creating a custom Network subclass would probably be more appropriate.