runOptions not respected for Docker and Google Batch

[VasLem]

Bug report

Neither nextflow.config runOptions and envWhileList arguments are considered when running on Google Batch.

Expected behavior and actual behavior

To assess this, have a docker image with an entrypoint expecting an environment variable:
entrypoint.sh

#!/bin/bash
if [ -z "$VARIABLE_LOADED" ]; then
    echo -e "Error: Environment variable VARIABLE_LOADED is not set."
    exit 1
fi
exec "$@"

and use this entrypoint in this example Dockerfile:

FROM ubuntu:22.04
SHELL ["/bin/bash", "-c"]
USER root
ENV HOME=/home/root
WORKDIR $HOME
COPY entrypoint.sh /home/root/entrypoint.sh
ENTRYPOINT "/home/root/entrypoint.sh"

to build and push an image called test on ${CONTAINERS_REPO}/${CONTAINERS_PROJECT_ID}/${CONTAINERS_REGISTRY}.

Then, have the following docker configuration in nextflow.config:

docker {
      enabled = true
      runOptions = '-e VARIABLE_LOADED=true'
  }
process {
    container = "${CONTAINERS_REPO}/${CONTAINERS_PROJECT_ID}/${CONTAINERS_REGISTRY}/test:latest"
}
profiles {
  local {
    process {
        executor = 'local'
    }
  }
  googlebatch {
    process {
            executor = 'google-batch'
            machineType = 'e2-small'
            disk = '20GB'
        }
  }
}

as well as the configuration required to run the process on Google Batch.
Finally, have this example process and workflow:

process check {
    script:
        """
        echo "Success"
        """
}

workflow {
    check()
}

This workflow fails on Google Batch (with the process unexpectedly terminating, without any error), i.e., when using googlebatch profile, but succeeds locally, (with local profile).

I noticed, after looking at the nf-google code, and in particular

nextflow/plugins/nf-google/src/main/nextflow/cloud/google/batch/GoogleBatchTaskHandler.groovy

Line 243 in cc0163a

def containerOptions = task.config.getContainerOptions() ?: ''

, that if instead I override the process containerOptions in the configuration, the workflow works on Google Batch. That is setting:

 googlebatch {

    process {
      containerOptions = "-e VARIABLE_LOADED=true"
    }
}

I believe this is kind of a serious bug. Also, it was extremely difficult to follow the entire process, as there is nowhere in .command.run anything about how the container is run, everything is being obfuscated in the following cryptic nxf_launch in .command.run, which just calls itself (?):

nxf_launch() {
    /bin/bash /mnt/disks/WORKDIR_BUCKET/ca/fa862f1da1db063082fd752027869d/.command.run nxf_trace
}

Environment

• Nextflow version: 24.11.0-edge