382 questions
0
votes
1
answer
52
views
How to bind-mount folders for Apptainer in Snakemake workflow
My Snakemake workflow processes one big files which is stored in a POSIX folder. The rules have been crafted so that a symlink to the input file is created in the workdir, and then the processing is ...
- 909
1
vote
0
answers
30
views
How to use yum inside the %post section of Apptainer definition file?
I haven't gotten it to work even though the same command works fine on my host machine.
This is what I use in the definition file (all package dependencies have been downloaded into the same folder ...
- 137
0
votes
0
answers
442
views
ERROR: Will not install to the user site because it will lack sys.path precedence ... using venv inside docker
I am having troubles building docker from ubuntu:24.04, Python version 3.12.3 and pip 25.1.1.
Since the recent (I think) defualt pip behavior throwing
error: externally-managed-environment
when ...
- 21
0
votes
0
answers
66
views
rocker/tidyverse SSL errors on apptainer
When using the rocker/tidyverse:4.4 image with apptainer, anything using SSL fails. This does not happen in rocker/tidyverse:4.3.2 nor does it happen when using this image from Docker instead of ...
- 311
0
votes
1
answer
155
views
Slurm starting job without gpu memory requirements being met
I have a singularity image that I need to run on a slurm cluster.
Now i run into an issue that the gpu memory requirements seem to be ignored by slurm, several of my jobs get started and a few seconds ...
- 146
0
votes
0
answers
68
views
provision a custom python virtualenv with apptainer
In order to run a software that require 32 bit environment on HPC, I have to build a container with Apptainer.
The problem is, I need to run a python script with extra dependencies in the container.
I ...
- 1,987
0
votes
1
answer
41
views
Should I loop a container or loop inside a container?
I want to call genetic variants with DeepVariant on an HPC for about 1000 cereal lines. I successfully ran DV for one line with the docker image they provide using Apptainer/Singularity, but for the ...
- 65
0
votes
0
answers
76
views
Running Nice DCV inside a singularity container in an EC2 instance (g4dn.2xlarge)
I have tried multiple things but nothing seems to be working. I managed to do it inside a docker container luckily there is documentation available for the same. But with singularity its a whole ...
3
votes
0
answers
114
views
Load library directory error (R, Julia and container)
I am using an R script with Julia functions to run the code. It works perfectly on my computer, but when I try to set it up in the apptainer, it gives me an error. I've created a container (ubuntu 22....
2
votes
1
answer
205
views
How to pass host environment variables to singularity build?
system: Ubuntu 22.04.5 LTS
In my institution there is an internal proxy. Because of that I need to add to my .bashrc something like:
export wss_proxy="VALUE"
export http_proxy="VALUE&...
- 31
0
votes
0
answers
75
views
Binding single file in Singularity apptainer
I want to bind a single .json file to my apptainer container instead of binding the directory in which the .json file lies. However, when I try to bind the file like this:
--bind /PATH/config.json:/...
- 3
1
vote
0
answers
190
views
Apptainer not using GPU
I am trying to run an apptainer container and I am very new to Apptainer/Docker/containerization. I did not build the container myself, I received it from someone so I also don't have writing access.
...
- 11
1
vote
1
answer
83
views
Syntax error in R code executed with Slurm and Singularity
This might be a very stupid question but I can't find a solution. I trying to use srun from SLURM which run in a singularity container a simple R script like so :
project_dir=$HOME/project_directory/
...
- 25
0
votes
0
answers
68
views
Singularity Container not finding files after transformation from docker
I have a docker image like this
FROM nvidia/cuda:12.6.3-runtime-ubuntu20.04
#Set environment variables to avoid some warnings during apt-get install
ENV DEBIAN_FRONTEND=noninteractive
#Install ...
- 146
1
vote
1
answer
49
views
Intall python package with pip, respecting current installation
I want to use a singularity container with jax preinstalled e.g. based on this image. On top of that, I want to install PyMC with numpyro, to get NUTS sampling with GPU acceleration.
My attempt is to ...
- 4,944
1
vote
1
answer
66
views
Is the singularity environment preserved for Python's subprocess?
I am working on a Python program and I wish to use Python's subprocess's run/Popen functions. I am working on a singularity environment and I found the following post which asks somewhat of a similar ...
1
vote
1
answer
281
views
Singularity : run container and preserving env variables
I would like to run a docker image with singularity exec.
I would like to preserve the env variables of the docker container notably the PATH variable. For that I used --envclean. However, thought ...
- 799
0
votes
1
answer
98
views
how to find the executable file in sigularity sif
I have pulled a docker file and have built a singularity (.sif) image.
When I run the following command:
singularity exec software.sif command
I get the error below:
FATAL: "command": ...
- 2,224
0
votes
1
answer
209
views
Apptainer/Singularity Container built in local dev environment functions well, but does not work at all on cluster. How can I troubleshoot this?
I have built a singularity/apptainer container that I am trying to deploy in a cluster computing environment. The container is built by pulling from a docker image with the following command:
...
- 25
0
votes
1
answer
365
views
Starting Nextflow in a singularity container and use it to run another singularity container
I want to use nextflow in a singularity container that will further run my nextflow pipelines packed in a singularity. This will help us bypass installation of nextflow in all our test servers. I am ...
1
vote
1
answer
175
views
Is there a way to run snakemake within a singularity container with computing cluster submission for individual rules?
Is there a way to get the rules submitted to the cluster to inherit the parent command singularity container?
I'm running the base snakemake within a container for portability with most rules using ...
- 85
0
votes
1
answer
582
views
singularity failing due to error squashfs filesystem seems not enabled
Singularity version 4.1.4-1 and squashfs-tools version 4.3 are installed on RHEL 7.9 (Maipo).
Running a container fails with the following message:
(base) -bash-4.2$ singularity --verbose shell ...
1
vote
1
answer
87
views
Singularity/Nexflow script won't load Tensorflow from .sif on cluster
I'm trying to run a singularity/nextflow script on an HPC. This script utilizes tensorflow, which is specified in the docker image I used to pull the initial .sif file from lpryszcz/deeplexicon:latest....
- 11
0
votes
0
answers
119
views
Singularity container with OpenMPI and InfiniBand (UCX)
I'm working as an intern currently, and I was asked to build a Singularity container for OpenMPI to make distributed programming possible on multiple machines of our HPC cluster using containers.
...
0
votes
1
answer
487
views
Singularity build crashes because of pip memory usage
I am currently trying to build quite a simple .sif image from a .def file with singularity 4.1.3.
The problem is that whatever settings I try to use for pip install, I always fail the build due to:
...
- 21
2
votes
1
answer
5k
views
How to correctly activate micromamba environment inside Apptainer container automatically?
I am trying to build a micromamba based container, that automatically activates the correct environment to run an application (here jupyter-lab).
Apptainer.def
Bootstrap: docker
From: mambaorg/...
- 9,305
0
votes
1
answer
524
views
How to Run Commands in a Singularity from Shell Script
I wish to run a Python script in a singularity container.
In particular, I have attempted writing the following in the shell script
singularity shell <name.sif>
python3 <python-script.py>
...
0
votes
0
answers
221
views
Spack Error: Additional properties are not allowed
I am trying to build lbann in a container. The lbann package is installed within the container using spack in such a way that it makes use of the openmpi package built from source within the container....
- 75
0
votes
1
answer
65
views
Issue with Singularity Command Recognition for Fastq.gz Files
I am developing a shotgun metagenomics pipeline using Snakemake and Singularity, but I'm facing difficulties when trying to execute the Singularity command within Snakemake to analyze fastq.gz files. ...
0
votes
1
answer
136
views
The /etc directory appears empty when I try to access files using R
I have a bash script where I use singularity exec to run a program from inside the container. I want to call this script in R using the system() function. Everything works fine until Singularity needs ...
- 47
0
votes
0
answers
81
views
Call singularity exec from a shell script in R
I have a bash script where I use singularity exec to run a program from inside the container. I want to call this script in R using the system() function. Everything works fine until it reaches the ...
- 47
1
vote
1
answer
72
views
--no-home Singularity flag doesn't work on Docker container in a cluster node
When I use a Docker container on a HPC cluster's login machine using Singularity, it works as expected and I can access the container's /home:
singularity exec docker://quay.io/comparative-genomics-...
- 463
0
votes
0
answers
38
views
How to run piped command with Python subprocess without deadlocking?
I have a python script that runs
for cmd in chr_commands:
info("Run: "+cmd)
subprocess.run(cmd)
Where each cmd is equivalent to
samtools view -F 0x0004 input.bam chr3:0-500000 2>&...
- 8,864
1
vote
0
answers
648
views
conveyor failed while building an apptainer image from defintion file
I am trying to run inference results from https://github.com/mlcommons/inference_results_v3.1/blob/main/closed/Intel/code/gptj-99/pytorch-cpu/docker/Dockerfile_int4 which uses docker however instead ...
- 1,402
1
vote
2
answers
194
views
Bind and run a Python module with singularity from a remote directory
I need to run a personnal Python module from a singularity container.
I usually run it with the following command :
singularity run ./Image/myimage.sif -m mymodule.start
However it's only work if I ...
- 61
1
vote
1
answer
1k
views
Encountered Errors in Storing Images using Singularity in Cluster
I am trying to get myself familiar with operations related to our cluster, so I tried using singularity there. I am having issues with pulling a specific docker image into singularity in the cluster. ...
- 51
0
votes
1
answer
1k
views
singularity exec is not recognizing executables in container $PATH (converted micromamba docker image to singularity image)
There is a similar post but this post is regarding building a singularity container: Singularity containers: adding custom packages to $PATH and pass it to singularity exec
I have converted a Docker ...
- 31.1k
-1
votes
1
answer
266
views
singularity build cannot install build-essentials
I am trying to build a singularity container (recipe file at the end of this post) and it is not able to install build-essentials. Execution fails with:
E: Unable to locate package build-essentials
...
- 161
1
vote
0
answers
285
views
I cannot execute a program in a container when working with Apptainer becaouse the path is not found
I have pulled a Docker container to create the .sif file
apptainer pull docker://mblanche/ngmerge:latest
there is no documentation so I have explored the sif file to know where is the tool I want to ...
- 1,403
1
vote
0
answers
229
views
Running c++ on cluster with slurm and sif container
I'm attempting to execute a C++ code on a cluster that utilizes Slurm. Since the server did not have g++ installed, I created a Singularity container (SIF) to facilitate the execution. However, I've ...
- 11
1
vote
1
answer
197
views
Intermediate build checkpoints for singularity containers?
I have to build a container with singularity/apptainer. However, I'm in the process of writting the definition file, which needs some iteration and building the image is pretty long;
I'm pretty new to ...
- 31
0
votes
0
answers
683
views
Singularity container cannot find files after being created from a Docker image
I am creating a Singularity container from a Docker image. However, the Singularity container gives me errors that weren't present while I used Docker.
This is how I create my Docker image -
FROM ...
- 1,859
0
votes
1
answer
74
views
loop through list element one at the time
Hi there I've been experimenting with a job array using Singularity, the problem is that singularity takes in input only global/environment variables within the container — to the best of my knowledge ...
- 537
0
votes
1
answer
184
views
Toil-cwl-runner not using default-container option with singularity option
Issue
We are attempting to run toil-cwl-runner with a default singularity image. We got this to work as we intended with cwltool, but toil-cwl-runner seems like it is ignoring the default image to use....
0
votes
0
answers
1k
views
How to activate conda and a specific conda env for each shell and subshell in a singularity container
I have been trying this for a few days and no success. I am building a Singularity container (version 4.0.2-jammy), and I want to use conda within it. I am able to install conda, create the ...
- 101
0
votes
1
answer
894
views
Running a command within a launched shell within an apptainer instance
I'm working towards setting up a script for batch submission to an HPC system which uses software in an apptainer image. Here's what I have in mind:
apptainer run container.sif runjob.sh
runjob.sh
----...
- 67
2
votes
1
answer
2k
views
How to start a conda/mamba environment in a container during execution
I would like to do exec on my container and execute my command:
apptainer exec myContainer.sif myCommand
(or singularity)
but myCommand is within a conda/mamba environment, so I am doing:
apptainer ...
- 115
2
votes
0
answers
822
views
FATAL: While performing build: conveyor failed to get: while converting reference: pinging container registry registry-1.docker.io:
I install apptainer by following the instructions from https://github.com/apptainer/apptainer/issues/url. After fresh installation "apptainer build" command gives error while building from ....
- 253
0
votes
1
answer
217
views
Apptainer build gets stuck at "Geographic area" selection
I tried to build an apptainer image from the following definition file:
Bootstrap: docker
From: pytorch/pytorch:2.0.1-cuda11.7-cudnn8-devel
%files
./requirements.txt /src/requirements.txt
%post
...
- 347
0
votes
1
answer
587
views
How to set a shell variable in a Singularity definition file so that it is seen by several sections
I need to set a version string for an app that I am packaging in a Singularity container. This version string is used in several places in the definition file (*.def).
I know that I can set a shell ...
- 9,810