Grid5000

On the Grid5000 testbed experiment platform there is no first class support for Nix (as of now).

NixOSCompose relies at least on the presence of the Nix store on the frontend commands that are not init and build and access to the Nix package manager for those two. To have it, we need to use some workarounds.

Except for the init and build commands (see details here), that will requiere a local installation of Nix followed by a chroot, the rest of the commands are only relying on Python. That means that for building your distributed environments you will need the reservation of resource.

Nix installation on frontend

We are going to use the script nix-user-chroot.sh.

This script provides Nix without root permissions on the frontend. At its first call it will install the Nix package manager and create all required folders in your home folder (e.g. ~/.nix/store/) then change the apparent root directory so that the nix command works as expected. All subsequent call to this script will only do the second part followed by the activation of the nixcommand.

curl -L -O https://raw.githubusercontent.com/oar-team/nix-user-chroot-companion/master/nix-user-chroot.sh
chmod +x nix-user-chroot.sh

On a interactive session on a node (using oarsub -I)

login@dahu-17:~$ ./nix-user-chroot.sh 
...
Activate Nix
login@dahu-17:~$

Tip

The call nix-user-chroot.sh can be done either on the frontend or on a node.

Doing it on the frontend would be the easy-route when you want to use the Nix store for small things (e.g. when using nix shells) but we advise to avoid building phases overall on the frontend (any commands that will populate the Nix store) to prevent performance issues for other users of the frontend.

Installation of NixOSCompose

The three following sections describe alternative approaches to install NixOSCompose.

Local installation with a poetry env

Install poetry locally in your home

curl -sSL https://install.python-poetry.org | python3 -

Clone NixOSCompose project then install all python dependencies thanks to Poetry. Finally enter the env.

git clone https://gitlab.inria.fr/nixos-compose/nixos-compose.git
cd nixos-compose
poetry env activate
poetry install
source "$( poetry env list --full-path | grep nixos-compose | grep py3.12 | cut -d' ' -f1 )/bin/activate"

For further use only source "$( poetry env list --full-path | grep nixos-compose | grep py3.12 | cut -d' ' -f1 )/bin/activate" is needed.

DEPRECATED: Python virtualenv with poetry then poetry env

Caution

Due to Grid5000’s installed python version (3.9) being out-of-date, this method is currently unavailable.

Similar to the method above but avoid the poetry instalation in your home directory and instead does it in a virtualenv.

git clone https://gitlab.inria.fr/nixos-compose/nixos-compose.git
cd nixos-compose
python3 -m venv /path/to/nxcEnv
. /path/to/nxcEnv/bin/activate
pip install poetry
poetry install
deactivate

Drops you in the virtualenv with NixOSCompose available.

Nix chroot on frontal with nix develop

Note

This method prevents the use of all OAR tooling (oarsub).

As seen in the local installation it is preferable to use the same nxc version that the one used in the project. The downside of this method is that you are loosing access to the specific commands of the platform (e.g. oarsub command is not acessible in a nix-shell) so you need either do your reservation of ressources before entering those commands or use multiple terminal or a terminal multiplxer (for instance tmux).

./nix-user-chroot.sh #activate Nix
nix develop .#nxcShellLite