Generalization of the composition

As you can see from the previous section, the scaling of the number of computing node is a bit cumbersome.

Fortunately, NixOS Compose provides the notion of role to tackle this issue.

A role is a configuration. In our case, we actually have only two roles: the NFS server and the compute nodes. The configuration of the compute nodes is the same no matter how many compute nodes. Thus having to define the configuration for node1 and node2 is redundant.

roles = {

  node = { pkgs, ... }:
  {
    # add needed package
    environment.systemPackages = with pkgs; [ openmpi ior ];
  
    # Disable the firewall
    networking.firewall.enable = false;
  
    # Mount the NFS
    fileSystems."/data" = {
      device = "server:/";
      fsType = "nfs";
    };
  };

  server = { pkgs, ... }:
  { # ... };
}

Building

nxc build -f g5k-nfs-store

Deploying

Reserving the resources

export $(oarsub -l nodes=3,walltime=1:0:0 "$(nxc helper g5k_script) 1h" | grep OAR_JOB_ID)

Getting the machine file

oarstat -u -J | jq --raw-output 'to_entries | .[0].value.assigned_network_address | .[]' > machines

Starting the nodes

The nxc start command can take an additional yaml file as input describing the number of machines per role, as well as their hostnames.

nxc start -m machines nodes.yaml

The following yaml file will create 3 machines: server, node1 and node2.

# nodes.yaml
node: 2
server: 1

You can specify the hostnames in the yaml file:

# nodes.yaml
node:
    - foo
    - bar
server: 1

The above yaml file will create 3 machines: server, foo and bar.