Creating a service

Currently, the creation of the GlusterFS volume is manual, but we would like to do it automatically at boot time.

To do this, we will create a systemd service and use it in the composition.

In NixOS, a service (or module) is composed of two parts: the interface and the implementation.

In the previous sections, you already interacted with services ! For example:

  ...
  services.glusterfs.enable = true;
  ...

Creating the module

Let's create a new file to store the content of the service:

# my-module.nix
{ config, lib, pkgs, ... }:

with lib;
let
  cfg = config.services.my-glusterfs;
in
{
  ################################################
  #
  # Interface
  #
  options = {
    services.my-glusterfs = {
      enable = mkEnableOption "My glusterfs";

      package = mkOption {
        type = types.package;
        default = pkgs.glusterfs;
      };

      volumePath = mkOption {
        type = types.str;
        default = "/srv";
      };

      volumeName = mkOption {
        type = types.str;
        default = "gv0";
      };
    };
  };

  ################################################
  #
  # Implementation
  #
  config = mkIf (cfg.enable) {
    systemd.services.my-glusterfs = {
      description = "My GlusterFS module";
      wantedBy = [ "multi-user.target" ];
      after = [ "glusterd.service" "glustereventsd.service" ];
      serviceConfig.Type = "oneshot";
      script =
        ''
        if [ ! $(${cfg.package}/bin/gluster volume list | grep ${cfg.volumeName}) ]
        then
          mkdir -p ${cfg.volumePath}/${cfg.volumeName}
          ${cfg.package}/bin/gluster volume create ${cfg.volumeName} server:${cfg.volumePath}/${cfg.volumeName}
          ${cfg.package}/bin/gluster volume start ${cfg.volumeName}
        fi
        '';
    };
  };
}

Ok, let's decypher what all of this means.

  • we created a service called my-glusterfs

  • the services has 4 options:

    • enable : to enable or not the service

    • package : the Nix package containing the glusterfs binaries

    • volumePath : the path on the server where the volume will be created

    • volumeName : the name of the volume

Then, in the implementation part, we explicit:

  • that this service is wanted by the multi-user service

  • that this service needs to be executed after that the GlusterFS deamon has been started (glusterd.service and glustereventsd.service)

  • that this service must only be ran once (oneshot)

  • and finally, the commands to run. Here the commands are the same as the ones seen in the previous section, but we are using the configuration of the service: cfg.VolumePath, cfg.VolumeName.

Call this service

Let's now use this service in the composition.

server = { pkgs, ... }: {
  # We import the definition of the service
  imports = [ ./my-module.nix ];

  services.my-glusterfs = {
    enable = true;        # We activate our service
    volumePath = "/srv"; # We define where the volume will be
    volumeName = "gv0";   # and the name of the volume
  };

  networking.firewall.enable = false;
  services.glusterfs.enable = true;
  fileSystems = {
    "/srv" = {
      device = "/dev/disk/by-partlabel/KDPL_TMP_disk0";
      fsType = "ext4";
    };
  };
  environment.systemPackages = with pkgs; [ htop ];
};

Now, everything the server boots, it will create the volume and start it so that it should be available for the nodes to mount.

Building

nxc build -f g5k-nfs-store

Deploying

Reserving the resources

This time two nodes.

export $(oarsub --project lab-2025-compas-nxc -l nodes=2,walltime=1:0:0 "$(nxc helper g5k_script) 1h" | grep OAR_JOB_ID)

Starting the nodes

nxc start -m OAR.$OAR_JOB_ID.stdout -W

Connect

nxc connect

Re-mount the volumes on the node

On the node:

systemctl restart data.mount

You can now use the volume from the node !