Skip to main content

Installing slurm-factory Tool

This guide covers how to install the slurm-factory build tool - a Python CLI for building custom Slurm packages.

Note: If you just want to install pre-built Slurm binaries, you don't need this tool. See Installing Slurm from Buildcache instead.

What is slurm-factory?​

slurm-factory is a Python command-line tool that:

  • Builds custom Slurm packages from source
  • Uses Docker for isolated, reproducible builds
  • Leverages Spack for dependency management
  • Creates relocatable tarballs for easy deployment
  • Can publish to buildcaches with GPG signing

Install slurm-factory when you need:

  • Custom build configurations
  • To build for unsupported OS/architectures
  • To create your own buildcache
  • Full control over the build process

Don't install slurm-factory if:

  • You just want to use pre-built binaries (see here)
  • You don't need custom builds
  • You want the fastest deployment (buildcache is 5-15 min vs 45-90 min building)

Prerequisites​

Before installing Slurm Factory, you need to have Docker installed and configured:

# Install Docker
curl -fsSL https://get.docker.com | sh

# Add your user to the docker group (avoids needing sudo)
sudo usermod -aG docker $USER
newgrp docker

# Verify Docker is working
docker --version
docker run hello-world

System Requirements:

  • OS: Ubuntu 20.04+ or equivalent Linux distribution
  • Docker: 24.0+ (latest stable version recommended)
  • Python: 3.12+ (required for modern type hints and features)
  • Memory: 4GB+ RAM recommended for builds (16GB+ optimal)
  • Storage: 50GB+ free space for build cache and packages

Installation Methods​

The easiest way to install Slurm Factory is from PyPI using pip:

# Install slurm-factory from PyPI
pip install slurm-factory

# Verify installation and check CLI commands
slurm-factory --help

# Test with version info
slurm-factory build-slurm --help

Option 2: Install with pipx (Isolated Environment)​

For an isolated installation that doesn't interfere with your system Python packages:

# Install pipx if you don't have it
pip install pipx
pipx ensurepath

# Install slurm-factory with pipx in isolated environment
pipx install slurm-factory

# Verify installation
slurm-factory --help

# Upgrade when needed
pipx upgrade slurm-factory

Option 3: Install from Source (Development)​

For development or to get the latest features before release:

# Install uv (modern Python package manager)
curl -LsSf https://astral.sh/uv/install.sh | sh

# Clone the repository
git clone https://github.com/vantagecompute/slurm-factory.git
cd slurm-factory

# Install dependencies and create virtual environment
uv sync

# Run slurm-factory in development mode
uv run slurm-factory --help

# Run tests to verify installation
uv run pytest

Option 4: Development with Justfile​

For active development with our task runner:

# Clone and enter repository
git clone https://github.com/vantagecompute/slurm-factory.git
cd slurm-factory

# Install just task runner
curl --proto '=https' --tlsv1.2 -sSf https://just.systems/install.sh | bash -s -- --to ~/.local/bin

# Install dependencies
just install

# Run tests
just unit

# Build package locally
just build

# See all available tasks
just --list

Verification​

After installation, verify that Slurm Factory is working correctly:

# Check CLI is working
slurm-factory --help

# Check available commands
slurm-factory build-slurm --help
slurm-factory clean --help

# Verify Docker integration (should show Docker version)
docker --version

# Test basic functionality (builds a Slurm package)
slurm-factory build-slurm --slurm-version 25.11

Configuration​

Environment Variables​

Slurm Factory supports these environment variables:

# Set default project name (used in container naming)
export IF_PROJECT_NAME=my-slurm-builds

# Now all commands use this project by default
slurm-factory build-slurm --slurm-version 25.11

Cache Directory​

Build caches are stored in your home directory:

# Default cache location
~/.slurm-factory/
├── builds/           # Built packages (TAR files)
├── spack-buildcache/ # Compiled package binaries
├── spack-sourcecache/# Downloaded source archives
├── binary_index/     # Dependency resolution cache
└── ccache/          # Compiler object cache

Quick Start​

Once installed, you can immediately start building Slurm packages:

# Build latest Slurm with default compiler (GCC 13.4.0)
slurm-factory build-slurm --slurm-version 25.11

# Build with specific compiler for RHEL 8 / Ubuntu 20.04 compatibility
slurm-factory build-slurm --slurm-version 25.11 --compiler-version 10.5.0

# Build for RHEL 7 compatibility
slurm-factory build-slurm --slurm-version 24.11 --compiler-version 7.5.0

# Build with GPU support (CUDA/ROCm)
slurm-factory build-slurm --slurm-version 25.11 --gpu

# Use custom project name (for container naming)
slurm-factory --project-name production build-slurm --slurm-version 25.11 --compiler-version 13.4.0

# Clean up when done
slurm-factory clean --full

Available Versions:

  • Slurm: 25.11, 24.11, 23.11
  • Compilers: 14.2.0, 13.4.0, 12.5.0, 11.5.0, 10.5.0, 9.5.0, 8.5.0, 7.5.0

See Build Artifacts for pre-built S3 packages.

Troubleshooting​

Docker Permission Issues​

# Add your user to the docker group
sudo usermod -aG docker $USER

# Log out and back in, or use newgrp
newgrp docker

# Test Docker access
docker ps

Python Path Issues​

# If command not found after pip install
pip install --user slurm-factory
export PATH=$PATH:~/.local/bin

# Or use python -m for direct module execution
python -m slurm_factory --help

Storage Issues​

# Check available space (needs 50GB+)
df -h ~/.slurm-factory/

# Clean cache if needed
slurm-factory clean --full

# Or manually clean cache directory
rm -rf ~/.slurm-factory/spack-buildcache/

Next Steps​