Slurm Factory

Slurm HPC

⚡

Spack Packages

Slurm Factory Documentation

Relocatable HPC cluster builder with optimized caching and portable packages

Get Started View on GitHub

Slurm Factory is a modern Python CLI tool built with Typer that automates the building of relocatable Slurm workload manager packages using LXD containers and the Spack package manager. It features a modular architecture with comprehensive exception handling, intelligent caching, and portable package generation for HPC environments.

Key Features

• 🏗️ Modern Architecture: Built with Typer CLI framework and modular Python design
• 📦 Relocatable Packages: Runtime path configuration for cross-environment deployment
• ⚡ Intelligent Caching: Multi-layer build caching for ultra-fast rebuilds
• 🔧 Exception Handling: Comprehensive error management with custom exception hierarchy
• 🧪 Tested: 100% test coverage with 112 passing tests using proper LXD mocking
• 🚀 GPU Support: CUDA-enabled builds for GPU-accelerated workloads

Quick Start

Installation

# Install from PyPI (recommended)
pip install slurm-factory

# Or install with pipx for isolation
pipx install slurm-factory

# Verify installation
slurm-factory --help

Prerequisites

# Install and configure LXD
sudo snap install lxd
sudo lxd init

Build Your First Relocatable Package

# Build latest Slurm with optimizations
slurm-factory build --slurm-version 25.05

# Build with GPU support
slurm-factory build --slurm-version 25.05 --gpu

# Build with custom configuration
slurm-factory build --slurm-version 25.05 --config-file custom.yaml

# Show available options
slurm-factory build --help

Deploy Anywhere

# Standard deployment
sudo mkdir -p /opt/slurm
sudo tar -xzf ~/.slurm-factory/builds/slurm-25.05-software.tar.gz -C /opt/slurm/
sudo tar -xzf ~/.slurm-factory/builds/slurm-25.05-module.tar.gz -C /usr/share/lmod/lmod/modulefiles/

# Load with default path
module load slurm/25.05

# Or deploy to custom location  
sudo tar -xzf ~/.slurm-factory/builds/slurm-25.05-software.tar.gz -C /shared/apps/
export SLURM_INSTALL_PREFIX=/shared/apps/software
module load slurm/25.05

Key Features

Use Cases

• HPC Cluster Deployment: Standardized Slurm installations across heterogeneous clusters
• Development Environments: Quick Slurm setup for testing and development
• Multi-Version Support: Running different Slurm versions side-by-side
• Performance Testing: Optimized builds for specific hardware configurations
• Container Deployment: Portable packages for containerized HPC environments

Architecture Overview

┌─────────────────┐    ┌──────────────┐    ┌─────────────────┐
│   slurm-factory │───▶│ LXD Container│───▶│ Spack Build     │
│   CLI Tool      │    │ (Ubuntu 24.04)│    │ Environment     │
└─────────────────┘    └──────────────┘    └─────────────────┘
                                                      │
                                                      ▼
┌─────────────────┐    ┌──────────────┐    ┌─────────────────┐
│ Target HPC      │◀───│ Portable     │◀───│ Optimized       │
│ Cluster         │    │ Packages     │    │ Slurm Build     │
└─────────────────┘    └──────────────┘    └─────────────────┘

Package Information

Documentation Structure

Latest Features

• Multi-Version Support: Build and deploy Slurm versions 25.05, 24.11, 24.05, 23.11, 23.02
• GPU Optimization: Optional CUDA support for GPU-enabled HPC clusters
• Portable Packages: Self-contained deployments with module system integration
• Modern Architecture: LXD containers with Spack package management
• Performance Focused: CPU-optimized builds with minimal package sizes

Built with ❤️ by Vantage Compute