Researchers from the German Research Center for Artificial Intelligence (DFKI) presented groundbreaking work at the 38th GI/ITG International Conference on Architectures of Computing Systems (ARCS 2025). Their paper, titled “Spend More to Save More (SM²): An Energy-Aware Implementation of Successive Halving for Sustainable Hyperparameter Optimization” introduces a novel approach that embeds energy-efficiency directly into the hyperparameter tuning process for machine learning models.
This research builds on DFKI's ongoing commitment to sustainable AI, following their impactful presentation at ARCS 2024 of the paper, “The Power of Training: How Different Neural Network Setups Influence the Energy Demand”, The full proceedings of ARCS 2024 can be found here.
Introducing SM²: Spend More to Save More
Hyperparameter optimization (HPO) is a key phase in building performant ML models. It involves testing multiple configurations—such as batch sizes, learning rates, and model depths—to find the most accurate combination. However, this process can be extremely energy-intensive, requiring repeated model training and evaluation.
SM² (Spend More to Save More) addresses this inefficiency by introducing a two-stage process that accounts for both performance and energy consumption:
- Exploration Phase: Instead of fully training every configuration, SM² begins with a short, single-epoch training run on a small dataset. During this step, it monitors the system’s power usage in real time.
- Selection Phase: Only the most promising configurations—those that balance high accuracy and low energy use—are selected for full training. The remaining options are discarded early, saving computational resources.
This process is built on an adaptation of the Successive Halving Algorithm (SHA), repurposed here with a sustainability-first approach.
A Hybrid Strategy for Sustainable Optimization
Unlike conventional HPO methods that focus solely on performance, SM² evaluates each configuration through a hybrid objective that combines model accuracy and energy efficiency. The algorithm progressively narrows down candidates through successive comparisons, ensuring that only configurations with the best accuracy-to-energy ratio proceed.
This makes SM² especially suitable for large-scale experiments and energy-conscious deployments, where resources are limited or sustainability is a priority.
Real-Time Hardware Monitoring and Adaptation
A unique feature of SM² is its tight integration with system-level hardware monitoring. It leverages real-time energy measurements from the execution device—be it CPU, GPU, or emerging Processing-In-Memory (PIM) architectures—to make pruning decisions grounded in real consumption data.
This means SM² can:
- Adjust its behavior depending on the energy profile of each hardware type.
- Provide hardware-aware optimization, maximizing performance without overspending energy budgets.
- Offer transparent metrics to inform sustainable choices early in the ML development lifecycle.
Experimental Validation
DFKI validated SM2 through rigorous experiments across multiple ML scenarios, including:
- A ResNet-18 model on CIFAR-10 (image classification)
- An LSTM model on an energy-usage prediction dataset
- A Transformer model on WikiText-2 (language modeling)
Strategic Alignment with SustainML
The SM² framework echoes the core goals of the SustainML project—placing sustainability at the heart of machine learning development. By integrating energy efficiency into the hyperparameter tuning process, considering hardware-specific consumption, and promoting measurable environmental impact, SM² supports SustainML’s vision of a transparent, responsible, and resource-aware ML lifecycle.
Towards Greener Machine Learning
DFKI’s presentation at ARCS 2025 reinforces the growing momentum behind sustainable AI practices. By demonstrating that smarter HPO strategies can both reduce energy usage and preserve model quality, SM² offers a compelling path forward.
MORE INFORMATION ABOUT SUSTAINML:
If you have any questions, please get in touch with
This project has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement No 101070408.