Is Quality Enough? Integrating Energy Consumption in a Large-Scale Evaluation of Neural Audio Synthesis Models
Résumé
In most scientific domains, the deep learning community has
largely focused on the quality of deep generative models, resulting
in highly accurate and successful solutions. However,
this race for quality comes at a tremendous computational
cost, which incurs vast energy consumption and greenhouse
gas emissions. At the heart of this problem are the measures
that we use as a scientific community to evaluate our work.
In this paper, we suggest relying on a multi-objective measure
based on Pareto optimality, which takes into account both
the quality of the model and its energy consumption. By applying
our measure on the current state-of-the-art in generative
audio models, we show that it can drastically change the
significance of the results. We believe that this type of metric
can be widely used by the community to evaluate their
work, while putting computational cost – and in fine energy
consumption – in the spotlight of deep learning research.