Sie haben Javascript deaktiviert!
Sie haben versucht eine Funktion zu nutzen, die nur mit Javascript möglich ist. Um sämtliche Funktionalitäten unserer Internetseite zu nutzen, aktivieren Sie bitte Javascript in Ihrem Browser.

Change of Perspective. Show image information

Change of Perspective.

Photo: Paderborn University

Alexander Tornede

 Alexander Tornede

Intelligent Systems and Machine Learning

Member - Research Associate

Sonderforschungsbereich 901

Member - Research Associate

+49 5251 60-3352
Zukunftsmeile 2
33102 Paderborn

Open list in Research Information System


AutoML for Multi-Label Classification: Overview and Empirical Evaluation

M.D. Wever, A. Tornede, F. Mohr, E. Hüllermeier, IEEE Transactions on Pattern Analysis and Machine Intelligence (2021), pp. 1-1

Automated machine learning (AutoML) supports the algorithmic construction and data-specific customization of machine learning pipelines, including the selection, combination, and parametrization of machine learning algorithms as main constituents. Generally speaking, AutoML approaches comprise two major components: a search space model and an optimizer for traversing the space. Recent approaches have shown impressive results in the realm of supervised learning, most notably (single-label) classification (SLC). Moreover, first attempts at extending these approaches towards multi-label classification (MLC) have been made. While the space of candidate pipelines is already huge in SLC, the complexity of the search space is raised to an even higher power in MLC. One may wonder, therefore, whether and to what extent optimizers established for SLC can scale to this increased complexity, and how they compare to each other. This paper makes the following contributions: First, we survey existing approaches to AutoML for MLC. Second, we augment these approaches with optimizers not previously tried for MLC. Third, we propose a benchmarking framework that supports a fair and systematic comparison. Fourth, we conduct an extensive experimental study, evaluating the methods on a suite of MLC problems. We find a grammar-based best-first search to compare favorably to other optimizers.

    Predicting Machine Learning Pipeline Runtimes in the Context of Automated Machine Learning

    F. Mohr, M.D. Wever, A. Tornede, E. Hüllermeier, IEEE Transactions on Pattern Analysis and Machine Intelligence (2021)

    Automated Machine Learning (AutoML) seeks to automatically find so-called machine learning pipelines that maximize the prediction performance when being used to train a model on a given dataset. One of the main and yet open challenges in AutoML is an effective use of computational resources: An AutoML process involves the evaluation of many candidate pipelines, which are costly but often ineffective because they are canceled due to a timeout. In this paper, we present an approach to predict the runtime of two-step machine learning pipelines with up to one pre-processor, which can be used to anticipate whether or not a pipeline will time out. Separate runtime models are trained offline for each algorithm that may be used in a pipeline, and an overall prediction is derived from these models. We empirically show that the approach increases successful evaluations made by an AutoML tool while preserving or even improving on the previously best solutions.

      Automated Machine Learning, Bounded Rationality, and Rational Metareasoning

      E. Hüllermeier, F. Mohr, A. Tornede, M.D. Wever, 2021

      Coevolution of Remaining Useful Lifetime Estimation Pipelines for Automated Predictive Maintenance

      T. Tornede, A. Tornede, M.D. Wever, E. Hüllermeier, in: Proceedings of the Genetic and Evolutionary Computation Conference, 2021


      Extreme Algorithm Selection with Dyadic Feature Representation

      A. Tornede, M.D. Wever, E. Hüllermeier, in: Discovery Science, 2020

      Hybrid Ranking and Regression for Algorithm Selection

      J.M. Hanselle, A. Tornede, M.D. Wever, E. Hüllermeier, in: KI 2020: Advances in Artificial Intelligence, 2020

      LiBRe: Label-Wise Selection of Base Learners in Binary Relevance for Multi-Label Classification

      M.D. Wever, A. Tornede, F. Mohr, E. Hüllermeier, Springer, 2020

      In multi-label classification (MLC), each instance is associated with a set of class labels, in contrast to standard classification where an instance is assigned a single label. Binary relevance (BR) learning, which reduces a multi-label to a set of binary classification problems, one per label, is arguably the most straight-forward approach to MLC. In spite of its simplicity, BR proved to be competitive to more sophisticated MLC methods, and still achieves state-of-the-art performance for many loss functions. Somewhat surprisingly, the optimal choice of the base learner for tackling the binary classification problems has received very little attention so far. Taking advantage of the label independence assumption inherent to BR, we propose a label-wise base learner selection method optimizing label-wise macro averaged performance measures. In an extensive experimental evaluation, we find that or approach, called LiBRe, can significantly improve generalization performance.

        Towards Meta-Algorithm Selection

        A. Tornede, M.D. Wever, E. Hüllermeier, in: Workshop MetaLearn 2020 @ NeurIPS 2020, 2020

        AutoML for Predictive Maintenance: One Tool to RUL Them All

        T. Tornede, A. Tornede, M.D. Wever, F. Mohr, E. Hüllermeier, in: Proceedings of the ECMLPKDD 2020, 2020

        Run2Survive: A Decision-theoretic Approach to Algorithm Selection based on Survival Analysis

        A. Tornede, M.D. Wever, S. Werner, F. Mohr, E. Hüllermeier, in: ACML 2020, 2020

        Algorithm selection (AS) deals with the automatic selection of an algorithm from a fixed set of candidate algorithms most suitable for a specific instance of an algorithmic problem class, where "suitability" often refers to an algorithm's runtime. Due to possibly extremely long runtimes of candidate algorithms, training data for algorithm selection models is usually generated under time constraints in the sense that not all algorithms are run to completion on all instances. Thus, training data usually comprises censored information, as the true runtime of algorithms timed out remains unknown. However, many standard AS approaches are not able to handle such information in a proper way. On the other side, survival analysis (SA) naturally supports censored data and offers appropriate ways to use such data for learning distributional models of algorithm runtime, as we demonstrate in this work. We leverage such models as a basis of a sophisticated decision-theoretic approach to algorithm selection, which we dub Run2Survive. Moreover, taking advantage of a framework of this kind, we advocate a risk-averse approach to algorithm selection, in which the avoidance of a timeout is given high priority. In an extensive experimental study with the standard benchmark ASlib, our approach is shown to be highly competitive and in many cases even superior to state-of-the-art AS approaches.


          Automating Multi-Label Classification Extending ML-Plan

          M.D. Wever, F. Mohr, A. Tornede, E. Hüllermeier, 2019

          Existing tools for automated machine learning, such as Auto-WEKA, TPOT, auto-sklearn, and more recently ML-Plan, have shown impressive results for the tasks of single-label classification and regression. Yet, there is only little work on other types of machine learning problems so far. In particular, there is almost no work on automating the engineering of machine learning solutions for multi-label classification (MLC). We show how the scope of ML-Plan, an AutoML-tool for multi-class classification, can be extended towards MLC using MEKA, which is a multi-label extension of the well-known Java library WEKA. The resulting approach recursively refines MEKA's multi-label classifiers, nesting other multi-label classifiers for meta algorithms and single-label classifiers provided by WEKA as base learners. In our evaluation, we find that the proposed approach yields strong results and performs significantly better than a set of baselines we compare with.

          Algorithm Selection as Recommendation: From Collaborative Filtering to Dyad Ranking

          A. Tornede, M.D. Wever, E. Hüllermeier, in: Proceedings - 29. Workshop Computational Intelligence, Dortmund, 28. - 29. November 2019, KIT Scientific Publishing, Karlsruhe, 2019, pp. 135-146

          From Automated to On-The-Fly Machine Learning

          F. Mohr, M.D. Wever, A. Tornede, E. Hüllermeier. From Automated to On-The-Fly Machine Learning. 2019.

          Towards Automated Machine Learning for Multi-Label Classification

          M.D. Wever, F. Mohr, E. Hüllermeier, A. Hetzer. Towards Automated Machine Learning for Multi-Label Classification. 2019.


          jPL: A Java-based Software Framework for Preference Learning

          P. Gupta, A. Hetzer, T. Tornede, S. Gottschalk, A. Kornelsen, S. Osterbrink, K. Pfannschmidt, E. Hüllermeier. jPL: A Java-based Software Framework for Preference Learning. 2017.

          Open list in Research Information System

          The University for the Information Society