1: \begin{abstract}

2: Gradient boosted decision trees are a popular machine learning technique,

3: in part because of their ability to give good accuracy with small models.

4: We describe two extensions to the standard tree boosting algorithm designed

5: to increase this advantage. The first improvement extends the boosting

6: formalism from scalar-valued trees to vector-valued trees. This allows

7: individual trees to be used as multiclass classifiers, rather than requiring

8: one tree per class, and drastically reduces the model size required for multiclass

9: problems. We also show that some other popular vector-valued gradient boosted

10: trees modifications fit into this formulation and can be easily obtained in our implementation. The second extension, layer-by-layer boosting,

11: takes smaller steps in function space, which is empirically shown to lead to a faster convergence and to a

12: more compact ensemble. We have added both improvements to the open-source TensorFlow Boosted

13: trees (TFBT) package, and we demonstrate their efficacy on a variety of multiclass datasets. We expect these extensions will be of particular interest

14: to boosted tree applications that require small models, such as embedded devices,

15: applications requiring fast inference, or applications desiring more interpretable

16: models.

17: \end{abstract}

18: