1: \begin{abstract}

2: %

3: Hierarchical learning algorithms that gradually approximate a solution to a data-driven optimization problem

4: are essential to decision-making systems, especially under limitations on time and computational resources.

5: %

6: In this study, we introduce a general-purpose hierarchical learning architecture

7: that is based on the progressive partitioning of a possibly multi-resolution data space.

8: %

9: The optimal partition is gradually approximated by solving a sequence 

10: of optimization sub-problems 

11: % that  

12: % yield a sequence of partitions with increasing number of subsets.

13: % %

14: % We show that the solution of each optimization problem can be estimated 

15: online, using 

16: gradient-free stochastic approximation updates.

17: %

18: As a consequence, a function approximation problem can be defined within each subset of the partition

19: and solved using the theory of two-timescale stochastic approximation.

20: %

21: This simulates an annealing process and defines a robust and interpretable heuristic method 

22: to gradually increase the complexity of the learning architecture

23: in a task-agnostic manner, 

24: giving emphasis to regions of the data space that are considered more important according to 

25: a predefined criterion.

26: %

27: Finally, by imposing a tree structure in the progression of the partitions, 

28: we provide a means to incorporate potential multi-resolution structure of the data space into this approach,

29: significantly reducing its complexity, while introducing hierarchical 

30: variable-rate feature extraction properties similar 

31: to certain classes of deep learning architectures. 

32: %

33: Asymptotic convergence analysis and experimental results are provided

34: for supervised and unsupervised learning problems.

35: %

36: \end{abstract}

37: