1: \begin{abstract}

2: This study introduces and explores the concept of inter-architecture knowledge transfer (IAT) without limitations along with some novel computationally inexpensive methods that enable knowledge transfer between mismatched architectures and a simple toolkit implementation of said methods.

3: %

4: IAT is often unnamed and applied in part in neural architecture search algorithms [NAS], however, to the best of our knowledge it has never been posed as a separate problem without the commonly assumed limitations including arbitrary network scaling and branching.

5: %

6: Therefore, we explore the IAT without these limitations and propose a fast training-free framework to perform the transfer itself.

7: %

8: Given the experimental nature of deep learning, many network architectures must be tested before one is used in production. With the primary objective being speeding up the neural network training from scratch, we demonstrate that IAT from any of the tested network architectures is superior to random initialization.

9: %

10: Experiments prove our framework to be a valuable tool for both manual experimentation and the automated NAS, speeding up the convergence of the student network.

11: %

12: The idea to reuse the knowledge obtained in previous architecture iterations is eco-friendly, economical and has a wide variety of possible applications ranging from automated NAS to faster training whenever we modify an architecture with a well trained set of weights.

13: %

14: We also provide a new network architecture similarity measure strongly correlated to the effectiveness of our method.

15: \end{abstract}

16: