1: \begin{abstract}

2: The ability to dynamically adapt neural networks to newly-available data without performance deterioration would revolutionize deep learning applications.

3: Streaming learning (i.e., learning from one data example at a time) has the potential to enable such real-time adaptation, but current approaches $i)$ freeze a majority of network parameters during streaming and $ii)$ are dependent upon offline, base initialization procedures over large subsets of data, which damages performance and limits applicability.

4: To mitigate these shortcomings, we propose Cold Start Streaming Learning (CSSL), a simple, end-to-end approach for streaming learning with deep networks that uses a combination of replay and data augmentation to avoid catastrophic forgetting.

5: 

6: Because CSSL updates all model parameters during streaming, the algorithm is capable of beginning streaming from a random initialization, making base initialization optional.

7: Going further, the algorithm's simplicity allows theoretical convergence guarantees to be derived using analysis of the Neural Tangent Random Feature (NTRF).

8: In experiments, we find that CSSL outperforms existing baselines for streaming learning in experiments on CIFAR100, ImageNet, and Core50 datasets.

9: Additionally, we propose a novel multi-task streaming learning setting and show that CSSL performs favorably in this domain.

10: Put simply, CSSL performs well and demonstrates that the complicated, multi-step training pipelines adopted by most streaming methodologies can be replaced with a simple, end-to-end learning approach without sacrificing performance.

11: \end{abstract}

12: