1: \begin{abstract}

2: A foundation model is a machine learning model trained on a large and diverse set of data, typically using self-supervised learning-based pre-training techniques, that can be adapted to various downstream tasks. 

3: However, current research on time series pre-training has predominantly focused on models trained exclusively on data from a single domain. 

4: As a result, these models possess domain-specific knowledge that may not be easily transferable to time series from other domains.

5: In this paper, we aim to develop an effective time series foundation model by leveraging unlabeled samples from multiple domains. 

6: To achieve this, we repurposed the publicly available UCR Archive and evaluated four existing self-supervised learning-based pre-training methods, along with a novel method, on the datasets.

7: We tested these methods using four popular neural network architectures for time series to understand how the pre-training methods interact with different network designs.

8: Our experimental results show that pre-training improves downstream classification tasks by enhancing the convergence of the fine-tuning process. 

9: Furthermore, we found that the proposed pre-training method, when combined with the Transformer, outperforms the alternatives.

10: The proposed method outperforms or achieves equal performance compared to the second best method in $\sim93$\% of downstream tasks.

11: \end{abstract}

12: