Interpreting human electroencephalogram (EEG) is a challenging task and requires years of medical training. We present a framework for learning useful representations from EEG signals via contrastive learning. By recombining channels from multi-channel recordings and fusing multiple datasets, we increase the number of samples quadratically for self-supervised pretraining. We train a channel-wise feature extractor by extending the SimCLR framework to time-series data with a specific design choice of producing sequential representations with equal length to the input sequences. This property allows us to use the learned representations in a variety of downstream tasks that require different lengths of signals. We introduce a set of augmentations for EEG channels and study their efficacy on different classification tasks. Using 3 standard datasets, we demonstrate that the learned features improve EEG classification and reduce the amount of labeled data needed on three separate tasks: (1) Emotion Recognition (SEED), (2) Normal/Abnormal EEG classification (TUH), and (3) Sleep-stage scoring (SleepEDF). Our models show improved performance over previously reported supervised models on all three tasks with training on only 10% or less of the labels. Fine-tuning on 100% of the labels, we achieve the performance of 85.77% on Emotion Recognition and 85.12% on Sleep-stage scoring.