open training dataset release

RedPajama 1.2T-token dataset

An open 1.2T-token dataset intended to reproduce an important part of the LLaMA training-data stack.

Original sources

Open artifact View JSON