Apple AI Released a 7B Open-Source Language Model Trained on 2.5T Tokens on Open Datasets.

They managed a substantial incremental improvement over previous models by first creating a better set of data as their starting point.
https://huggingface.co/apple/DCLM-7B

Is this the one the 'research only' one that was trained on YouTube transcripts including mkbhd?

Hopefully they'll be able to put something together that can run locally, so they can finally stop using this "Open"AI bullshit