Any idea if that would mean auto compilation for your own files in a project? Or is it more for plugins and such?

It's for plugins: https://github.com/atom/atom/pull/5299

I'm curious as to how they will handle 6to5 versioning. 6to5 seems to be very much in a "move fast and break things" stage, and I'm not sure that's going to change any time soon.