For something that does this already, do checkout Next.js.
As well as taking care of babel and webpack, it handles server side rendering, code splitting, routing and dynamic routing automatically with zero config.
Unlike create-react-app it can be easily extended gracefully to add behaviour like direct SASS or LESS importing.
Although it’s built for React, you can even use it with projects that are not React based (e.g. Vue).
Probably the most well known site using it is https://www.npmjs.com