Unfortunately, no. CRA does not allow access or ways to customize the service worker out of the box, as it requires changes in the production configuration. That being said, instead of ejecting, you can fork and create your own instance of the ReactScripts package and make the customizations there, you can see this link for more info https://auth0.com/blog/how-to-configure-create-react-app/

However, currently there is not way to implement push notifications without doing these customizations