I finally figured it out for my Note 4.
My problem was Android Wear settings. There is an option to have the phone quiet for Notifications and Ringtones if the Watch is connected to the Note 4. Run the Android Wear App, click the little gear for settings and check that the notifications and...