If you have lowered the setting to the lowest setting and it is still too much the only other option is to turn it off. You could try putting a thick or heavy case on the device in hopes of dampening the vibrations but these could also intensify them by adding mass.
Since you want some vibration perhaps you could change the vibration pattern. By default most vibration settings are set to phone call, which is where the vibration matches the length and duration of the notification sound. On Samsung devices go to Settings>Sounds and vibration>Call Vibration/Notification vibration, and pick a shorter instance of the vibration.