A Factory Reset does one thing, it wipes the user data partition clean. This essentially just removes the user account data. It does not do anything to the installed Android operating system so a rooted phone will remain rooted after a FR, just like a phone currently running an upgraded Pie will not be returned to its originally installed Nougat.
The internal storage in any Android device is divided up into several partitions, most being dedicated to the operating system, and one solely as a user data partition. Those system partitions have protected, system-level restrictions from user interaction, the user data partition has wide open, fully accessible permissions. So when you do a Factory Reset, that wipes that user partition clean, but it doesn't alter those system partitions. You need to flash a clean, stock ROM to do what you're expecting. Flashing the ROM replaces the installed OS, a FR does not.
Go here to download the appropriate stock Samsung ROM for your SM-N920V
https://www.sammobile.com/samsung/galaxy-note-5/firmware/#SM-N920V