No you wont loose all your data.
No you won`t loose all your data I do it all the time, I even use custom recovery to, wipe cache and dalvik, as long as you select wipe catch partition not wipe data factory reset you should be sweet, hope I helped just reinforcing that it`s safe to do so.