No Problem Apple Charger 5.1 Volt, The Samsung is 5.0 Volt but considering the voltage supplied by a USB port is nominally 5 volts, but in practice is measured between 4.75 and 5.25 volts you did not do any damage. I use an Apple charger on my S4 daily for months at work without an issue.
Its not a big deal.The phone limits the charging rate even if you use the most powerful charger.
Only Samsung chargers can give best efficiency (even cable should be Samsung's otherwise the charging will be slower)
If you don't believe me , search google for xda thread regarding this information (currently I am lazy to provide a link lol)