Alarm rings only when Clock app is open in background

[AC Question]

I recently rooted my phone and removed the stock clock app and installled Timely instead.
Timely is a cool alarm app but the problem is the widget doesnt update and the alarm doesnt ring unless the app is open in the background.

So, I decided to reinstall the stock clock app from the Store and the same thing happens to it. As soon as I dismiss the Clock app from background, the alarm notification disappears and the alarm doesnt ring. I am pretty sure this wasnt the case before.

PS. I am running Lollipop on Lenovo A6000 PLUS