I may be on a 4G or H+ connection and at random times it switches to E.
The way I fix it is going to Settings->More Networks->Mobile Networks->Network Operators->And choose the first network from the 2 from the list from my provider (my provider actually provides 2 networks, I wonder why?)...