Not exactly. It's likely more that you only have 1 GB of RAM. Android will try to keep as much as possible in memory to speed loading of frequent apps, which also reduces battery life. It's this that task/app killer apps aren't the greatest idea anymore.
So, because your phone can't keep as much cached in memory it will feel slower.