I also have a Nexus 4 and that is mainly why I felt that I needed to upgrade from gen1 to gen2; I could feel the lag after using my buttery smooth N4.
But yeah, I love having both. When at work and home, I never use my phone except for actually calling people. I use MightyText to allow me to text from my N7, so even texting doesn't need the phone. Using a 7" in screen feels like night and day in comparison to the N4. Go into Best Buy or Staples or something and give it a test drive.