Aside from not being able to fly with it... What are the other downsides?
It will never receive any further Android updates, including security updates.
At the moment, Samsung is begging people to bring them back, and is offering incentives for you to do so. The instant the recall period is over, all liability for damages (including the value of the phone itself) is off Samsung and on you personally. That means that if you continue using it, you will (a) find it much harder for Samsung to even give a hoot about replacing it for you, and (b) be liable for potentially very large amounts of money in damages, including additional penalties for negligence should it hurt someone else or damage their property, should the unfortunate happen and yours decide to go Thermite on you.
Carriers are going to need to limit their liability soon, and that means that Note 7s will either be put in banned IMEI databases, or they will send out a kill switch firmware update. While I'm certain some third party will come out with a custom ROM to bypass the kill switch and keep it up to date, the instant you load a custom ROM that becomes a "user modified device" and you own all liability for damages from that point forward.
Basically, the current guesswork seems to indicate that any specific device has about a 1 in 25,000 chance of going Thermite. If it happens on a nonflammable surface, you and anyone around you will just be sucking in some toxic fumes. But it could start a serious fire and hurt or kill someone - and you would be 100% liable for all damages to property and people.
It's possible you could load up some third party firmware, go with the odds, use it for several years, and be quite happy with it. But understand that from this point forward it has no resale value whatsoever. It is illegal under US law to sell a recalled product, and the penalties start at $100,000 for attempting to do so. If you do manage to sell it and it blows up on your buyer, you remain personally liable for any damages it causes. So as soon as you tire of it your only legal option is to throw it away.