The problem with TRIM is that clearing app caches and the like will actually make the problem worse. It's not about the applications cache's getting stale, it's about the underlieing file system getting stale. Those cleared cache's means deleted files, it's those deleted files that are related to TRIM and the eventual slow down.
If you aren't installing un-installing a lot, don't have apps that create much in the way of file create/delete actions then yes, it can take a VERY long time for the TRIM issue to come up (and if you ever do a factory reset, you've cleaned up the file system so reset the issue).
What you suggested in the 10 minutes a week cleaning, would actually accelerate the TRIM issue as it generates the deleted files that add to the filesystems stale state. Now clearing the cache and such does help the system run better short term, but just doesn't solve the longterm TRIM issue (which only a factory reset of some kind of filesystem clean/repair tool would).