OK, I investigated a little bit more, and I'm thinking it is possible to downgrade. I, however, am certainly not going to try it on my own device, so take all this with a grain of salt. I'm just trying to help, and I'm NOT responsible if you brick your device.
See this link here:
How To Downgrade Samsung Galaxy S3 Jelly Bean To Ice Cream Sandwich (ICS) Stock Firmware - Vodeblog
Now, I disagree with a lot of this, as JB runs great on the S3 -- it's just that the OTA update method is awful and it is that which is primarily responsible for the lag and crashes some folks experience. A simple factory reset is usually all it needs. Since the OP has a different reason for wanting to downgrade, let's move forward....
You don't want to actually download any of the files from that site above, I posted it because the instructions are pretty straight forward, but you'll need the software for the VERIZON version of the S3, so let's go here:
[GUIDE][VZW Edition]How to root your Galaxy SIII & More!(ROMs/CWM/Stock) - xda-developers
and scroll down to SECTION 5 of the first post.
From there, you can download ODIN the stock ICS image for your phone, since you want to go back to ICS, that would be the "Official VRALG1 4.0.4" image.
Technically, the instructions listed on here on this XDA page are the same as in the first link I posted above, just a lot more condensed. Again, only reference that first link for more detailed instructions with screenshots on how to use ODIN.
WARNING: Doing this will completely wipe your phone. Backup whatever you can
WARNING: In no way do I guarantee this will work -- just posting the instructions that others have put online.