Two methods that I have heard of to calibrate the compass. Try the following with compass mode (or some other compass app) running:
-- Figure Eight: Hold the phone with the screen facing up. Move the phone as to draw a horizontal number 8 a few times. (This seems to work for my Epic.)
-- 3 Axis Rotation: Slowly spin the phone 3 times with the screen facing up, facing toward you, facing to the left or right (3 spins in each direction, 9 total).
HTH