It largely depends on what you're developing. Math is always involved -- to some degree -- with application development. It's hard to write an interactive application without using basic loops and numerical variables. Keeping track of these loops and variables as your methods execute is necessary. As your application grows (in size, not in popularity), the patterns that these loops and variables follow can become increasingly complex.
However, an application that lets you fine-tune your contacts would, more often than not, involve less intricate mathematics than a physics-intensive game or one which involves sophisticated mapping and location control.
Edit: I consider myself to have a strong grasp of up to mid-level collegiate mathematics, and I've noticed direct benefits from much of this knowledge while developing for mobile platforms.
I was very good at math in school, though in my experience, you don't generally need to use math for programming, unless the program itself requires it. For instance, the only app that I currently have that uses math is a weight & balance calculator for pilots.
But, the kind of logical thinking that you need for programming is very similar to what is needed for solving math problems. So, if you have a hard time with math, I suspect you might also have trouble with programming. If you haven't done any programming before, I'd recommend taking a look the MIT App Inventor (formerly the Google App Inventor): MIT App Inventor. App Inventor provides a graphical "drag and drop" interface for creating programs that should make it easy for you to get your feet wet and see if its something you enjoy. They also have lots of tutorials to get you started.