Of course it matters. If you just write a practical small program, it doesn't involve the development of large-scale software, program cracking, encryption technology or database, then you don't have to consider those messy algorithm problems. High school algebra should be more than enough, even a concept of "function" is enough.

In fact, programming has very high requirements for mathematics. Personally, I feel that programming needs the whole higher mathematics system to pave the way, and I even feel that it is not enough.