Directly answering your question: Yeah you can be too dumb to program. If you cannot read, write or count, you have learning to do before trying programming. I don't think that's you, because you were able to put several coherent thoughts together.
Let's talk about math class. Because I bet you're not bad at math, you're bad at math class, and this is because for the vast majority of students past about seventh grade, math class is so badly designed I'm going to call it an outright waste of time. Right around seventh grade, they throw away stuff like "If you have three pies and five friends, how do you cut the pies so everyone gets the same amount?" and start with "The transitive immutable property of additive inequality" or whatever. "For the rest of your adolescence and your entire early adulthood, math class is now about your ability to memorize and consistently apply completely arbitrary rules that we're only going to explain to you in nineteen dollar words.
That meme of 6/2(4+2) or whatever where people argue about whether it comes out to 0.5 or 18 is a symptom of this, because it turns out that meaningless math is meaningless. Funny how it's never a problem in shop class or science class where the numbers actually mean something.
Yes, there's math in programming. There's math in programming that your math teachers never even tried to teach you because the kinds of people who write school curricula aren't the kind of people who do things for society. You weren't taught boolean logic, you weren't taught base 2 or base 16 math, etc. Find a good tutorial that works for you and you'll learn it.
I'm also going to give this advice: Don't approach it from the perspective of "I'm going to learn a programming language." Because when you see how big a task that is, you'll be overwhelmed and quit. Instead, pick a project, something you want to build, and say "I'm going to learn how to make a calculator in this language." or "I'm going to make a simple game in this language." You'll learn the parts of the language you need for that, and get it done. Then pick another project, you'll find new aspects to learn about. Keep going in that fashion and before long you'll know how to program.