Go read some books on manual para-axial ray tracing and get back to me....
Before you go for Optics, skim the following texts... Order anti-depressants in advance...
If you like and UNDERSTAND these books, then go for optics...
Lens Design Fundamentals, Second Edition, Kingslake
Modern Optical Engineering, 4th Ed Warren Smith
Introduction to Lens Design, Geary
Quantum Electronics, Yariv..
Nonlinear Optics, Boyd..
Femtosecond Laser Pulses: Principles and ExperimentsClaude Rulliere
Then three years in a lab suffering, most of the time waiting for 80,000$ lasers to be serviced, and adjusted by insane service engineers. If you like breaking 7,000$ crystals that can take up to three months to get, be my guest.
Many of the systems I work on have 100+ lenses, mirrors, and non-linear crystals. Each of those optics has three to seven adjustments.
Quantum Optics is insane... Lens Design is boring...
Add Physics I, Physics II, Electromagnetics, Lens Design, huge amounts of Quantum Theory, crystallography, Laser Theory, Calc III, Differential Equations, etc to your class list...
You sure you want to do that... If it were me I'd talk to Arizona State, and see what I had to do to get that Masters... A lot of pretty women there, but you won't have time to meet them... I know, I was there a few times...
Then consider going to Texas, get the two or four year optics and laser technician degree, and call it quits..
You'd be far better off getting an EE, a ME, or CE , maybe a MBA, then getting a Masters in Optics... Because that leads to needing the PhD in Optics or Physics to be employed... As our Department Secretary says, "A Masters is a just piece of paper, a Phd gets you a job." And she has the Masters...
All of it needs two or three years of advanced math... Which is why I am always a Technician or Field Service Engineer... Even then I need complex algebra several times an hour, when at work... I'm currently teaching myself basic, very basic, Heat Transfer... If, as you say, you do not like math, this is NOT for you.
Steve