Sounds like you don't have a consistent, end-over-end roll. That's the first thing you need to fix. This is a long-ish thread, but has been invaluable to improving my putting. And @Obee is always good about answering any questions (and support!!!) along the way.
Something to check is your setup. I was a "have to have your eyes over the ball" guy forever. Found this video a while ago and that changed how I set up, how my putter is configured (flatter), and how I approach reading putts. Got a SAM lab fitting and also found my stroke needs significantly less loft than standard to get the right trajectory to start the putt.
I've gone from averaging 35.x putts per round to around 32. Doesn't sound like much...but it's a lot.