I played a Sit’n Go recently and had about 6000 chips my opponent had about 2000, we were heads-up.

Each time I raised he shoved and showed me a pair, I raised with hands like K10, OJ stuff like that but I never called his all-in.

Eventually he got back in the game and in the end I lost, should I have called his shoves and try to knock him out or was I right in waiting for a real hand ?

By his third consecutive all in shove I’d make the decision that he was bluffing, and I would call it with my next decent hand (say K10 or better). It could still go horribly wrong, but the odds are on your side.