Yeah, that bothered me too. But the explanation goes along the same lines of why Superman doesn’t end up naked every time someone uses a flamethrower on him.

Essentially, if we were playing by real physical rules, he couldn’t lift anything big because in order to “lift” something, you need to apply a force greater than its weight, and Superman would be applying such force to a part of the object the size of his hands. If he tried to lift a whale (or even a really fat person), he’d kill it because all the pressure needed to lift it would be applied to a single point and the whale’s body wouldn’t be strong enough to withstand it.

The explanation that is sometimes given is that just like his clothes become as strong as his skin just because they’re touching it, anything he tries to lift also becomes as structurally strong as his skin and thus capable of withstanding the force needed to lift them. It’s nonsense of course, but once you go past the part about a flying man who shoots X-rays and lasers out of his eyes, complaining about anything else becomes moot.