I don’t think this is meant to be literal. Besides, you never really see The Tower; it’s more like an avatar –everyone sees it a different way, like the face of Galactus or a Vorlon.

Although…the final book portrayed this particular shape as the “truest” version, standing in a vast field of roses. IIRC it’s supposed to be something like 600′ tall, so no bigger than an average NYC skyscraper (of course, with nothing else around for comparison, it would seem utterly colossal.)