If Islam opposes idol worship, why do Muslims pray to a square structure?

Although Muslims face the direction of Ka‘bah during prayer, they do not worship it. Muslims worship and pray only to God.

إياك نعبد وإياك نستعين

You alone we worship and You alone we ask for help.

—Quran 1:5