Does the earth have an unlimited supply of magma?

No

Explanation:

The magma supply of the Earth is not unlimited, but instead it has a fairly constant volume. On the other hand, the magma is not spending itself even though it constantly comes out on the surface of the Earth. The reason for that is that the magma that comes out on the surface, is actually replaced by newly formed magma. The newly formed magma is actually from the crust of the tectonic plates that are subducting. As the plates are subducting, they reach the mantle layer where the temperature and pressure are very high, so they melt the crust. The molten crust is becoming the new magma that replaces the one that is pushed toward the surface. This is a cycle that constantly goes on, so even though the magma is not endless, it manages to circle through the top two layers and replenish itself.

the rub-al-khali is also known as the arabian desert and the "empty quarter"