Yes, I am sure it is essentially more about class and wealth than about race, as such. If expats tend to be white people from the developed world, that’s because those are usually the people who have the economic and educational advantages that enable them to be expats.
Of course, that’s a very large issue of inequity in itself. But the point is that they aren’t termed “expats” as opposed to “immigrants” simply because they are white — it’s simply that the status of “being white” and “being an expat” tend to go together, for economic reasons.
An interesting illustration of this can be seen in the UK at the moment, where white people from eastern Europe are usually termed “immigrants” (even if they have no intention of long-term residence) because they take relatively low-skilled or low-paid jobs. By contrast, a wealthy Chinese businessperson living in London for a few years would much more likely be called an “expat” than an “immigrant”.