Steg calls attention to an interesting midrash, שמות רבה כב:א:
When the kingdoms would be allegorized, they would be compared to animals, and when the מצרים would be allegorized, they would be compared specifically to foxes —Rabbí El‘azar beRabbí Shim‘on said: the מצרים were tricky, therefore they were compared to foxes; and just as foxes walk while looking backwards, so too do the מצרים walk while looking backwards.
Steg points out that images of animals and humans from Egyptian hieroglyphics show them facing backward (over their shoulder, like an owl) and supplies this image:
Could be. I'm not so sure this is precisely what is intended because with a little checking it doesn't seem like human figures are always facing backward in Egyptian hieroglyphs.
However, the sort of stiffness of the human depictions in Egyptian art have long been noted, like this:
Indeed, such postures are a part of popular culture.
I believe Steg gave a beautiful explanation for the factual, historical basis of this midrash, although I think that perhaps it is a more general allusion to type of Egyptian depictions in reliefs which we are used to, rather than hieroglyphs specifically.
Or, maybe they did walk backwards!