Eating an oyster isn’t easy. First, you have to pry them off of rocks; then you have to pry them open without slicing off your fingers; then you have to suppress your gag reflex and suck their slimey goodness out. Imagine doing this in the days before oysters were served chilled with plenty of sauces on the side, and the mind boggles.
Oysters were, however, abundant at one time, which made them cheap and easily accessible, so they were a staple of the poor and desperate. Then something curious happened: We dumped a lot of pollution into the water and the oyster population declined alarmingly, driving up the price. Suddenly, rich folks were like, what’s this expensive stuff we’re not eating? And that’s basically how oysters became a pricey meal for people with money instead of something you ate when you literally couldn’t afford anything better.