Combine the first 3 ingredients in a small pot and heat over low flame just long enough to dissolve the sugar. Cool the mixture.
Shuck the oysters, reserving their liquor and half the shells (those with deeper cups).
Place the oysters in the cooled pickling liquid for 5 minutes. Remove the oysters from the pickling liquid, drain, then place them in the reserved oyster liquor.
Spoon 1 oyster into each of the retained shells and top with a light grating of fresh wasabi, a bit of olive oil and a sprinkle of caviar (if using).