Whalers Bay in Deception Island is actually the site of a volcanic caldera. This has the unique property that while the water there is ice cold, the sand on the beach is quite warm as steam rises through it. This was the site of my polar plunge while touring through Antarctica, which is an entirely different story. The short version of which is, do the plunge - you only live once.
Whalers Bay has been designated as a Historic Site in Antarctica, meaning its a protected site under the Antarctic Treaty System, of which 52 states are currently party to the treaty. There's a number of pre-1970 remains on the shore including those from the early whaling period (1906-1912), a whaling station established in 1912 and all artifacts related to its operation until 1931 as well as a small cemetery.
Old remnants fall into decay on Deception Island, part of the South Shetland Islands.