Ice is weird.
Constructions of ice won't melt.
Some constructions of Ice will allow miners to dig right through.
This is several saves from a vanilla fort, CombineSeals, and they show an ice experiment where a structure built under the sea is later mined out leaving empty space.
By disassembling walls, I have occasionally created impassible unusable unbuildable squares with no type. I couldn't reproduce that in the first try at CombinedSeals, but I will keep trying. To reproduce this, d->z remove upward stairs/ramps after deconstructing walls built over upward ice stairs.
Ice experiment is at HotKey F5
To dig through constructions:

1. dig out an area of ice 2 or more Z high
2. construct a room with a roof (floor) at least 1 Z underwater
3. allow to thaw, water covers the undersea roof/floor
4. allow to freeze
5. designate the whole area up/down stairs
6. optionally channel out the stairs above the roof.