I can't build stairs over the ones I removed, with the tiles being considered blocked. There are dwarves and stone on both sides.

Steps:
1. Dug stairs down from surface until I breached the caverns from above.
2. Dug a path sideways and then down so I could access the cavern floor.
3. Built constructed stairs over the natural stairs, hoping this would convert up/down stairs to up.
4. Removed constructed stairs, resulting in 2 down stairs and a stranded dwarf.
5. Constructed a floor over the down stair that breached the caverns.
6. Attempt to construct up/down stairs, but it's blocked somehow.

I tried removing the constructed floor to see if that was the problem, but that doesn't help.

Edit: Steps 3 and 4 can be skipped, building a floor on the up/down stair instead. Digging stairs adjacent to the tiles stops them from being blocked and allows the desired stairs to be constructed.