This is the entrance to the old Tenby Mortuary - dating from 1892.
There are a number of steps that lead down from the town wall to the mortuary (or up from the harbour) called the 'Dead House Steps'. - see the blue plaque on the wall in the photo.
Apparently inside the mortuary there are still two china slabs from the days when it was used to store bodies prior to post- mortem examination and burial.