A building on Inverloch’s foreshore has been abandoned for more than five years, with residents saying it should be redeveloped as soon as possible.