Plans for a five-story building on the south side of Main Street would block views of the Peconic River, some residents said.