“Compliance” builds gripping tension with its unsettling premise, and interesting performances, but it fumbles the ending, leaving the conclusion less impactful than the suspense leading up to it.