After they were spotted holding hands in N.Y.C., sources tell PEOPLE Swift and Healy are "having ... calling it "all fake." Two years later he addressed the rumors again in an interview with ...