Forster had broken former Aberdeen goalkeeper Bobby Clark's 43-year-old Scottish league record on Saturday and it took something special to finally beat him after 1,256 minutes as Hayes struck a spectacular long-range effort.
Having conceded for the first time in 14 Scottish Premiership games, the England international shipped another goal four minutes later when Adam Rooney sent a diving header past him from Niall McGinn's cross seconds before the interval.
Substitute James Forrest's excellent finish in the 61st minute got Celtic back in the game following a spell of pressure but they had been up against it from the 11th minute, when Virgil van Dijk was sent off for bringing down Peter Pawlett.
And the Dons held out to inflict Celtic's first league defeat for 10 months since they lost at Motherwell.
Celtic had won their previous 15 Premiership games and not lost in 26 this season but if any team were to break Celtic's impressive runs, Aberdeen would have been the obvious choice.
Forster previously conceded a Scottish Premiership goal against McGinn three months earlier and the Dons knocked them out of the William Hill Scottish Cup earlier this month.