The 39-year-old Haas, who is playing his last season before retirement, saved nine of the 12 break points he faced and converted two of his three chances to deal the Swiss great just his second defeat of the season.
"I've even shocked myself a bit," said Haas, who is ranked 302nd and was given a wild card for the grass-court tournament. "I'm a bit speechless. It's hard to find words."
Haas, who hadn't beaten his good friend since the 2012 final in Halle, improved to 4-13 against Federer, who hadn't played since winning the Miami Open in early April.
"If you don't take your chances like I didn't, leading a set and a break, you really only have yourself to blame at the end," Federer said. "You've got to acknowledge the fact that he was a bit better. It's quite frustrating, but that's the way it goes sometimes."
The 35-year-old Swiss had skipped the clay-court season to recuperate from a busy start to the year that saw him capture his 18th Grand Slam at the Australian Open.
Federer claimed his last grass-court title in 2015 in Halle, where he is an eight-time champion.
Also Wednesday, Philipp Kohlschreiber upset fifth-seeded Steve Johnson of the United States 7-6 (3), 5-7, 7-6 (6), while sixth-seeded Mischa Zverev, another German, defeated qualifier Yannick Hanfmann 7-6 (1), 6-2 to secure his place in the quarterfinals, where he next faces Haas.
No. 4 Lucas Pouille of France came back to eventually beat Germany's Jan-Lennard Struff 4-6, 7-6 (5), 7-6 (8) for a quarterfinal against Kohlschreiber.
Pouille, like top-seeded Federer, had been given a bye to the second round.