The Swiss great converted four of his 10 break points to end a two-match losing streak and start his grass-court season with a win.
Federer hadn't played since March, when he lost to Thanasi Kokkinakis and Juan Martin del Potro in Miami and Indian Wells, respectively.
"Three months is a long time, it's longer than the year-end break. So I'm very happy returning on good terms," said Federer, who skipped the entire clay-court season for the second year in a row.
Zverev had never won a set against Federer in their five previous meetings, but he took the first before the second-ranked Swiss upped his game and broke the German twice to force the decider.
Federer lost to another German, Tommy Haas, in his Stuttgart opener last year, but will now face Guido Pella or Prajnesh Gunneswaran in the quarterfinals.
Indian qualifier Gunneswaran upset sixth-seeded Denis Shapovalov of Canada 7-6 (6), 2-6, 6-3 in their first-round match.
Defending champion Lucas Pouille defeated German wild-card entry Rudolf Molleker 6-3, 6-4 to reach the quarterfinals.
There were also wins for Milos Raonic and Maximilian Marterer in the first round, while Denis Istomin defeated Florian Mayer 6-3, 6-4 in the second round.