If they're a GitHub App, they receive a token to authenticate into your account/org when you grant them access/enable the app.
Everyone should audit their GitHub Apps periodically/avoid using them if at all possible IMO. Most of these integrations are just a convenience for adding webhooks, which you can do yourself without compromising security. Always prefer "outbound" integrations.
I saw this pop up based on this Reddit thread and on Twitter as well:
https://www.reddit.com/r/ExperiencedDevs/comments/1bf7eqa/ni...
This seems serious? Is this really serious?
Why would they need to save these tokens in the first place?