To sign out of iCloud on all of your devices at once, you can head to iCloud's website and change your password.
To use Sign In with Apple, you simply need to select the option when asked to log into a participating app or website.