You can download an email from Gmail, or email threads and attachments, and save them to your computer.
You can delete your Google account by going to Google's "delete or service your account" page, but you can export your data beforehand.