VPS connection question

[Maxime]

I know I can connect to a VPS via remote desktop protocol. But I don't want to leave any trace on my computer that I've ever connected to a VPS. And remote desktop protocol records and saves every connection & history. Even when that info is deleted, the deleted info can still be retrieved.
Is there any app that allows me to connect to a VPS and doesn't save or store trace of connection history?
Thanks