Yes, I did forget to mention that this isn't actually "remote desktop" But you are not limited only to command-line access - you can also access the server (i.e. your computer) through many different programs, some of which offer a simple way to manipulate files.
Actually, you can indeed have "remote desktop", it's called X forwarding, it's part of the SSH protocol. However, the implementation like almost everything in UNIX is not universal. So you need to do some research for your server. All you need is to install the X window system and a preferred desktop system and off you go, remote desktop!