I have a small server (not set up yet) that will eventually host a small community. (Unless we must expand). Until now I've been using a hosting service that uses Multicraft. I'm not sure if this is the most optimized way to run a minecraft/bukkit server, but I don't really like the lack of customizability. Now, if a VPS is what I think a VPS is, I think I can install linux on it (such as Ubuntu server) and instead of uploading files from my computer (I have bad internet speed), I can tell the server through command line to download them? I would very much like some of those abilities.
I'm not sure what to look for. I'm on a budget of $10 per month. I've read on another post that OpenVZ is bad. I'm also not sure what to look out for in a VPS. Could somebody enlighten me?