The banks are mostly and seemingly owned by foreign principals, but according to local laws aren't connected to them. Think of those as franchises. There are some local banks, too, and they aren't much different.
They have bankomats (a nice word, better than ATM) pretty much everywhere nowadays, and these operate in serbian and english - no matter the language the bank's principal's country (which may be Austria, France, Hungary, Greece, Italy, Germany). The banknotes you get are of 1000 RSD (serbian dinars). It's possible that a bankomat in a suburb runs out of cash on weekends. You can also add money to your cell phone at a bankomat, provided that the machinery can call your provider, which I never saw working - probably my provider didn't cut a deal with my bank.
(update, 2016: not seeing this option anymore, it was either abandoned as hard to maintain, or as too much of a competition to traditional points of sale, like kiosks).
The banks being disconnected from their principals has several drawbacks. First, you may not be able to receive money from abroad to your account. Or, you may, if your sender agrees to go through an intermediary bank (which is most frequently the principal of your bank), or if you agree to sharkly rates at which Western union or Moneygram and such operate here. Which is also the reason Paypal doesn't operate here at all.
The banks here issue two kinds of credit or debit cards - one for retail and bankomats, the other for internet. Both have the usual 16 digit code, and are in the system of Visa, Master or Diners or whoever, but the first kind displays only 4 digits to you - and you don't know the other 12. Which means you can't use it anywhere where the chip reader (the cards with magnetic tape never caught here) is absent, like online. For that, you are issued (after some paperwork) an internet card, on a separate account, and you have to move money around the two account as you need - this is probably to discourage people from wasting too much money online.
The other drawback is the online operation - it MUST be done in IE with most banks, and their websites are overcomplicated, the instructions full of sales pitch, proprietary terms and weird settings. These rely on some ActiveX being installed on your machine... and even then, it's not sure to work.