Whether that's right or not be aware that keyloggers could capture password and account information if you type them in. It's probably less likely that a random script kidde could figure out what site you were looking at if you used on-flash software, but by no means is the information hidden.
Unless this U3 has its own processor, keyboard and display then keystrokes, unencrypted traffic and perhaps even encrypted traffic could theoretically be captured by the PC if any malware were installed. (The PC encrypts the data, so if the PC is compromised....)
Using such a device may (or may not) make it less likely to get observed by malware, but as described on the front page I don't imagine it makes your data, passwords or surfing really safe from an untrusted PC.
I wouldn't dare try it for banking info myself. (Again, my knowlege is a look at the front page and a few deductions about how it could work.) I'm even picky about which work PCs I use for personal banking.