cioran: For an open-ended sci-fi shooter with RPG elements somewhat like what Borderlands is anticipated to be you may want to try STALKER (it's a really good game if your machine can run it), Farcry (phenomenal sandbox shooter game, but no RPG elements) or even the somewhat less notable Boiling Point (very buggy, but if they patched it by now, it'd be a pretty decent game - heavy RPG elements for a shooter). They're not straight up shmups though, so you can't just run around shooting everything.
lowyhong: The only game I haven't tried in that list is Boiling Point. I've been very interested to try it, but even today, I still see people criticizing it for bugs and issues. Can any of you here recommend it to me? I like the concept, and the reviews really sell the game to me, until I read about the game's inherent problems.
BP's not as bad as the reviews state (it has a steep learning curve and the first hour is terrible which is where most reveiwers stopped playing), but otherwise they're mostly accurate. It actually does open-ended modern RPG pretty well (IMO, the game kind of plays like an archer in Elder Scrolls except not as deep and with vehicles). It doesn't do shooter as well. It is bugged to hell too. Personally, I like it (and from a design standpoint, conceptually it's brilliant in its integration of FPS, FPRPG, and GTA-style sandbox games), but I can only make a highly qualified recommendation based on the amount of issues the game has. Save early, save often.
You may still want to buy it. I wouldn't spend more than $10 though. 50/50 you might hate it. I don't know anyone who loves it, either. Its main problem was that it was a game that lacked the graphical technology to adequately support it on PCs when it was released (e.g. Nocturne, Vampire the Masquerade:Bloodlines). It wasn't coded particularly well, either and poor sales led to a lack of updates (a la Temple of Elemental Evil, Anachronox, the already mentioned VtM:B, Arcanum, Daggerfall). It may fare better today if some dedicated fans updated the patches on their own. I doubt it though. It never had a rabid fanbase. It did have a really cool theme song, though.
Have you played Vampire:the Masquerade Bloodlines? That's a great FP CRPG set in the modern era (now that's it's all patched up and PCs are faster). You can play it using firearms, too if you want.