First, note that I haven't yet played the
Siege of Dragonspear expansion, so my comments don't take that into account. And I'll be mainly talking about the Baldur's Gate games rather than Icewind Dale, which you also mention.
That aside, jumping in and enjoying is probably the right attitude to have. Don't be afraid to do so without knowing a lot, and don't be afraid to learn as you go - including learning some things the hard way! If you do want to know more about the system and the setting, what's in the manual should more than suffice - don't go poring through online walkthroughs as it will probably diminish your enjoyment, at least as a newcomer.
Regarding your concerns, I'll answer the two separately:
k_adsl123: I want to go with the best race, class, stats, and so on.
This is reasonable. These games can be difficult, especially for someone not familiar with the system, so there's no shame in building an effective character. Obviously that partly depends on the difficulty setting you choose (my preference is always the AD&D Core Rules setting, but that's partly because of my experience with the pencil-and-paper game - choose what works for you). There isn't a single "best" class or race, but certain classes, especially mages, can be very challenging for new players at the beginning of the game because they can die very easily without allies and higher-level spells to protect them (though later, especially in BG2, mages become extremely powerful). So some pointers follow (just my opinions, others may disagree).
I'd recommend you play your first game as a fighter, paladin, ranger or barbarian (assuming you're playing the Enhanced Edition - BG1 classic, unmodded, doesn't have barbarians), as they're all good front-line combatants, can wear armour and can take more damage than other classes. Have a look at the abilities of those classes and their various "kits", i.e. variants (but again, BG1 classic doesn't have kits), and see which you like the look of.
As for race, there's no particular advantage to being a human unless your class requires it (e.g. paladin) or you plan to dual-class later (which I recommend you don't until you become more experienced - N. B. dual-class and multi-class are different things). See which races your preferred class allows, and decide which you like the look of; I'll only say I advise against being a halfling because of their penalty to Strength.
Which brings me on to stats. Again, I think Baldur's Gate justifies a certain amount of min-maxing, especially as the details of how your ability scores affect your capabilities can be counterintuitive (AD&D was a bit of a weird system). Go for 18 Strength, or 19 if you're a half-orc. If it's 18, don't worry about the numbers after the slash (which only the warrior classes get); higher is better, but regardless it will be enough. Go for at least 16 Constitution and Dexterity, as they both help keep you alive; if you want to focus on archery, go for the maximum Dexterity as it helps you hit things. Intelligence, Wisdom and Charisma are less important for these classes' abilities, though some of them require a certain number in them (e.g. paladins need 17 charisma!) - but this happens automatically, as you choose your class before you roll your ability scores.
The above is enough to give you, if not the absoutely optimal build, at least a perfectly viable character that an inexperienced player shouldn't have too much trouble with.
k_adsl123: I want to be sure that I don't miss any items, cut-scenes, party members, all that. Other RPG games are mostly about this kind of thing but I understand that D&D style games aren't really.
A nice thing about these games is that you're free either to take such an approach (which can reward you with more XP, items etc.) or just concentrate on the main plot. There's lots of exploring to do (particularly in BG1 which has lots of wilderness-type areas) and lots of side-quests of various sizes. As for party members, talk to as many characters as you can and, of those who turn out to be recruitable, see which ones you like. You can always replace them.
You also mentioned Icewid Dale. Although it uses the same game system (with some variations), it's quite a different style of game - it's more heavily focused on dungeon-crawling and less on plot, characterisation and so on. Also you create the whole party yourself rather making a single character and recruiting allies. It's a good game, and may lend itself better to the sort of approach you say you're used to, but not as good as the Baldur's Gate games, and I'd certainly recommend playing BG1 first.
Finally, you say your interest in these games was sparked by a desire to play D&D. That's great, and good luck finding a group, but just be aware that the currently published edition of D&D has changed a lot from the one the Baldur's Gate games were based on, so if you play one and then the other you should be prepared for a rather different system.
I hope the above is helpful. Feel free to ask follow-up questions.