Can I install whatever gog game over itself, so the new version would take the place of the old. or
do I have to uninstall the GOG game then install the new version of the game?

It likely varies from game to game.

I myself do this with new versions of Titan Quest Anniversary(just updating the current version, not stuff like adding expansions) and it seems to work each time I do so....the saves work and etc. Dunno about other games though.

The answer is, it depends. What exactly are you trying to do. Is it exactly the same game, I.e. from same install, not from different stores, or different versions. Are you using mods, it’s always a good idea to do a clean install if modding. What game, as they have files all over the show. My recommendation is to uninstall (use geek uninstall), then reinstall from fresh. It’s just cleaner and safer.

99.999% of the time it's fine. That time it isn't, though....

To clarify, this is only assuming it's GOG version over newer GOG version. Might be more useful to know which games. A great example of the most likely scenario is minecraft update history (which isn't sold on GOG): they no longer even use the same save format, so there was a warning to use a certain version to upgrade worlds. Even still, any chunks that were done after the udpate would have drastic terrain changes, due to the huge change in how worlds are generated. This mentality applies to just about every game. So, for example, if upgrading stardew valley, and there's a 3 heart event and you're 4 hearts or higher, you're likely to miss that 3 heart event.

However, there's nothing inherent in GOG versions that makes things this way. GOG largely does fake returns for the steam API stuff, so it'd be something that would apply to version of a particular game bought elsewhere if there's a problem with the overwrite thing.

100% of the time it is fine, there is no instance when it won't be fine. Galaxy checks to see if the files you have in the folder match the hash for the files it is prepared to download, if they match it doesn't download. This means that if you had a steam version of the game, you could spare yourself downloading the majority of the files by installing the GoG version directly over the steam version, you'll probably just get a new exe. The downloaded installer will replace all files in the folder regardless.

That being said, of course saves and preexisting mods can break, but the worst that can happen is that you have extra files the game doesn't use anymore.