Although not installed on the system drive, the client does store data on it, e.g. %PROGRAMDATA%\"GOG.com\Galaxy, about 400 MB. Is there a way to install the client in a way that it keeps all data in one installation folder? Thanks!

See these:

https://portableapps.com/node/52864

https://www.gog.com/wishlist/galaxy/portable_version

Short answer; no, depending on how you define "portable".

Thank you!
I don't want it to store larger amounts of data (config files or similar would be OK) on the system drive.

I too do not wish to clutter my OS partition with stuff that isn't essential to run the OS and a few apps that I have always installed on all of my computers, so not only have moved the GOG.com folder away from the ProgramData and linked it back with a junction point, I have done the same for all savegame folders too as I got tired of moving most of the saves away before imaging my OS partition and then copying them back and then having to remember to use a liveCD or otherwise access the OS partition to backup the saves again if I ever needed to restore the OS from an image.

Now everything is much simpler as not only can I easily backup the new gamesaves folder separately from imaging the OS, I could also copy it to another computer, check the batch-file in it to see the original paths for the saves and see if there are any saves on that computer I might like to keep before removing any existing folders and then possibly edit the batch file if the account name and/or drive letter in the paths need to be changed before running it to create any missing junction points.

But beware! As I tried this also on my notebook that I used for testing Galaxy 2.0, I accidentally had mistyped the path while creating the junction point and then as I tried to launch the client it broke itself so utterly that even after purging the registry manually from anything that even remotely seems to be related to GOG and/or the client before reinstalling it, the client still gets stuck while being launched until it eventually offers me to write a crash report. So unless there is some hidden value still on my computer that causes this, I can only suspect that the incident somehow caused the client to inform GOG to revoke my beta access on the server side without notifying me about that...

Let me guess: you have a tiny SSD... ;-)

Me too, I try to keep away a lot of "user files" from my OS SSD; but I'm not that OCD about it, it's 256 GB.
And about backing up save games? You should take a look at GameSave Manager! No need to do it all manually.

I'm thinking of filling that empty M.2 slot on my motherboard - bigger SSDs have become really (relatively) cheap recently. Maybe get a 1TB one, for only the most recent games.

If you really don't want a launcher to clutter up the sys drive/partition, do this (and this is where the "portable"-part gets somewhat loose):

1. Remove Galaxy!

2. Download and use another launcher. You can literally find 100s of them on the net only using a few MB + some of them ARE portable, f.ex. D-Fend reloaded (originally for Dosgames but you can use it for Windows games as well)

3. Download and use GOGs offline installers.

4. Many games are "portable" - you install them one time and they work even if you format the sys drive or swap the harddrive to another computer. I have a HDD with 20-30 games on that are "semi-portable" and still works when I format my sys drive twice a year. F.ex. Torchlight, I also just copied the installed folder to an external drive and played it on a another computer without any fuzz.

Which ones works or not is something you just have to test for yourself XD

Sorry, wrong guess, my main gaming PC has one 500 GB and one 1TB SSD plus a few large HDDs, I just don't like to deal with the situtation where I need to restore my OS, but would first have waste time by accessing the OS partition in order be sure that I don't lose any saves. And as I have many gigabytes worth of saves too, I really don't want to have them take space in the multiple OS images that I keep at hand just in case the latest image migth not fix whatever issue I may be dealing with.

I've noticed that (in my case), the games with the biggest save game files (big enough to be relevant) usually use the My Documents folder. And that one even has standard Windows functionality to refer it to a different (non-OS) drive.

For backup purposes, have a look at (like I said before) GameSave Manager. It's free.

Well, I've always been using the gog downloader and the offline installers.
But then came Blood FS, which got frequent updates (they were quickly available via Galaxy), incl. betas, and Galaxy now supports beta channels. So, regarding that Galaxy has quite some benefits now.
Titan Quest AE also is still getting frequent updates.