My manifest file has always been able to "grow", to be written over. So now its more than 22MB big. The problem I observed was the "log" file never did get any bigger. I purposely applied and then disabled the "Read-only" attribute to the whole folder and now I have the log successfully at 4MB (the earlier version of gogrepoc runs made it over 20MB also, but I deleted that file for "cleaning" purposes while trying to diagnose this problem.)

The disk is a 10TB Seagate drive. I found another user had a similar problem with his 8TB drive. The smaller 2TB for him and 4TB for me, ran the script with no problem. Can't say what the problem would be there.

Gogrepoc is on an external drive inside a folder named GOG. The disk still has 4,55TB of space.

No, the disk is NTFS formatted. That error appeared only when intentionally using the latest pre 3 Python.

UPDATE

OK, so as far as I can tell, the script behaves differently depending in what environment I run it in. CMD, PowerShell or Terminal. On earlier versions of Windows I could run it in CMD with no problem. I even made a batch file that I could just double click and it would run the update, download, verify, clean and trash commands without me present. Now I get those permission errors.

BTW do we still need to do "clean" and "trash"? I think you made it so nothing gets left behind and the folders are clean and tidy with just download and verify.

If I just run it in PowerShell the result is exactly the same as in CMD.

If I run it in either ones with the shortcut made to run it as admin, I get a whole new environment where I need to install every dependency and setting up everything from scratch. I need to log in gogrepoc again, a new manifest, new download. Like gogrepoc doesn't know it already did that.

So now what I need to do at the moment on my 24H2 install is I need the latest Python with all the dependencies updated, App execution aliases for Python and Python3 "OFF" and I need to run the script in Terminal, e.g. admin PowerShell.

I believe I got the permission error once more, but once I got it I went to the download folder and deleted the files in there. Then the download continued with not problems and I was able to finalize my last 150GB of downloads. (before it would randomly bug out in the middle of downloading, not even while it was writing anything. Though it could have been trying to log something while downloading.)

I ran the verify then update, download and verify again, and so far it looks like it worked.

But I sill am not a 100% sure as to what was the final catalyst for a successful operation. This is what I have done.

Took permission of the whole drive and deleted the System Volume Information folder. !NOT ON MY MAIN DISK!
Applied and disabled "Read-only" for the whole disk.
Disabled "App execution aliases" for Python and Python3
Ran the script in a PowerShell window with admin rights (Terminal)
If I got the error, I deleted the "!downloading" folder and ran the script again, now with no problem. (The error happened only 1 times, though I can't say why. Probably because I didn't clean the "!downloading" folder prior to downloading from an earlier failed attempt)
The script completed successfully. Update, download and verify with all "options" applied.

P.S.: I am using the "fixed" gogrepoc you sent me via DropBox.

That's very strange. Are you trying to execute gogrepoc.py directly eg by ./gogrepoc.py in PowerShell ?

The behaviour you're describing sounds like the automatic association is different between environments ( which is possible) and the admin behaviour is what you'd expect if you had multiple python installs and they were calling different ones, which is weird.

You probably need trash still to deal with orphaned files efficiently. I'm not sure if clean is needed, I can't remember how verify handles unknown files it encounters but I'll check for you.

I am not sure what the "trash" command is, but doesn't hurt to try them, if they do anything for you? You can even try them with the --dryrun option so that they don't actually do what they are going to do.

In Windows, I always run gogrepoc from a command line.

Frankly you seem to have so weird issues that if possible, I would start from a scratch, which I've done a couple of times already. Meaning something like:

1. Copy all your existing downloaded GOG game installers to other hard drives.

2. Wipe that hard drive completely clean, reformat it.

3. Install the newest Python 3 (and dependencies/pip modules/whatever) and use it, unless there is a very good reason for you to keep using (also) Python 2.

4. Delete or move away your existing gogrepoc log files, manifest files (also the partial files)... The login token you can keep if it still works for you. So start from the scratch, recreating also your manifest file. (Frankly, most problems people here seem to have is that their old manifest file, which they have apparently used for years across several gogrepo branches, bugs out. Yeah you can waste time trying to figure out how to fix that... or just start from a scratch with a clean manifest file.)

5. Move your GOG offline installers back to that reformatted drive, then run gogrepoc update (recreating a fresh new manifest file for you), then download and clean etc. (maybe first with --dryrun to make sure you are pointing to the right installer directory and you don't accidentally e.g. move all your existing GOG games to the orphaned directory). You know the gist.

I did exactly this even before I posted my errors on this forum. I was then convinced it wasn't a problem on my end and it turned out I was partially correct.

Now that I've got it working I will try to "start" from scratch and from CMD. But I'm a little scared that it will just start in a "new environment" and mess things up again. :D I have 4,5TB of games to copy/download, that takes a lot of time just to copy from disk to disk.

Ah ok I missed that, if you already reformatted the external hard drive where you both keep your GOG installers and run the gogrepoc script itself. I have a similar setup, a 18TB WD USB hard drive which contains all my GOG installers and gogrepoc (+manifest files + log file etc.) are there too.

That setup is the easiest for me because then I can run gogrepoc on different computers (Windows, Linux and even Raspberry Pi4), the only requirement is that said computer has python3 and the modules installed that gogrepoc requires.

Yeah when I had some odd filesystem corruption problem on that 18TB USB hard drive, I had to copy over 11TB of data to several smaller (2-5TB) hard drives, before I could reformat it and copy the files back. After that, smooth sailing.

In my case some folder on that NTFS-formatted hard drive became read-only and I just couldn't get it fixed no matter what, and disk check (chkdsk) claimed everything is fine on that filesystem even though e.g. trying to write (or even read?) from that certain folder warned about filesystem corruption... Had to start from a clean table.

OK, I found something. This is my batch file: When I run my batch file, it creates a GOG folder inside my G:\GOG folder where all my games are installed so then I get G:\GOG\GOG alongside all my installs. Even though all my gogrepoc files (py, manifest, log and token) are in G:\GOG.

But if I run the gogrepoc command from inside the terminal where I'm inside G:\GOG, then it finds all the correct games already installed and verifies them with the new manifest with no problem.

Then it occurred to me that there might be some more random GOG folders in my system. And there were. One was in System32 folder and one in my C:\Users\"me" folder. They had just the logs inside. And maybe, because gogrepoc wanted to write the log in System32 folder, that they couldn't since it's protected.

I tried to move the batch file to the root of my G: drive and run it from there (so it wouldn't create a folder inside my G:\GOG folder), but it just "orphaned" my whole collection and I'm currently re-copying from my backup and then re-downloading almost my whole collection of games XD

The question now is what is the correct way to input the location of gogrepoc.py for my batch file to work properly and to actually use the folder that it's in? On my WIN10 install the current code worked, but no more.

I don't consider myself an expert on gogrepo but I have noticed one thing: Your batch file is calling gogrepo.py, not gogrepoc.py. That is the old version by woolymethodman that hardly gets updates any more. gogrepoc is Kalanyr's fork that is getting regular updates and bugfixes.

If you run two vastly different versions of gogrepo and let both work on the same manifest file I wouldn't be surprised if all different sorts of errors creep up since in the end the manifest will be compatible to neither version since both programs make changes to it that aren't compatible with the other version.

You should choose which version you want to use and remove the other one.

The solution of my problem in the end is perfectly logical and simple. The same commands I have to put in the terminal work in normal CMD even if run from shortcut for easy access. I read at the description page that you need to specify the folder you want gogrepoc to download or verify files. At lest for me, now it is not needed. On my previous Windows install (maybe even previous gogrepoc.py version) I had to specify the folder to verify. Now it just works. And I feel very silly...

My setup is a bit similar as yours, ie. I have a NTFS-formatted USB hard drive where I have a GOG subdirectory within which I have gogrepoc.py, the manifest file etc.

Below that I have another GOG subdirectory to which gogrepoc downloads all the offline installers within their respective subfolders, and it also has the !downloading and !orphaned subdirectories as well.

So I run gogrepoc.py from X:\GOG\, and the installers get downloaded into X:\GOG\GOG\<title>\ subdirectories.

These are the commands I use if I am running gogrepoc in Windows:

python gogrepoc.py update -lang en -os windows
python gogrepoc.py download gog\
python gogrepoc.py clean gog\
python gogrepoc.py verify gog\

If I am on Linux, I run these instead:

python3 gogrepoc.py update -lang en -os windows
python3 gogrepoc.py download ./GOG/
python3 gogrepoc.py clean ./GOG/
python3 gogrepoc.py verify ./GOG/

I noticed you used .\GOG\ (with the period) in Windows, while I simply point to GOG\ which is correctly understood as "the GOG subdirectory under the current directory where I am running this command".

Is that what was bugging out in your setup? I don't even know how that ".\" is used correctly in Windows.
In Linux I use "./" which means the same.

Anyway, good thing you got it working. I didn't know you don't nowadays need to specificy the path to the installers with download/clean/verify, hopefully it still works as before if you do.

I'm sure now that this was the reason. I did not know that the correct command for a folder in CMD is just gog\. You would think you needed to write the letter of the disk in front, but I think the "\" is actually understood as an "enter" or something to go to the next line or execute the command. In PowerShell on my earlier system, I needed to put "%\" after every command, so it would execute them. Now all I need is just write like I would in the command window. No "enter" required.

And even though I thought I was just specifying where gogrepo.py is by writing it in a path (G:\GOG\gogrepo.py), that actually made the script or CMD (don't know which) confused, so it started downloading in the drive that Python was installed and run from (once from System32 folder and once from my User folder).

But something else is still a mystery to me. If I want to run the script as an administrator via the shortcut, then it would behave as though I have not logged in or downloaded anything. Probably uses a different folder to run that too instead of just from where it is.

Yeah but now it works like a charm, like the earliest version of the batch file I had. Just plain simple basic commands.

Since the remaining problems seem to be unique to your situation, I've rolled out the bugfixes. Thank you everyone for your reports and feedback.


The only major change for people who haven't already had issues is likely to be that GOG fixing the slugs for a bunch of games that ended with _copy3 or were misnamed as different games in their series should work and not crash ( I coded the solution to this ages ago but GOG fixed the underlying issue that caused me to before I could test it and I forgot to cast an int to a string )