Replies: 6 comments 3 replies
-
|
info.txt doesn't control anything, it's just a report that's generated
before downloads are done so that you know what's supposed to be in the
folder if you download it.
This sounds like something is going wrong in the update step. Potentially
not properly detecting the OS / Language versions it should be downloading.
Which version of Gogrepoc are you using ? Master or Dev ?
What OS are you running gogrepo on ? Are you running it on a device that
might be behaving weirdly (eg some NAS's don't actually report a locale ) ?
And in case I need to walk you through some troubleshooting how familiar
would you say you are with Python ?
…On Tue, Dec 12, 2023 at 12:20 AM mharmless ***@***.***> wrote:
Multiple games are just showing as no game items. Stellaris hasn't updated
since 3.7.4, now shows blank items, and new purchases like against the
storm download nothing at all.
python gogrepoc.py update -ids against_the_storm
fetches game details just fine, no errors reported. but downloading will
show nothing to download presumably because !info.txt says so.
1.
2. -- Against the Storm --
3.
4. title.......... against_the_storm
5. genre.......... Strategy
6. game id........ 1460447406
7. url............ https://www.gog.com/en/game/against_the_storm
8. release date... November 01, 2022
9.
10. game items.....:
11.
12.
13. changelog......:
14.
15. Hotfix 0.43.3 (February 3rd, 2023)
changlog cut short here. And this seems to be the problem for a laundry
list of games
1.
2.
-- Age of Wonders 4 --
3.
4.
title.......... age_of_wonders_4
5.
genre.......... Strategy
6.
game id........ 1915432764
7.
url............ https://www.gog.com/en/game/age_of_wonders_4
8.
release date... May 02, 2023
9.
10.
game items.....:
11.
12.
-- Stellaris --
13.
14.
title.......... stellaris
15.
genre.......... Strategy
16.
game id........ 1508702879
17.
url............ https://www.gog.com/en/game/stellaris
18.
user rating.... 78%
19.
release date... October 01, 2018
20.
21.
game items.....:
22.
23.
24.
changelog......:
—
Reply to this email directly, view it on GitHub
<#94>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABKZ33ZPK5MPG3EAGD4Z2LDYI4JCJAVCNFSM6AAAAABAP3ASHSVHI2DSMVQWIX3LMV43ERDJONRXK43TNFXW4OZVHE2DSMBZG4>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
The python script shows v0.4.0-a, and I believe it was downloaded from the master updated 8 months ago. I have done a small amount of scripting in a professional environment, but virtually nothing in python specifically. I can probably follow troubleshooting directions. More info on the way this runs; a weekly cron job runs a script to update games, and after noticing that stellaris hadn't updated and some new purchases were missing I've been poking at it with a remote desktop session. I'm not sure how long it has been failing to download content. This is the script that runs, very straightforward. Manual running update initially showed 12 updates found, same as the gog website showed. Nothing was downloaded anyway with a subsequent download run, although update no longer claims to see new content so it changed something. If I run without the os restriction, it does try to download stellaris 3.10.3 linux version, so it sees that one at least. If I run without skipfiles or lang, no change. If I remove a previous version, such as 3.7.4 which was most recently downloaded, it does not attempt to redownload that; it will still say nothing to download. |
Beta Was this translation helpful? Give feedback.
-
|
The problem appears to be that you're letting update autodetermine what os
and language you're using ( which will probably be Linux for the OS ) but
specifying to download only windows.
You should run the update command with the same -os / -lang you use for
download.
However given how out of date the windows info in your manifest is you
probably need to disable the chron, manually run the following command:
python gogrepoc.py update -os windows -lang en -full
It should take about an hour for every 300 games you own.
Then you need to manually run your download command, that might take
upwards of a day to finish depending on how many missing windows
updates you have ( which is why we needed to disable the chron jobs
for this )
Once that's done, I recommend you make use of the verify command:
python gogrepoc.py verify /mnt/fakeraid/Games/__GoG\ Archives/ -clean
-os windows
Which should use the md5 data from GOG to make sure your files are
correct, it should move any wrong ones to a subdirectory called
!orphaned
That could take a long time if you have a lot of game files.
Let me know what verify outputs when it's down and I'll let you know
how to proceed from there ( I think there's a reasonable chance
there'll be a few issues given how out of date things probably are )
( Verify is safe to use in a chron usually because it remembers files
that it's already verified so subsequent runs are much faster )
On Tue, 12 Dec 2023, 05:56 mharmless, ***@***.***> wrote:
The python script shows v0.4.0-a, and I believe it was downloaded from the
… master updated 8 months ago.
I'm running it on a custom storage server that hosts media and game files.
Off the shelf hardware, a basic install of linux mint 21, along with
snapraid and mergerfs to handle the storage.
I have done a small amount of scripting in a professional environment, but
virtually nothing in python specifically. I can probably follow
troubleshooting directions.
More info on the way this runs; a weekly cron job runs a script to update
games, ad after noticing that stellaris hadn't updated and some new
purchases were missing I've been poking at it with a remote desktop
session. I'm not sure how long it has been failing to download content.
This is the script that runs, very straightforward.
python gogrepoc.py update
python gogrepoc.py download /mnt/fakeraid/Games/__GoG\ Archives/ -os windows -lang en -skipfiles patch_* *_demo_* *.dmg *32bit*
Manual running update initially showed 12 updates found, same as the gog
website showed. Nothing was downloaded anyway with a subsequent download
run, although update no longer claims to see new content so it changed
something.
If I run without the os restriction, it does try to download stellaris
3.10.3 linux version, so it sees that one at least. If I run without
skipfiles or lang, no change.
If I remove a previous version, such as 3.7.4 which was most recently
downloaded, it does not attempt to redownload that; it will still say
nothing to download.
—
Reply to this email directly, view it on GitHub
<#94 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABKZ33YGAHEYIU75ZOHUGVTYI5QNZAVCNFSM6AAAAABAP3ASHSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TQMRTHAYDQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
For the verify outputs we can just start with the little summary at the
end, you don't have to send me the full thing.
…On Tue, 12 Dec 2023, 10:18 Kalanyr, ***@***.***> wrote:
The problem appears to be that you're letting update autodetermine what os
and language you're using ( which will probably be Linux for the OS ) but
specifying to download only windows.
You should run the update command with the same -os / -lang you use for
download.
However given how out of date the windows info in your manifest is you
probably need to disable the chron, manually run the following command:
python gogrepoc.py update -os windows -lang en -full
It should take about an hour for every 300 games you own.
Then you need to manually run your download command, that might take upwards of a day to finish depending on how many missing windows updates you have ( which is why we needed to disable the chron jobs for this )
Once that's done, I recommend you make use of the verify command:
python gogrepoc.py verify /mnt/fakeraid/Games/__GoG\ Archives/ -clean -os windows
Which should use the md5 data from GOG to make sure your files are correct, it should move any wrong ones to a subdirectory called !orphaned
That could take a long time if you have a lot of game files.
Let me know what verify outputs when it's down and I'll let you know how to proceed from there ( I think there's a reasonable chance there'll be a few issues given how out of date things probably are )
( Verify is safe to use in a chron usually because it remembers files that it's already verified so subsequent runs are much faster )
On Tue, 12 Dec 2023, 05:56 mharmless, ***@***.***> wrote:
The python script shows v0.4.0-a, and I believe it was downloaded from the
> master updated 8 months ago.
> I'm running it on a custom storage server that hosts media and game
> files. Off the shelf hardware, a basic install of linux mint 21, along with
> snapraid and mergerfs to handle the storage.
>
> I have done a small amount of scripting in a professional environment,
> but virtually nothing in python specifically. I can probably follow
> troubleshooting directions.
>
> More info on the way this runs; a weekly cron job runs a script to update
> games, ad after noticing that stellaris hadn't updated and some new
> purchases were missing I've been poking at it with a remote desktop
> session. I'm not sure how long it has been failing to download content.
>
> This is the script that runs, very straightforward.
>
> python gogrepoc.py update
> python gogrepoc.py download /mnt/fakeraid/Games/__GoG\ Archives/ -os windows -lang en -skipfiles patch_* *_demo_* *.dmg *32bit*
>
> Manual running update initially showed 12 updates found, same as the gog
> website showed. Nothing was downloaded anyway with a subsequent download
> run, although update no longer claims to see new content so it changed
> something.
>
> If I run without the os restriction, it does try to download stellaris
> 3.10.3 linux version, so it sees that one at least. If I run without
> skipfiles or lang, no change.
>
> If I remove a previous version, such as 3.7.4 which was most recently
> downloaded, it does not attempt to redownload that; it will still say
> nothing to download.
>
> —
> Reply to this email directly, view it on GitHub
> <#94 (reply in thread)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABKZ33YGAHEYIU75ZOHUGVTYI5QNZAVCNFSM6AAAAABAP3ASHSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TQMRTHAYDQ>
> .
> You are receiving this because you commented.Message ID:
> ***@***.***>
>
|
Beta Was this translation helpful? Give feedback.
-
|
Doing an update with -os windows and -ids stellaris followed by a download has it downloading the 3.10.3 version right now, so this looks to be exactly the problem. I must have run this in the past at some point specifying to download all OS data. For some reason I just assumed update grabbed all possible files and it was up to download to filter what was wanted from the pile. Thank you for helping me see the error. Manually tested against_the_storm, one of the new purchases, and same deal as stellaris it appears to work just fine now that update has the correct options to pull the right data. Full update is running now, will check back in on it later today and start the downloads. There are ~700 games in the library so it will take a bit to bring back the verify data, and in fact may end up having to move into January before finishing due to monthly data limits depending on how that looks. |
Beta Was this translation helpful? Give feedback.
-
|
Awesome, glad I could help. Keep me posted on the progress. If you need to
leave more than a couple of days between the downloads you should run a
standard update before doing more downloading too ( otherwise you're pretty
much guaranteed to be trying to download files that have since been removed
and replaced with new versions ).
…On Tue, 12 Dec 2023, 19:57 mharmless, ***@***.***> wrote:
Doing an update with -os windows and -ids stellaris followed by a download
has it downloading the 3.10.3 version right now, so this looks to be
exactly the problem. I must have run this in the past at some point
specifying to download all OS data. For some reason I just assumed update
grabbed all possible files and it was up to download to filter what was
wanted from the pile. Thank you for helping me see the error.
Manually tested against_the_storm, one of the new purchases, and same deal
as stellaris it appears to work just fine now that update has the correct
options to pull the right data.
Full update is running now, will check back in on it later today and start
the downloads. There are ~700 games in the library so it will take a bit to
bring back the verify data, and in fact may end up having to move into
January before finishing due to monthly data limits depending on how that
looks.
—
Reply to this email directly, view it on GitHub
<#94 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABKZ334NMV2SVDJCMX65H3DYJATCHAVCNFSM6AAAAABAP3ASHSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TQMRZGA2TQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
740 gigs to download, will definitely have to split that across the monthly limit reset. |
Beta Was this translation helpful? Give feedback.
-
|
22:11:36 | --totals------------ Summary looks good, the missing items are 32bit installers, I had a typo in my skipfiles entry when I typed in the verify command. Shocked at how relatively fast the verification was considering just how many games are in the set. Thank you again for your help, and have a good holiday season. |
Beta Was this translation helpful? Give feedback.
-
|
That's excellent, as I side not I recommend changing your chron to:
update
download
clean
verify
or
update
clean
download
verify
( there's trade offs either way, cleaning second is safer but can you use
extra bandwidth when there's only small changes to a file that can be
healed with the md5 chunk data , whereas cleaning second prioritises saving
bandwidth but can require an additional run of the download if the existing
files end up failing it's checks )
…On Fri, 15 Dec 2023, 13:27 mharmless, ***@***.***> wrote:
22:11:36 | --totals------------
22:11:36 | known items......... 1853
22:11:36 | pre-verified items.. 2
22:11:36 | have items.......... 1811
22:11:36 | skipped items....... 454
22:11:36 | missing items....... 42
22:11:36 | md5 mismatches...... 0
22:11:36 | size mismatches..... 0
22:11:36 | zipfile failures.... 0
22:11:36 | cleaned items....... 0
22:11:36 | --
22:11:36 | total time: 4:49:24.787877
22:11:36 | exiting...
Summary looks good, the missing items are 32bit installers, I had a typo
in my skipfiles entry when I typed in the verify command. Shocked at how
relatively fast the verification was considering just how many games are in
the set.
Thank you again for your help, and have a good holiday season.
—
Reply to this email directly, view it on GitHub
<#94 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABKZ333AXBKKE4IWKCFSR7TYJO7Q3AVCNFSM6AAAAABAP3ASHSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TQNJZHEZTS>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
Multiple games are just showing as no game items. Stellaris hasn't updated since 3.7.4, now shows blank items, and new purchases like against the storm download nothing at all.
python gogrepoc.py update -ids against_the_storm
fetches game details just fine, no errors reported. but downloading will show nothing to download presumably because !info.txt says so.
Hotfix 0.43.3 (February 3rd, 2023)
changlog cut short here. And this seems to be the problem for a laundry list of games
-- Age of Wonders 4 --
title.......... age_of_wonders_4
genre.......... Strategy
game id........ 1915432764
url............ https://www.gog.com/en/game/age_of_wonders_4
release date... May 02, 2023
game items.....:
-- Stellaris --
title.......... stellaris
genre.......... Strategy
game id........ 1508702879
url............ https://www.gog.com/en/game/stellaris
user rating.... 78%
release date... October 01, 2018
game items.....:
changelog......:
Beta Was this translation helpful? Give feedback.
All reactions