Google Drive downloads broken because of updated API

[spijet]

Hello @PhasecoreX!

First of all, thank you for your awesome work in dockerizing the dedicated servers for UT99 and UT2004!

Today I tried to deploy a UT2004 server using your image and found that the GDrive link always responds with a "this file may contain viruses" banner, and now there's no confirm field in the cookie the server sends to the user. After some digging around I found that the confirm cookie is now always t, but now we also need to send a unique-for-every-request UUID that is contained inside the banner page:

<form id="download-form" action="https://drive.usercontent.google.com/download" method="get">
    <input type="submit" id="uc-download-link" class="goog-inline-block jfk-button jfk-button-action" value="Download anyway">
    <input type="hidden" name="id" value="XXXX">
    <input type="hidden" name="export" value="download">
    <input type="hidden" name="confirm" value="t">
    <input type="hidden" name="uuid" value="9a863b7f-d1d3-43aa-8b4f-f9352715d79e">
</form>

I'm currently trying to come up with a good enough way to parse it and will ping you back if I get any meaningful results.

[spijet]

I've created #20 with a fix that works for me, please check it out when you have time.

[PhasecoreX]

Hello! Glad you and the community are enjoying the containers! Makes me very happy to see them being useful, as I love these games.

I don't have access to a computer at the moment, so I can't do much debugging right now. But, I saw this gist was updated recently, maybe it can help? https://gist.github.com/tanaikech/f0f2d122e05bf5f971611258c22c110f Otherwise, we can look into choosing another less complicated file host.

[spijet]

I tried the approach from the gist and it seems to be working:

❯ curl -L 'https://drive.usercontent.google.com/download?id=${archive_id}&export=download&confirm=t' -O
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
 12  711M   12 88.2M    0     0  21.1M      0  0:00:33  0:00:04  0:00:29 21.1M

So maybe I over-engineered a bit. :D

I can add this new URL to the script in my PR instead of the pup-based solution, if that's OK.

[PhasecoreX]

Yes, if you can update it to the new URL one instead of pup, that would be appreciated! Once that's done I'll merge it, and then we can update the UT99 one as well.

I just worry that google's going to update this again and break it lol. Guess I'll follow updates on that gist.

[spijet]

In that case we could try to use the pup approach after a fail or, for example, when some specific environment variable is set (e.g. GDRIVE_SCRAPE_API, with false as the default), what do you think?

[PhasecoreX]

Eh, I'd say just go with the direct URL option, no need to complicate things. I don't really want a ton of google specific hacks if I can help it, and if they break this new URL thing I'll just switch to another provider to host the file.

[spijet]

OK, I'll update the PR right away.

[spijet]

Also, maybe it would make sense to feed the archive URL and expected checksum via environment variables as well (with some sane defaults set in the Dockerfile)? This will allow people to use any mirror/hosting they like without modifying the image.

[PhasecoreX]

At the moment, they can just use SKIP_INSTALL environment variable and manually download the server themselves. But having a download and checksum base server override environment variable might be a good idea. Only issue is that I want to keep both this and the ut99 downloader script the same, and with the ut99 one, there's the base server and then the 469d version update. Technically if there was ever an update to ut2004, that would be included here as well, but I'm not holding my breath lol. So would specifying your own download URL mean just the server download, or server and any potential updates? If the latter, then the SKIP_INSTALL would make more sense I guess.

[freedbygrace]

Wow! Just wow! This is amazing and I would very much like to play in the future or get connected to some community that still plays? In any case, I was wondering if bolstering this container up with the packages required for a redirect server and some kind of automated way to compress additional files when detected. That would make the client be able to connect with fast downloads sourcing from the server or a separate container as part of a stack with a read-only volume to the server files. From there, a reverse proxy could be used to serve the files, but somehow I don't think the game is new enough to support SSL?