Fixed bug, download.py will now reject responses from captive portals…

… and will not replace existing cdf and netcdf files.

In some cases with captive wifi portals, the wifi responds with 200-OK but the downloaded file is not the file requested by the user, it only contains some html code, usually asking the user to login into the system. Previously, this fake file could replace an already existing cdf file, and then it would be deleted since it cannot be loaded into tplot. This bug was fixed, and now download.py will reject the fake file before replacing any existing files. This only works for cdf and netcdf files.

pyspedas/utilities/download.py

@@ -225,7 +225,8 @@ def download_file(
         return None
 
     if needs_to_download_file:
-        ftmp = NamedTemporaryFile(delete=False)
+        froot, fsuffix = os.path.splitext(filename)
+        ftmp = NamedTemporaryFile(delete=False, suffix=fsuffix)
 
         with open(ftmp.name, "wb") as f:
             if text_only:
@@ -241,13 +242,18 @@ def download_file(
             os.makedirs(os.path.dirname(filename))
 
         # if the download was successful, copy to data directory
-        copy(ftmp.name, filename)
+        if check_downloaded_file(ftmp.name):
+            copy(ftmp.name, filename)
+            logging.info("Download complete: " + filename)
+        else:
+            logging.error("Download of '" + filename + "' failed. The temp file will be removed.")
+            logging.error("If the same file has been already downloaded previously, it might be possible to use that instead.")
 
+        # cleanup
         fsrc.close()
         ftmp.close()
         os.unlink(ftmp.name)  # delete the temporary file
-
-        logging.info("Download complete: " + filename)
+
 
     # At this point, we check if the file can be opened.
     # If it cannot be opened, we delete the file and try again.