Failure to start JVM with jpype 1.5.1 on Windows

[villares]

With 1.5.0 on this env I could start the JVM.

C:\Users\alexandre.villares>python
Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import jpype
>>> jpype.__version__
'1.5.0'
>>> jpype.startJVM()
Picked up JAVA_TOOL_OPTIONS: -Djava.vendor="Sun Microsystems Inc."
>>> jpype.getDefaultJVMPath()
'C:\\Users\\alexandre.villares\\Downloads\\thonny-4.1.6-portable\\user_data\\jdk-17\\bin\\server\\jvm.dll'
>>>

Updating to jpype 1.5.1 it fails:

C:\Users\alexandre.villares>python -m pip install jpype1 --upgrade
Requirement already satisfied: jpype1 in c:\users\alexandre.villares\downloads\thonny-4.1.6-portable\lib\site-packages (1.5.0)
Collecting jpype1
  Downloading jpype1-1.5.1-cp310-cp310-win_amd64.whl.metadata (5.0 kB)
Requirement already satisfied: packaging in c:\users\alexandre.villares\downloads\thonny-4.1.6-portable\lib\site-packages (from jpype1) (24.2)
Downloading jpype1-1.5.1-cp310-cp310-win_amd64.whl (356 kB)
Installing collected packages: jpype1
  Attempting uninstall: jpype1
    Found existing installation: JPype1 1.5.0
    Uninstalling JPype1-1.5.0:
      Successfully uninstalled JPype1-1.5.0
  WARNING: Failed to remove contents in a temporary directory 'C:\Users\alexandre.villares\AppData\Local\Temp\pip-uninstall-lxb7b830'.
  You can safely remove it manually.
Successfully installed jpype1-1.5.1

C:\Users\alexandre.villares>python
Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import jpype
>>> jpype.__version__
'1.5.1'
>>> jpype.startJVM()
Picked up JAVA_TOOL_OPTIONS: -Djava.vendor="Sun Microsystems Inc."
Error occurred during initialization of VM
Could not find agent library instrument on the library path, with error: Can't find dependent libraries
Module java.instrument may be missing from runtime image.

C:\Users\alexandre.villares>python
Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import jpype
>>> jpype.getDefaultJVMPath()
'C:\\Users\\alexandre.villares\\Downloads\\thonny-4.1.6-portable\\user_data\\jdk-17\\bin\\server\\jvm.dll'
>>>

[Thrameos]

Do you happen to know the path to the org.jpype.jar? If it is locatable and on the path, then the next issue is why java.instrument is missing.

[villares]

Do you happen to know the path to the org.jpype.jar? If it is locatable and on the path, then the next issue is why java.instrument is missing.

I'll try to find out (I don't have access to Windows at home) and I'll report!

[villares]

Maybe some new information!

I have reproduced the issue on another machine. org.jpype.jar in both 1.5.0 and 1.5.1 is present at:
C:\Users\abav\Downloads\thonny-4.1.6-portable\Lib\site-packages\org.jpype.jar

Python 3.10.11 (C:\Users\abav\Downloads\thonny-4.1.6-portable\python.exe)
>>> import jpype
>>> jpype.__version__
'1.5.0'
>>> jpype.getDefaultJVMPath()
'C:\\Users\\abav\\Downloads\\thonny-4.1.6-portable\\user_data\\jdk-17\\bin\\server\\jvm.dll'
>>> jpype.startJVM()
>>> 

Then:

Python 3.10.11 (C:\Users\abav\Downloads\thonny-4.1.6-portable\python.exe)
>>> import jpype
>>> jpype.__version__
'1.5.1'
>>> jpype.getDefaultJVMPath()
'C:\\Users\\abav\\Downloads\\thonny-4.1.6-portable\\user_data\\jdk-17\\bin\\server\\jvm.dll'
>>> jpype.startJVM()
Error occurred during initialization of VM
Could not find agent library instrument on the library path, with error: Can't find dependent libraries
Module java.instrument may be missing from runtime image.

Process ended with exit code 1.
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Python 3.10.11 (C:\Users\abav\Downloads\thonny-4.1.6-portable\python.exe)
>>> 

Fiddling around I noticed jpype.startJVM() worked fine if I didn't set JAVA_HOME and jpype found jre-1.8

Python 3.10.11 (C:\Users\abav\Downloads\thonny-416-plain\python.exe)
>>> import jpype
>>> jpype.getDefaultJVMPath()
'C:\\Program Files\\Java\\jre-1.8\\bin\\server\\jvm.dll'
>>> jpype.startJVM()
>>> 

So I moved my jdk-17 to a similar place, and it worked!

Python 3.10.11 (C:\Users\abav\Downloads\thonny-416-plain\python.exe)
>>> import jpype
>>> jpype.getDefaultJVMPath()
'C:\\Program Files\\Java\\jdk-17\\bin\\server\\jvm.dll'
>>> jpype.startJVM()
>>>

Now.... what is wrong with the previous jdk-17 path? I have no idea and I hope you can figure out!

[Thrameos]

It sounds like someone has shipped a partial copy of the JVM. There are lots of reasons that that can happen, though there is never a guarantee we will run without a full JVM. It must be JE only version. I will have to have to make some options to catch that case.

[Thrameos]

It seems like the best plan would be to make agent an option rather than the default and add logic to specifically add org.jpype to the classpath automatically. Our original solution was bad because our module was a second class citizen cause we were never loaded into the main system class loader. This always presented a problem because I run my testing with org.jpype on the classpath which meant I was constantly getting better results than typical runs. I tried to rectify this by loading as agent which has the huge advantage that we guarantee to load first, and we can register ourselves as the controlling module. Running as an agent is a privileged position, but it requires java.instrumentation module which apparently some distributions are shipping without. The problem is that we have thus created a new requirement and someone is shipping out stripped down copies of the JVM. Stripped down copies are for light installs and typically used when you are serving just one application thus you know in advance all the classes that need to be provided. There was NEVER a guarantee that JPype will run with a stripped down JVM. It is always subject to breakage because at any time we may add a new call to org.jpype and if stripped down JVM doesn’t have it then it will fail. So the logic choice is try to get everything working as a first class citizen and do a 1.5.2 release. I am guessing a month or so depending on my availability over the holidays. In the meantime either 1.5.0 or if Python 3.13 is required then we must use a full copy of the OpenJDK.

[ryanmkurtz]

I just did a bit of testing and it seems the problem is actually occurring on a full OpenJDK from https://jdk.java.net/23/. This is not the distro I typically use but I bet a lot of people do.

UPDATE: Actually something else weird might be going on in my case....OpenJDK may be fine after all.

[ryanmkurtz]

The real issue is looking like a conflict between 2 different versions of Java being used at the same time...one on the PATH, and a different one on JAVA_HOME.

[astrelsky]

I may have an appropriate fix. It seems to be an issue that has been present in jpype for a long time and the changes in 1.5.1 just only caused it to appear. I'll give it a go tonight.

[ryanmkurtz]

A temporary workaround that I've confirmed works is to ensure that your JAVA_HOME matches the Java that is on your path. I'm guessing that those don't match for everyone this isn't working for.

[Thrameos]

@ryanmkurtz Please test with #1245

@astrelsky I put a PR that fixes it on my side, but as I still don't know the exact origin of the problem there are still questions. I am going to also take a shot at the non-ascii path issue in the same go.

[ryanmkurtz]

I can test on Friday.

[stulacy]

A temporary workaround that I've confirmed works is to ensure that your JAVA_HOME matches the Java that is on your path. I'm guessing that those don't match for everyone this isn't working for.

In case it's useful, although it looks like it's been solved, but this wasn't the case for me.

PS C:\Users\sl561
> echo $Env:JAVA_HOME
C:\Users\sl561\AppData\Local\Programs\Microsoft\jdk-11.0.25.9-hotspot

PS C:\Users\sl561
> which java
/c/Users/sl561/AppData/Local/Programs/Microsoft/jdk-11.0.25.9-hotspot/bin/java

[Thrameos]

I also see reports of some failures on osx. So clearly there is more than one issue. We are currently guessing:

1. partial jvm installs (missing instrumentation)
2. conflicted jvm (path and java_path mismatch)

We have two PR addressing these. Hopefully there isn't a third.

[ryanmkurtz]

Just doing a quick check in on the progress of JPype 1.5.2 to address this issue.