How to set a different .CondaEnv directory?

[Datseris]

I have a Julia project that I sync over the cloud in my work and home computers. I've been having a lot of delay and sync problems with this project, and only now I've realized why: PythonCall.jl makes a local folder, in the same directory as Project.toml, that has thousands and thousands of files, which is a bad idea for any online cloud-based syncing software. The folder is called .CondaEnv.

How can I tell PythonCall.jl to not make this local folder, and instead make it somewhere else, like in e.g., my julia package installation folder?

The documentation yields 0 results when I search "CondaEnv". I believe there should be something in the docs about this.

[Datseris]

I've found this in the docs now, when searching for conda instead of CondaEnv: https://juliapy.github.io/PythonCall.jl/stable/pythoncall/#pythoncall-config

It tells me that I can specify a Python directory. I don't have python and I am fine letting PythonCall.jl do all the configuration. I trust it more than me. So the question remains, is there a way to tell PythonCall.jl to install python itself, just not locally on the same folder as my Project.toml?

[Datseris]

From here: https://github.com/JuliaPy/CondaPkg.jl?tab=readme-ov-file#conda-environment-path

it appears that if I set something like ENV["JULIA_CONDAPKG_ENV"] = "@python" before installing, it will use a shared environment instead.

But, for this to work I need to first install CondaPkg.jl by itself, so that it registers the above option. And then use also the option ENV["JULIA_CONDAPKG_BACKEND"] = "Current".

So, the solution is:

Set

ENV["JULIA_CONDAPKG_BACKEND"] = "Current"
ENV["JULIA_CONDAPKG_ENV"] = "@python"

in your startup.jl. Then Install CondaPkg by itself. Then install PythonCall.

[cjdoris]

Yep that is indeed the best solution to your problem right now.

A downside is that this uses the same Conda environment for all your Julia projects, whereas the default behaviour uses a different one for each project, maintaining separation of dependencies. But you might not mind that.

It would be nice if CondaPkg had an option to keep project-specific Conda environments somewhere other than the project directory itself, but such a feature doesn't exist yet.

[Datseris]

whereas the default behaviour uses a different one for each project, maintaining separation of dependencies. But you might not mind that.

I don't understand this statement. It does not appear correct to me (forgive my ignorance). THe dependencies are tracked in the CondaPkg.toml file, that remains local to the project. Like Project.toml. Hence, the dependencies remain separated. It is their binaries that are not separated, they are all in the same folder, but why would someone care about where the dependencies are installed? Julia has shown us rather clearly that having all the package binary/source code at the same place is better as it allows reuse.

Am I mising something?

[cjdoris]

No you're right I was being unclear, the dependencies are indeed still separated. The difference comes if you switch between multiple Julia projects which use the same shared @python Conda environment - you will find that each time you switch project, the dependencies will be reinstalled because the set of packages that should be in the environment has changed. This is of course fine if you rarely switch projects.