Julia

[krosenfeld-IDM]

From Open Design Questions:

Does it make sense to provide a Julia version of our model? What would that entail? What are the pros and cons?

@jonathanhhb

[krosenfeld-IDM]

from @KevinMcCarthyAtIDM

I think we made a functional decision here, and maybe without ever seriously exploring implementation options. To keep momentum going, I think we should continue developing within a Python framework. However, I'd like to identify a point to come back around to this - I expect we will likely hit a place where we have a v0 of this reference design, and now we're thinking about the v1, seeing where choices we made are constraining or awkward, and planning a bit of coding push / refactor. I think that makes a sensible time to revisit the choice of core technology.

[krosenfeld-IDM]

I'm not sure I see the benefit of providing a julia version (implying that there is another version such as python or R). If we want to use julia I see it as a commitment - the code is written in julia, utilizing the benefits of julia that would not available in e.g. R or python.

[jonathanhhb]

That's an interesting and potentially important way to frame the question. Do we want to do a Julia version, or do we want to pivot to Julia?

I'm not sensing any real organizational appetite or driver for the latter. I think Julia would have to offer almost a 10x advantage over what we're already doing with Python and compiled C/OpenMP/SIMD and/or Numba to outweigh the downside risks of "accessibility". And I don't think we have any reason to believe it does at this point.

That said, personally and professionally I would love us at some point -- like after we have a v1, as Kevin suggests -- to do some kind of due diligence exploration in which we build something equivalent in Julia even if it gets thrown away, so we can know rather than guess what the specific pros and cons are.

Finally, just to respond to:

"I think we made a functional decision here, and maybe without ever seriously exploring implementation options."

1. I like to avoid implicit decisions that we didn't seriously explore. :) Hence my pushing on some of the other conversations.
2. I think Chris and I did do some Julia exploration earlier this year and I do feel like the decision we made was at least rational and defensible -- which isn't the same as saying that I know it was correct. That's not supposed to come across as defensive, but I'm comfortable with where we're at on this.

If we had another whole developer person, I would feel comfortable tasking them with full-time Julia and Rust explorations in parallel with the work Chris & I are doing, but that might be a little thankless spending all one's time on projects that have a very small likelihood of being used. :D

[KevinMcCarthyAtIDM]

@jonathanhhb & @krosenfeld-IDM : I think both of you have hit the same point, and I agree. A Julia "version" probably isn't worth much - maybe this is over broad, but I assume that because Python is the more dominant paradigm, more of the Julia community went that direction by "active choice" than by "default", and so most potential LASER users who are "Julia-first" coders are probably Python-competent enough to use it. If we're looking at "versions" as a way to capture more users, R interoperability makes more sense.

@jonathanhhb - Thanks, and point taken. I didn't mean to imply that no exploration was done, or that the decision to move forward with Python wasn't justified or defensible! And recognize as well that you could just as easily say that I didn't seriously engage with the idea of an SQL-based model.
I think I still would have liked to try out an implementation of a basic Spatial SEIR in Julia that adheres to some of the broader design decisions, just to see what it looks like as a user. But definitely recognize that resources are limited, and that picking a dominant paradigm and moving forward in that has enabled exploration of a whole bunch of other questions, optimizations, etc. that might have been delayed by prolonging the language choice question. But glad to hear you're on-board with a revisit when there's more of a v1 (which will be a logical point to revisit lots of decisions we made along the way). Depending on when that happens, I might be able to argue for some IDM contractor budget for it, to get "another whole developer person" involved; that wasn't an option for 2024.

I think it's super interesting for this problem, and also know that part of my interest is that I think the organization always ought to be exploring the whole universe of new tools available to us. I think that's pretty common practice by software professionals, to be aware of new entrants to the field, new standards, etc. For us researchers, it's not always so much of an expectation - for one, we're generally interested in a smaller set of languages specifically geared at scientific computing, visualization, data analysis, and for a pretty broad set of research use cases any of the R, Python, Julia, MATLAB ecosystems are roughly "equally good", and so people can get pretty comfortable staying in their favorite languages over a whole career. So I'd kind of love to be part of a project done in Julia as a learning experience and with an explicit parallel goal of evaluating Julia in general as a language for IDM-y problems.

[krosenfeld-IDM]

I recall that @clorton had looked into some of the parallelization (multi-threading and GPU) and performance (particularly vs numba) and @jonathanhhb had gotten julia installed and looked some performance metrics using notebooks? Sorry if I missed anything, just want to make sure I understand. I think @KevinMcCarthyAtIDM and myself had shared some code / basic implementation between the group... I can dig that up for the thread later - might be good to capture if we do revisit.