Feature Request: Watch for updates to the underlying .jl file

[MasonProtter]

One thing that would be nice is if I have a Pluto notebook, Foo.jl, and then if I opened Foo.jl in my editor, changed and saved something in the editor, I would want the Pluto notebook to automatically update.

[fonsp]

It's not part of my overall goal of making computing more accessible, but it is something that I (as an experienced programmer) would like to experiment with.

I am also reluctant, I feel like it is a lot of work to get this right, because there is no obvious bijection between Julia files and Pluto notebooks. See my comments here about why this is difficult: #131 (comment) . One thing that I would like to emphasize is that Pluto is a project to design an IDE, this is what I find fascinating to work on!

[shashi]

One thing that was annoying to me is if I have a notebook open, and do a git pull which applies a change cleanly to the notebook, I still see the old notebook until I restart Pluto (refreshing the page also doesn't change it.)

[antoine-levitt]

It's not part of my overall goal of making computing more accessible, but it is something that I (as an experienced programmer) would like to experiment with.

I feel there is a bit of a false dichotomy here - that experienced programmers want this kind of features, but novice users should not interact with text any other ways than through a plain text box. Inputting text is a serious business, one to which entire projects have been dedicated, and which is not easy to replicate in a project such as Pluto (and indeed Jupyter as well). Anybody serious about computing (in the broadest sense) should spend at least some time learning the tools of the trade (any good editor, as long as it's not a plain text box), and hybrid solutions (eg a notebook that allows users to use a text editor) are a productivity boost for any user. Attaching Pluto notebooks to a particular file and having two-way synchronization seems to be a very good way to achieve that. I understand it's not a priority though.

[fonsp]

Thanks for your input! I did not look at it that way :)

[fonsp]

PRs welcome as an experiment, but it's not a priority, and it will not be implemented if it gets too complicated.

[Moelf]

can we make an intermediate step where we can manually refresh a notebook?

Pluto already has a way of tracking if the cell has been changed, can we manually ask pluto to:

1. update cell text
2. check each cell if changed
3. do normal pluto thing?

this should pave the way for auto watching&update

if someone can point me to relevant code I'm willing to take a dive

[fonsp]

That would be a good intermediate step for the experiment! But the end result should definitely be automatic. Some pointers (but I'm afraid that this experiment still needs a general understanding of how pluto works which is an adventure that you need to take some time for):

You can deserialize (load) a notebook file into a Notebook struct using this function:

Pluto.jl/src/notebook/Notebook.jl

Line 138 in 928bf8e

function load_notebook_nobackup(io, path)::Notebook

Then you compare that to the current Notebook. How you get it depends on where your code will live, but it is stored in the current ServerSession. You can also run your code in an @async while true as an experiment here:

Pluto.jl/src/webserver/SessionActions.jl

Line 48 in 928bf8e

nb

[fonsp]

Schermopname.2021-03-26.om.01.42.33.mov

[MasonProtter]

Amazing @fonsp! Is there a public branch where one can play around with this? Is it just #1008?

[antoine-levitt]

I really like the concept of stateless notebooks, but there's no way I'm leaving emacs to write code, so I'm super excited for this, can't wait to try it out!

[fonsp]

no way I'm leaving emacs to write code, so I'm super excited for this, can't wait to try it out!

I should say that this use case is more involved than just auto-refreshing Pluto when the file changes, because editing the notebook file can lead to invalid notebook files. The file format is not designed to be human-written (and I am not personally interested in making that). This is the concern that I wanted to express in #240 (comment)

The primary reason to implement this feature is git #240 (comment), and we are experimenting in using it for #1045 and https://github.com/JuliaPluto/PlutoSliderServer.jl

[antoine-levitt]

Sure, but being able to do interactive things in Pluto but edit larger cells (like function definitions) in emacs would already be nice.

[holomorphism]

I've been trying PR #1029 for a while to edit notebook files directly in my editor, and this feature is extremely useful. Thank you very much!

In my environment (Ubuntu 18.04 & vim), it's working fine so far.

As a tip for Vim users, to get the same feel as VS Code in Fons' video, you may want to enable the autoread feature in vim.
https://stackoverflow.com/a/18866818

[GuanglinDu]

@fonsp A similar problem: Pluto cannot update the file with new changes edited with another Pluto.

Scenario: I run Pluto in the cloud and locally on my desktop or laptop PC at the same time, and work on both sides as needed. I use Syncthing to sync all the files between devices. I only edit the files with Pluto from the browser. However, Pluto cannot update the changes from either side, even if I refresh the browser or re-open the *.jl file. I checked the *.jl files with textual editors and found that they're in sync all the time.

Workaround: Edit only one-sidedly. Chances are that I'll have to work outside of my office network with my laptop running Pluto locally. Once our office network is available, Syncthing will automatically update the files on the server side. However, the changes cannot show up in Pluto. Any suggestions?

[fonsp]

@GuanglinDu did you try the new feature? You need to enable it with Pluto.run(auto_reload_from_file=true)

[GuanglinDu]

@fonsp Many thanks. That flag does the magic!
julia> import Pluto; Pluto.run(;host="0.0.0.0", port=1234, launch_browser=false, require_secret_for_open_links=false, require_secret_for_access=false, auto_reload_from_file=true)