[Refactor] Expose a higher-level API

[Yoric]

We expect that our users will be more interested in machine-learning than in quantum computers, at least at first, so we expose a new, higher-level, API, that hides most of the quantum details. With this API, switching between the QutipEmulator, emu-mps or QPU is just a few lines of code (well, one line of code + the connection details, username, password, project id).

This results in a tutorial that spends less time on the quantum aspects and more on the machine-learning.

Also:

• more tests;
• more doc;
• logging;
• a workaround for We sometimes produce graphs that pass is_embeddable but cannot be compiled #29 .

More details

This change comes from a conversation with @ferrulli1pasqal, who suggested that we do not want to overwhelm our users with details on sequences, devices, etc. during the tutorial. So we now have an API that handles all those details, filters out graphs that cannot be compiled to sequences or executed on the device, and also handles saving the processed data while we're at it.

Before this change, to run the QutipEmulator, we executed

list_of_graphs = []
for data in tqdm(og_ptcfm):
    graph = qek_datatools.MoleculeGraph(data=data, device=pl.AnalogDevice)
    list_of_graphs.append((graph, graph.pyg.y.item()))

dataset_sequence = []

for graph, target in tqdm(list_of_graphs):
    # Some graph are not compatible with AnalogDevice, just skip them.
    if graph.is_embeddable():
        dataset_sequence.append((graph.compute_sequence(), target))

# In this tutorial, to make things faster, we'll only run the first compatible entry in the dataset.
# If you wish to run more entries, feel free to increase this value.
MAX_NUMBER_OF_DATASETS = 1

for seq, target in tqdm(dataset_sequence[0:MAX_NUMBER_OF_DATASETS]):
    simul = QutipEmulator.from_sequence(sequence=seq)
    results = simul.run()

after this change, to run it, we execute

compiler = qek_graphs.MoleculeGraphCompiler()
# Use the Qutip Extractor.
extractor = qek_extractors.QutipExtractor(
    # Once computing is complete, data will be saved in this file.
    path="saved_data.json",
    compiler=compiler
)

# Add the graphs using the compiler we've picked previously.
extractor.add_graphs(graphs=og_ptcfm)

# We may now compile them.
extractor.compile()
max_qubits = 5
`processed_dataset` list[ProcessedData] = await extractor.run(max_qubits=max_qubits) # Don't forget to `await`!

to use emu-mps, just replace QutipExtractor with EmuMPSExtractor. To use a QPU, just replace it with QPUExtractor (and specify username, password, project id and optionally, the batch_ids if you're resuming from a previous computation).

[Yoric]

Cc @MatthieuMoreau0 for the use of the QPU.
Cc @a-quelle for the use of emu-mps.

[a-quelle]

I've not looked at all the code in detail, since I don't consider myself a contributor to the package. Regarding the API, I think at some point we should add the ability to configure the backends. @Yoric suggests doing that in a separate PR, which is fine by me, but it really should be done at some point, because at larger qubit numbers, emu-mps becomes increasingly dependent on good config values.

[Yoric]

I've not looked at all the code in detail, since I don't consider myself a contributor to the package. Regarding the API, I think at some point we should add the ability to configure the backends. @Yoric suggests doing that in a separate PR, which is fine by me, but it really should be done at some point, because at larger qubit numbers, emu-mps becomes increasingly dependent on good config values.

As a side-note, my assumption is that, at some point, as we publish more open-source packages, this class hierarchy will move to another library and will progressively grow into something quite generic. So we will definitely want more configuration. On the other hand, we may want to wait until we have several applications before we make it overly generic.

[RolandMacDoland]

Hey @Yoric what you seem to be doing feels very much like what Qadence2-Platforms provides. Myabe we should discuss it together with @Doomsk ?

[Yoric]

Hey @Yoric what you seem to be doing feels very much like what Qadence2-Platforms provides. Myabe we should discuss it together with @Doomsk ?

Happy to discuss, but if I recall, we cannot depend on qadence2 because we expect to release QEK in ~3 weeks, while qadence2 is expected much later.

[RolandMacDoland]

Hey @Yoric what you seem to be doing feels very much like what Qadence2-Platforms provides. Myabe we should discuss it together with @Doomsk ?

Happy to discuss, but if I recall, we cannot depend on qadence2 because we expect to release QEK in ~3 weeks, while qadence2 is expected much later.

Yep, but that can be in the scope of some longer term roadmap.

I am absolutely interested in moving to a qadence2 back-end once that is available, yes!

[RolandMacDoland]

Thanks @Yoric few minor comments from my side otherwise LGTM.

[MatthieuMoreau0]

nit: this is a list so I suggest naming this batch_ids

[MatthieuMoreau0]

Unless I'm missing something, this is going to loop forever as we are not refreshing the batch data at each iteration

This may be a sign that we are missing a test for the qpu extractor; where we simulate the sequences execution taking a few iterations to run to completion

[Yoric]

Oh, yes, we're absolutely missing a test.
I'll try and find time to write one.

[MatthieuMoreau0]

currently max runs for Fresnel is 500, so the batch creation fails. Either create two jobs with 500 runs each, or lower this value to 500 if that's sufficient