Accessing internal functions

[Aidajafarbigloo]

Hello @dgarijo ,

We have been exploring ways to enhance the capabilities of our tool, and we believe that utilizing Somef's internal functions could help us in this way.

Our purpose is to extract and export more comprehensive metadata with the help of Somef. Currently, we are able to obtain the results from Somef in JSON files, but for further processing within our tool, we require access to the extracted metadata from Somef in the form of Python dictionaries.

We tried to modify the Somef source code to get the result from functions like "save_codemeta_output ", which export CodeMeta format of Metadata, in a form of Python dictionary. However, we recognize that accessing the functions, which export metadata, directly would be a more ideal solution, and give us the possibility to use all three main output formats in form of python dictionaries, not stored in JSON files. Could you share any advice on solving this problem?

[dgarijo]

Hello, Thanks for your interest in the tool! If you use cli_get_data function in the somef_cli class you will obtain a an in-memory dictionary returned by the tool (no files are stored). Is this what you are aiming for?

[Aidajafarbigloo]

Hello,

Thank you for the answer.
Yes, right. Also we'd like to access the other output formats of Somef (e.g.: CodeMeta). Using cli_get_data function helps to get the main output.

[dgarijo]

For ttl you have to use data_graph.somef_data_to_graph(repo_data.results). This export has to be improved a bit but should work.
For the codemeta dictionary, have a look at save_codemeta_output, which will do the transformation from the dictionary to codemeta and print it out. In this case, the function prints the codemeta file rather than return a dictionary. I can separate it in two functions if you prefer

[Aidajafarbigloo]

Thanks, it would be great to have them separated and access CodeMeta dictionary and not producing a file.

I'm trying to test save_codemeta_output, but I face:

I've changed my GitHub token, and reconfigured Somef, it shouldn't related to hitting the rate limitation, right?

[dgarijo]

If you have the token correctly configured, you should be able to do several thousands repositories per hour. It looks like the error is because the zip file of that repo is/was not available when you tried. I recommend retrying. i just did now and it works:

Downloading https://github.com/dgarijo/Widoco/archive/master.zip
13-Mar-24 15:01:57-DEBUG-Starting new HTTPS connection (1): github.com:443
13-Mar-24 15:01:57-DEBUG-https://github.com:443 "GET /dgarijo/Widoco/archive/master.zip HTTP/1.1" 302 None
13-Mar-24 15:01:57-DEBUG-Starting new HTTPS connection (1): codeload.github.com:443
13-Mar-24 15:01:57-DEBUG-https://codeload.github.com:443 "GET /dgarijo/Widoco/zip/refs/heads/master HTTP/1.1" 200 None

You'll probably see some errors for this repo, but it works fine in the end.

[Aidajafarbigloo]

Thank you, the token was configured correctly, but the error was because of the scope of the token. I was using a token with not sufficient permissions.

[dgarijo]

I see. We should refine the errors that we get when the token is incorrect. I guess another issue to open :(