Reviewing the Protocol

[essepuntato]

Hi @Asemica-me @Pepe066 @Sergpoipoip @angstigone,

As I mentioned in today's lecture, you should review the Protocol developed by the others. You can work as a group and/or provide personal comments to the others, to write constructive critiques that the others can use to improve the Protocol. Please use the commenting functionality GitHub offers in the issue panel to provide comments - you can specify one or more comments, depending on how you would like to organise the discussion.

Some initial suggestions for reviewing the Protocol can be found online at the PLOS ONE Guidelines for Reviewers webpage, which has a section dedicated to Study Protocols submissions. Please, follow them to review the Protocol.

If you have any more questions, please don't hesitate to contact me in the Discord group.

[Pepe066]

The protocol in my opinion, rapresents a good starting point but, as discussed in class, some informations like the fields of the CSV or specific tools used could be added.

Additionally, the protocol covers the majority of the applicable considerations made here:(https://journals.plos.org/plosone/s/reviewer-guidelines#loc-study-protocols).

[Asemica-me]

Protocol remarks:

• Clarify point 3:
  The current description, “From Software Heritage use API to identify and extract metadata related to code/software authored by UNIBO researchers”, is too general. As noted in our preliminary analysis, metadata retrieved from Software Heritage (SWH) is returned in JSON format. A more precise and reproducible approach for interacting with the API and processing this metadata should be provided.

• At the beginning of the Data Gathering section, consider adding three widgets summarizing the raw datasets. Each widget should display the dataset title and a placeholder link for the data source origin (e.g. an endpoint or something related), which will be updated once available. These will be followed with a more detailed of our datasets eventually.

• Specify expected otput in point 7:
  Point 7 lacks a clear definition of the expected outcome. Detailing the format or structure (e.g. dataframe headers) of the final mashup dataset or output would improve clarity.

• Write down our RQs in "Data Analysis" section, like:

  RQ1: What is the current coverage of these kinds of research objects created by the personnel of the University of Bologna in existing (institutional, disciplinary, and generalistic) repositories?
  RQ2: Is there any overlap among these repositories – i.e. research objects deposited in more than one?
  RQ3: How many citations (incoming and outgoing), as in OpenCitations (https://opencitations.net/), are these research objects involved in?
  RQ4: How much of such research objects are actually mapped in IRIS?

• Mention use of Python and dependencies:
  Python is used in the project, but this is not mentioned explicitly. A brief note on the programming environment and dependencies would enhance reproducibility. I recommend adding a short “Before You Start” paragraph immediately after the abstract:

  Before You Start
  Ensure that Python 3.x is installed on your system. To run the provided scripts correctly, install the required libraries listed in the requirements.txt file. OpenRefine is also used for data cleaning and transformation. [...]

[Sergpoipoip]

Based on PLOS ONE Guidelines:

The protocol prerequisites -- maybe it's better to remove Findings section or move to future work, as no analyses should have been executed

RQs -- for clarity it worth to spell out the RQs directly

Technical soundness -- matching across repositories solely by DOI/title similarity may under‑detect overlap where identifiers are missing; it makes sense to clarify matching heuristics

Data availability -- the protocol promises CSVs and notebooks but does not specify where they will be deposited. Adding Data Availability section that explains where (in Zenodo?) the raw and processed data will be deposited

Manuscript title -- should contain the word "Protocol"

[vattelalberto]

We decided not to add specifications about Python and dependancies since they would be all placeholders at this stage, and we think this should be something to add in a "Before You Start" section only in the final version of the protocol.

Unfortunately, we were not able at this stage to express in more detail the approach for interacting with the Sfotware Heritage API.

All other remarks were incorporated in the new version of the protocol, which if we have correctly understood needs to be approved by @essepuntato as the owner of the protocol (but should still be accessible at https://www.protocols.io/view/open-science-24-25-protocol-gyyabxxsf even if it is not still published)

[Asemica-me]

Unfortunately, we were not able at this stage to express in more detail the approach for interacting with the Sfotware Heritage API.

@vattelalberto here is an initial attempt for interacting with the SWH archive:

import json
import requests
from tqdm import tqdm
import time

# Utility to pretty-print json
def jprint(obj):
    print(json.dumps(obj, sort_keys=True, indent=4))

base_url = "https://archive.softwareheritage.org/api/1/"
university_keywords = ["unibo", "alma mater", "alma mater sudiorum", "university of bologna", "almamaterstudiorum", "università di bologna", "universita di bologna"]  # Keywords to search for origins

found_origins = []

print("Searching for origins related to the University of Bologna...")

for keyword in tqdm(university_keywords, desc="Searching for keywords"):
    search_url = f"{base_url}origin/search/{keyword}/"
    try:
        resp = requests.get(search_url)
        resp.raise_for_status()  # Raise an exception for bad status codes
        origins = resp.json()
        print(f"\nFound {len(origins)} entries for keyword: '{keyword}'")
        for origin in origins:
            if origin not in found_origins:
                found_origins.append(origin)
                print(f"- URL: {origin['url']}")
        time.sleep(2)  # Be polite to the API
    except requests.exceptions.RequestException as e:
        print(f"Error during search for '{keyword}': {e}")

print("\n" + "=" * 40)
print(f"{' ' * 10}Total Unique Origins Found: {len(found_origins)}")
print("=" * 40 + "\n")

As you can see if you run the code, the API queries return results in JSON format; however, this script primarily focuses on retrieving the number of results (that is "Total Unique Origins Found: 152") matching the specified keywords.
Extracting additional metadata is more complex and requires further consideration.

[vattelalberto]

Protocol v2 https://dx.doi.org/10.17504/protocols.io.4r3l264ppv1y/v2