3289 harvest via oai pmh of rein d space fails#3321
Conversation
…pes of exceptions caught
| if(this.protocol == null || this.authority == null || this.identifier == null) { | ||
| return ""; | ||
| } else if(HTTP_PROTOCOL.equals(this.protocol) || HTTPS_PROTOCOL.equals(this.protocol)) { | ||
| return this.protocol + ':' + this.protocol + "://" + this.authority + this.identifier; |
There was a problem hiding this comment.
Two time this.protocol this was intended?
There was a problem hiding this comment.
yes - we have the following global id types: "doi", "hdl" "http" and "https"
in case of doi the string representation shall be
doi:blabla
in case of hdl the string represenation shall be
hdl:blablabla
so in case of http the string representation shall be
http:http://blablabalaba
ans in case of https the string respresentation shall be
htttps:https://blablabla
| final ArrayList<Node> identifiers = getNodes(document, "identifier"); | ||
| if(identifiers.isEmpty()) { | ||
| throw new EJBException("Missing dc:identifier xml element"); | ||
| } | ||
| for(final Node node : identifiers) { | ||
| version.addField(newField(otherId, null). | ||
| addChild(newField(otherIdValue, node.getTextContent()))); | ||
| } | ||
| dataset.setGlobalId(createGlobalId(identifiers)); |
There was a problem hiding this comment.
Maybe this code could be moved where title is parsed. Both are required fields so maybe could be in the same place
There was a problem hiding this comment.
moved it - tests broke. not worth it .
| if (harvestingClient == null || harvestingClient.getDataverse() == null) { | ||
| public Dataset doImportHarvestedDataset(final DataverseRequest request, | ||
| final HarvestingClient client, final String identifier, | ||
| final HarvestImporterType importType, final String xml) throws ImportException { |
There was a problem hiding this comment.
I would not change last parameter from metadata -> xml. In OAI-PMH specification content is called metadata
There was a problem hiding this comment.
maybe - but naming it xml states exactly what this variable holds - an xml text
|
|
||
| private GlobalId createGlobalId(final ArrayList<Node> identifierNodes) { | ||
|
|
||
| final List<String> identifiers = identifierNodes |
There was a problem hiding this comment.
If in this code we are only interested in first id, maybe just get first id and the do the logic without without for each type? final Optional<GlobalId> doi = identifiers .stream() .filter(GlobalId::isDOI) .findFirst()
There was a problem hiding this comment.
in this code fists we try to find UNBROKEN id of type HANDLE (there are broken ones in harvested xml documents)
it this fails we try finding UNBROKEN id of type HANDLE (again there are broken ones in receaved xml documents)
if this fails we try finding UNBROKEN is that is a URL (HTTP or HTTPS that is not DOI or HANDLE) (again there are bronke ones in received xml documents)
it it fails we fail
Id's can be placed in documents in any order but there are no more than one id of particular type in a single document.
Generally theseIDs are mess the code needs to handle somehow to maximise harvesting yeld.
2 participants
No description provided.