From 826e9bf0c1e87991d7b42313e89e4d0c51d50cc2 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Thu, 19 Dec 2024 15:33:00 -0500 Subject: [PATCH 01/17] add ROR to authorIdentifierScheme #11075 --- scripts/api/data/metadatablocks/citation.tsv | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/scripts/api/data/metadatablocks/citation.tsv b/scripts/api/data/metadatablocks/citation.tsv index abc09465603..a98f16c64c7 100644 --- a/scripts/api/data/metadatablocks/citation.tsv +++ b/scripts/api/data/metadatablocks/citation.tsv @@ -140,6 +140,7 @@ authorIdentifierScheme DAI 5 authorIdentifierScheme ResearcherID 6 authorIdentifierScheme ScopusID 7 + authorIdentifierScheme ROR 8 language 'Are'are alu 0 alu language 'Auhelawa kud 1 kud language A'ou aou 2 aou @@ -8061,4 +8062,4 @@ publicationRelationType IsSupplementTo RT3 3 publicationRelationType IsSupplementedBy RT4 4 publicationRelationType IsReferencedBy RT5 5 - publicationRelationType References RT6 6 \ No newline at end of file + publicationRelationType References RT6 6 From 29902a39a892fcbdbdba9a406c6b96a9c426aeec Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Thu, 19 Dec 2024 15:42:31 -0500 Subject: [PATCH 02/17] add release note #11075 --- doc/release-notes/11075-ror.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 doc/release-notes/11075-ror.md diff --git a/doc/release-notes/11075-ror.md b/doc/release-notes/11075-ror.md new file mode 100644 index 00000000000..711488ed8cb --- /dev/null +++ b/doc/release-notes/11075-ror.md @@ -0,0 +1,15 @@ +### ROR (Research Organization Registry) as Author Identifier Type + +ROR (Research Organization Registry) has been added as an Author Identifier Type (alongside ORCID, etc.). See #11075 and #11118. + +## Upgrade Instructions + +6\. Update metadata blocks + +These changes reflect incremental improvements made to the handling of core metadata fields. + +```shell +wget https://raw.githubusercontent.com/IQSS/dataverse/v6.4/scripts/api/data/metadatablocks/citation.tsv + +curl http://localhost:8080/api/admin/datasetfield/load -H "Content-type: text/tab-separated-values" -X POST --upload-file citation.tsv +``` From 0d5665c25d98ff790588c36310756243ad02dec6 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Fri, 20 Dec 2024 10:54:51 -0500 Subject: [PATCH 03/17] add ROR to citation.properties #11075 --- src/main/java/propertyFiles/citation.properties | 1 + 1 file changed, 1 insertion(+) diff --git a/src/main/java/propertyFiles/citation.properties b/src/main/java/propertyFiles/citation.properties index 5899523da67..3e5e75ec94a 100644 --- a/src/main/java/propertyFiles/citation.properties +++ b/src/main/java/propertyFiles/citation.properties @@ -305,6 +305,7 @@ controlledvocabulary.authorIdentifierScheme.gnd=GND controlledvocabulary.authorIdentifierScheme.dai=DAI controlledvocabulary.authorIdentifierScheme.researcherid=ResearcherID controlledvocabulary.authorIdentifierScheme.scopusid=ScopusID +controlledvocabulary.authorIdentifierScheme.ror=ROR controlledvocabulary.language.'are'are='Are'are controlledvocabulary.language.'auhelawa='Auhelawa controlledvocabulary.language.a'ou=A'ou From 029c2477b7dcbd70d535914bd2525f43d0207c51 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Fri, 20 Dec 2024 11:58:49 -0500 Subject: [PATCH 04/17] update display order of Author Identifier (ORCID then ROR) #11075 --- scripts/api/data/metadatablocks/citation.tsv | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/scripts/api/data/metadatablocks/citation.tsv b/scripts/api/data/metadatablocks/citation.tsv index a98f16c64c7..dea23aa9a73 100644 --- a/scripts/api/data/metadatablocks/citation.tsv +++ b/scripts/api/data/metadatablocks/citation.tsv @@ -133,14 +133,14 @@ contributorType Work Package Leader 15 contributorType Other 16 authorIdentifierScheme ORCID 0 - authorIdentifierScheme ISNI 1 - authorIdentifierScheme LCNA 2 - authorIdentifierScheme VIAF 3 - authorIdentifierScheme GND 4 - authorIdentifierScheme DAI 5 - authorIdentifierScheme ResearcherID 6 - authorIdentifierScheme ScopusID 7 - authorIdentifierScheme ROR 8 + authorIdentifierScheme ROR 1 + authorIdentifierScheme ISNI 2 + authorIdentifierScheme LCNA 3 + authorIdentifierScheme VIAF 4 + authorIdentifierScheme GND 5 + authorIdentifierScheme DAI 6 + authorIdentifierScheme ResearcherID 7 + authorIdentifierScheme ScopusID 8 language 'Are'are alu 0 alu language 'Auhelawa kud 1 kud language A'ou aou 2 aou From 335cf7288fa84090dd8bda3c44df05a0606b23b7 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Fri, 20 Dec 2024 11:59:54 -0500 Subject: [PATCH 05/17] bump to next version: 6.6 #11075 --- doc/release-notes/11075-ror.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/release-notes/11075-ror.md b/doc/release-notes/11075-ror.md index 711488ed8cb..0f2460b8df9 100644 --- a/doc/release-notes/11075-ror.md +++ b/doc/release-notes/11075-ror.md @@ -9,7 +9,7 @@ ROR (Research Organization Registry) has been added as an Author Identifier Type These changes reflect incremental improvements made to the handling of core metadata fields. ```shell -wget https://raw.githubusercontent.com/IQSS/dataverse/v6.4/scripts/api/data/metadatablocks/citation.tsv +wget https://raw.githubusercontent.com/IQSS/dataverse/v6.6/scripts/api/data/metadatablocks/citation.tsv curl http://localhost:8080/api/admin/datasetfield/load -H "Content-type: text/tab-separated-values" -X POST --upload-file citation.tsv ``` From ddb6bc96a39ab2f758d1dce1e1ea234bee2f44d2 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Fri, 20 Dec 2024 12:12:37 -0500 Subject: [PATCH 06/17] put ROR after ORCID to match displayOrder #11075 --- src/main/java/propertyFiles/citation.properties | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/main/java/propertyFiles/citation.properties b/src/main/java/propertyFiles/citation.properties index 3e5e75ec94a..9a1e6f280ec 100644 --- a/src/main/java/propertyFiles/citation.properties +++ b/src/main/java/propertyFiles/citation.properties @@ -298,6 +298,7 @@ controlledvocabulary.contributorType.supervisor=Supervisor controlledvocabulary.contributorType.work_package_leader=Work Package Leader controlledvocabulary.contributorType.other=Other controlledvocabulary.authorIdentifierScheme.orcid=ORCID +controlledvocabulary.authorIdentifierScheme.ror=ROR controlledvocabulary.authorIdentifierScheme.isni=ISNI controlledvocabulary.authorIdentifierScheme.lcna=LCNA controlledvocabulary.authorIdentifierScheme.viaf=VIAF @@ -305,7 +306,6 @@ controlledvocabulary.authorIdentifierScheme.gnd=GND controlledvocabulary.authorIdentifierScheme.dai=DAI controlledvocabulary.authorIdentifierScheme.researcherid=ResearcherID controlledvocabulary.authorIdentifierScheme.scopusid=ScopusID -controlledvocabulary.authorIdentifierScheme.ror=ROR controlledvocabulary.language.'are'are='Are'are controlledvocabulary.language.'auhelawa='Auhelawa controlledvocabulary.language.a'ou=A'ou From 6ae9c5571ca77bd60ccb6b744c3e4bcee448ce63 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Fri, 20 Dec 2024 12:16:30 -0500 Subject: [PATCH 07/17] explain ROR use case, link to https://ror.org #11075 --- doc/release-notes/11075-ror.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/release-notes/11075-ror.md b/doc/release-notes/11075-ror.md index 0f2460b8df9..014ced2dd73 100644 --- a/doc/release-notes/11075-ror.md +++ b/doc/release-notes/11075-ror.md @@ -1,6 +1,6 @@ ### ROR (Research Organization Registry) as Author Identifier Type -ROR (Research Organization Registry) has been added as an Author Identifier Type (alongside ORCID, etc.). See #11075 and #11118. +ROR (Research Organization Registry) has been added as an Author Identifier Type (alongside ORCID, etc.) for when the author is an organization rather than a person. See , #11075, and #11118. ## Upgrade Instructions From a9b221653432f990946490a33262b12dedad7ca8 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Tue, 7 Jan 2025 16:52:38 -0500 Subject: [PATCH 08/17] explicitly indicate that reordering is supported #11075 --- doc/sphinx-guides/source/admin/metadatacustomization.rst | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/doc/sphinx-guides/source/admin/metadatacustomization.rst b/doc/sphinx-guides/source/admin/metadatacustomization.rst index e5326efebef..fed5ab67d26 100644 --- a/doc/sphinx-guides/source/admin/metadatacustomization.rst +++ b/doc/sphinx-guides/source/admin/metadatacustomization.rst @@ -270,7 +270,11 @@ Each of the three main sections own sets of properties: | | “Value” field is used as the identifier. | | +--------------+--------------------------------------------+-----------------------------------------+ | displayOrder | Control the order in which the enumerated | Non-negative integer. | -| | values are displayed for selection. | | +| | values are displayed for selection. When | | +| | adding new values, you don't have to add | | +| | them at the end. You can renumber existing | | +| | values to update the order in which they | | +| | appear. | | +--------------+--------------------------------------------+-----------------------------------------+ FieldType definitions From ac1a881ff9d6a69f748235d3445e4b5a8bcfb52d Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Fri, 17 Jan 2025 15:15:19 -0500 Subject: [PATCH 09/17] add test asserting current ROR regex behavior #11075 --- .../iq/dataverse/DatasetFieldValueValidatorTest.java | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/src/test/java/edu/harvard/iq/dataverse/DatasetFieldValueValidatorTest.java b/src/test/java/edu/harvard/iq/dataverse/DatasetFieldValueValidatorTest.java index b753f534c6b..da9496e899e 100644 --- a/src/test/java/edu/harvard/iq/dataverse/DatasetFieldValueValidatorTest.java +++ b/src/test/java/edu/harvard/iq/dataverse/DatasetFieldValueValidatorTest.java @@ -157,6 +157,14 @@ public void testIsValidAuthorIdentifierGnd() { assertFalse(validator.isValidAuthorIdentifier("junk", pattern)); } + @Test + public void testIsValidAuthorIdentifierRor() { + DatasetFieldValueValidator validator = new DatasetFieldValueValidator(); + Pattern pattern = ExternalIdentifier.valueOf("ROR").getPattern(); + assertTrue(validator.isValidAuthorIdentifier("https://ror.org/03vek6s52", pattern)); + assertFalse(validator.isValidAuthorIdentifier("junk", pattern)); + } + final Validator validator = Validation.buildDefaultValidatorFactory().getValidator(); @ParameterizedTest From 36d45896d073035dfe796e12e190df50cb1f9897 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Tue, 21 Jan 2025 09:23:50 -0500 Subject: [PATCH 10/17] add DataCite export assertions for authors with RORs #11075 --- .../doi/datacite/XmlMetadataTemplateTest.java | 38 +++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/src/test/java/edu/harvard/iq/dataverse/pidproviders/doi/datacite/XmlMetadataTemplateTest.java b/src/test/java/edu/harvard/iq/dataverse/pidproviders/doi/datacite/XmlMetadataTemplateTest.java index 2bd6818821d..7faa9a0a6fa 100644 --- a/src/test/java/edu/harvard/iq/dataverse/pidproviders/doi/datacite/XmlMetadataTemplateTest.java +++ b/src/test/java/edu/harvard/iq/dataverse/pidproviders/doi/datacite/XmlMetadataTemplateTest.java @@ -34,6 +34,7 @@ import edu.harvard.iq.dataverse.util.testing.JvmSetting; import edu.harvard.iq.dataverse.util.testing.LocalJvmSettings; import edu.harvard.iq.dataverse.util.xml.XmlValidator; +import io.restassured.path.xml.XmlPath; import jakarta.json.JsonArray; import jakarta.json.JsonObject; import jakarta.json.JsonString; @@ -113,6 +114,8 @@ public void testDataCiteXMLCreation() throws IOException { df2.setDatasetFieldType(dft2); df2.setSingleValue("Harvard University"); alice.setAffiliation(df2); + alice.setIdType("ORCID"); + alice.setIdValue("0000-0002-1825-0097"); DatasetAuthor bob = new DatasetAuthor(); DatasetField df3 = new DatasetField(); df3.setDatasetFieldType(dft); @@ -122,9 +125,27 @@ public void testDataCiteXMLCreation() throws IOException { df4.setDatasetFieldType(dft2); df4.setSingleValue("QDR"); bob.setAffiliation(df4); + DatasetAuthor harvard = new DatasetAuthor(); + DatasetField df5 = new DatasetField(); + df5.setDatasetFieldType(dft); + df5.setSingleValue("Harvard University"); + harvard.setName(df5); + harvard.setIdType("ROR"); + harvard.setIdValue("03vek6s52"); + DatasetAuthor qdr = new DatasetAuthor(); + DatasetField df6 = new DatasetField(); + df6.setDatasetFieldType(dft); + df6.setSingleValue("Qualitative Data Repository"); + qdr.setName(df6); + qdr.setIdType("ROR"); + // This value is set improperly as a URL. It should be just + // the identifier (014trz974) as in the ORCID example above. + qdr.setIdValue("https://ror.org/014trz974"); List authors = new ArrayList<>(); authors.add(alice); authors.add(bob); + authors.add(harvard); + authors.add(qdr); doiMetadata.setAuthors(authors); doiMetadata.setPublisher("Dataverse"); XmlMetadataTemplate template = new XmlMetadataTemplate(doiMetadata); @@ -167,6 +188,23 @@ public void testDataCiteXMLCreation() throws IOException { System.out.println("Invalid schema: " + e.getMessage()); } + assertEquals("Alice", XmlPath.from(xml).getString("resource.creators.creator[0].creatorName")); + assertEquals("https://orcid.org/0000-0002-1825-0097", XmlPath.from(xml).getString("resource.creators.creator[0].nameIdentifier")); + assertEquals("ORCID", XmlPath.from(xml).getString("resource.creators.creator[0].nameIdentifier.@nameIdentifierScheme")); + assertEquals("https://orcid.org", XmlPath.from(xml).getString("resource.creators.creator[0].nameIdentifier.@schemeURI")); + assertEquals("Bob", XmlPath.from(xml).getString("resource.creators.creator[1].creatorName")); + assertEquals("Harvard University", XmlPath.from(xml).getString("resource.creators.creator[2].creatorName")); + // FIXME: nameIdentifierScheme, nameIdentifierScheme, and schemeURI should be populated + assertEquals("", XmlPath.from(xml).getString("resource.creators.creator[2].nameIdentifier")); + assertEquals(null, XmlPath.from(xml).getString("resource.creators.creator[2].nameIdentifier.@nameIdentifierScheme")); + assertEquals(null, XmlPath.from(xml).getString("resource.creators.creator[2].nameIdentifier.@schemeURI")); + assertEquals("Qualitative Data Repository", XmlPath.from(xml).getString("resource.creators.creator[3].creatorName")); + // FIXME: the URL below is not right + assertEquals("https://ror.org/https://ror.org/014trz974", XmlPath.from(xml).getString("resource.creators.creator[3].nameIdentifier")); + assertEquals("ROR", XmlPath.from(xml).getString("resource.creators.creator[3].nameIdentifier.@nameIdentifierScheme")); + assertEquals("https://ror.org", XmlPath.from(xml).getString("resource.creators.creator[3].nameIdentifier.@schemeURI")); + assertEquals("Dataverse", XmlPath.from(xml).getString("resource.publisher")); + } /** From 746d5834399e73beaa07847e85e33e50765a57dc Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Tue, 21 Jan 2025 14:55:19 -0500 Subject: [PATCH 11/17] add ROR tests for author affiliation and funder #11075 --- .../harvard/iq/dataverse/api/DatasetsIT.java | 172 ++++++++++++++++++ 1 file changed, 172 insertions(+) diff --git a/src/test/java/edu/harvard/iq/dataverse/api/DatasetsIT.java b/src/test/java/edu/harvard/iq/dataverse/api/DatasetsIT.java index e26064a24ef..7f2569982f3 100644 --- a/src/test/java/edu/harvard/iq/dataverse/api/DatasetsIT.java +++ b/src/test/java/edu/harvard/iq/dataverse/api/DatasetsIT.java @@ -4303,6 +4303,178 @@ public void testCitationDate() throws IOException { .statusCode(OK.getStatusCode()); } + @Test + public void testDataCiteExport() throws IOException { + + Response createUser = UtilIT.createRandomUser(); + createUser.then().assertThat().statusCode(OK.getStatusCode()); + String username = UtilIT.getUsernameFromResponse(createUser); + String apiToken = UtilIT.getApiTokenFromResponse(createUser); + + Response createDataverse = UtilIT.createRandomDataverse(apiToken); + createDataverse.then().assertThat().statusCode(CREATED.getStatusCode()); + String dataverseAlias = UtilIT.getAliasFromResponse(createDataverse); + Integer dataverseId = UtilIT.getDataverseIdFromResponse(createDataverse); + + JsonObjectBuilder datasetJson = Json.createObjectBuilder() + .add("datasetVersion", Json.createObjectBuilder() + .add("license", Json.createObjectBuilder() + .add("name", "CC0 1.0") + .add("uri", "http://creativecommons.org/publicdomain/zero/1.0") + ) + .add("metadataBlocks", Json.createObjectBuilder() + .add("citation", Json.createObjectBuilder() + .add("fields", Json.createArrayBuilder() + .add(Json.createObjectBuilder() + .add("typeName", "title") + .add("value", "Test dataset") + .add("typeClass", "primitive") + .add("multiple", false) + ) + .add(Json.createObjectBuilder() + .add("value", Json.createArrayBuilder() + .add(Json.createObjectBuilder() + .add("authorName", + Json.createObjectBuilder() + .add("value", "Simpson, Homer") + .add("typeClass", "primitive") + .add("multiple", false) + .add("typeName", "authorName") + ) + .add("authorAffiliation", + Json.createObjectBuilder() + .add("value", "https://ror.org/03vek6s52") + .add("typeClass", "primitive") + .add("multiple", false) + .add("typeName", "authorAffiliation") + ) + ) + ) + .add("typeClass", "compound") + .add("multiple", true) + .add("typeName", "author") + ) + .add(Json.createObjectBuilder() + .add("value", Json.createArrayBuilder() + .add(Json.createObjectBuilder() + .add("datasetContactEmail", + Json.createObjectBuilder() + .add("value", "hsimpson@mailinator.com") + .add("typeClass", "primitive") + .add("multiple", false) + .add("typeName", "datasetContactEmail")) + ) + ) + .add("typeClass", "compound") + .add("multiple", true) + .add("typeName", "datasetContact") + ) + .add(Json.createObjectBuilder() + .add("value", Json.createArrayBuilder() + .add(Json.createObjectBuilder() + .add("dsDescriptionValue", + Json.createObjectBuilder() + .add("value", "Just a test dataset.") + .add("typeClass", "primitive") + .add("multiple", false) + .add("typeName", "dsDescriptionValue")) + ) + ) + .add("typeClass", "compound") + .add("multiple", true) + .add("typeName", "dsDescription") + ) + .add(Json.createObjectBuilder() + .add("value", Json.createArrayBuilder() + .add("Other") + ) + .add("typeClass", "controlledVocabulary") + .add("multiple", true) + .add("typeName", "subject") + ) + .add(Json.createObjectBuilder() + .add("value", Json.createArrayBuilder() + .add(Json.createObjectBuilder() + .add("authorName", + Json.createObjectBuilder() + .add("value", "https://ror.org/05h1kgg64") // NIH + .add("typeClass", "primitive") + .add("multiple", false) + .add("typeName", "grantNumberAgency") + ) + .add("authorAffiliation", + Json.createObjectBuilder() + .add("value", "12345") + .add("typeClass", "primitive") + .add("multiple", false) + .add("typeName", "grantNumberValue") + ) + ) + ) + .add("typeClass", "compound") + .add("multiple", true) + .add("typeName", "grantNumber") + ) + ) + ) + )); + + Response createDatasetResponse = UtilIT.createDataset(dataverseAlias, datasetJson, apiToken); + createDatasetResponse.prettyPrint(); + Integer datasetId = UtilIT.getDatasetIdFromResponse(createDatasetResponse); + String datasetPid = JsonPath.from(createDatasetResponse.getBody().asString()).getString("data.persistentId"); + + Response publishDataverse = UtilIT.publishDataverseViaNativeApi(dataverseAlias, apiToken); + publishDataverse.prettyPrint(); + publishDataverse.then().assertThat().statusCode(OK.getStatusCode()); + Response publishDataset = UtilIT.publishDatasetViaNativeApi(datasetId, "major", apiToken); + publishDataset.prettyPrint(); + publishDataset.then().assertThat().statusCode(OK.getStatusCode()); + + Response exportDatasetAsDataCite = UtilIT.exportDataset(datasetPid, "Datacite", apiToken, true); + exportDatasetAsDataCite.prettyPrint(); + exportDatasetAsDataCite.then().assertThat() + .body("resource.creators.creator[0].creatorName", equalTo("Simpson, Homer")) + // see below for additional affiliation assertions, which can vary + .body("resource.creators.creator[0].affiliation.@schemeURI", equalTo("https://ror.org")) + .body("resource.creators.creator[0].affiliation.@affiliationIdentifierScheme", equalTo("ROR")) + // see below for additional fundingReference assertions, which can vary + .body("resource.fundingReferences.fundingReference[0].awardNumber", equalTo("12345")) + .statusCode(OK.getStatusCode()); + + // Out of the box :CVocConf is not set. If you set it with + // https://github.com/gdcc/dataverse-external-vocab-support/blob/de011d239254ff7d651212c565f8604808dcd7e9/examples/config/grantNumberAgencyRor.json + // you can expect different results. + boolean authorsOrcidAndRorEnabled = false; + if (authorsOrcidAndRorEnabled) { + exportDatasetAsDataCite.then().assertThat() + .body("resource.creators.creator[0].affiliation", equalTo("Harvard University")) + // Once https://github.com/IQSS/dataverse/pull/11175 is merged the equalTo bellow + // should be "https://ror.org/03vek6s52" instead of "Harvard University". + .body("resource.creators.creator[0].affiliation.@affiliationIdentifier", equalTo("Harvard University")); + } else { + exportDatasetAsDataCite.then().assertThat() + .body("resource.creators.creator[0].affiliation", equalTo("https://ror.org/03vek6s52")) + .body("resource.creators.creator[0].affiliation.@affiliationIdentifier", equalTo("https://ror.org/03vek6s52")); + } + + // Out of the box :CVocConf is not set. If you set it with + // https://github.com/gdcc/dataverse-external-vocab-support/blob/de011d239254ff7d651212c565f8604808dcd7e9/examples/config/grantNumberAgencyRor.json + // you can expect different results. + boolean grantNumberAgencyRorEnabled = false; + if (grantNumberAgencyRorEnabled) { + exportDatasetAsDataCite.then().assertThat() + .body("resource.fundingReferences.fundingReference[0].funderName", equalTo("National Institute of Health")) + .body("resource.fundingReferences.fundingReference[0].funderIdentifier.@funderIdentifierType", equalTo("ROR")) + .body("resource.fundingReferences.fundingReference[0].funderIdentifier.@schemeURI", equalTo("https://ror.org")) + .body("resource.fundingReferences.fundingReference[0].funderIdentifier", equalTo("https://ror.org/05h1kgg64")); + } else { + exportDatasetAsDataCite.then().assertThat() + .body("resource.fundingReferences.fundingReference[0].funderName", equalTo("https://ror.org/05h1kgg64")) + .body("resource.fundingReferences.fundingReference[0].awardNumber", equalTo("12345")); + } + } + @Test public void getVersionFiles() throws IOException, InterruptedException { Response createUser = UtilIT.createRandomUser(); From 8653513715e4c94f86a9cd3c9bfeabcd1d4ba465 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Tue, 21 Jan 2025 15:38:50 -0500 Subject: [PATCH 12/17] note that it take several seconds to load the citation block #11075 --- doc/release-notes/11075-ror.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/release-notes/11075-ror.md b/doc/release-notes/11075-ror.md index 014ced2dd73..dc8b68512d3 100644 --- a/doc/release-notes/11075-ror.md +++ b/doc/release-notes/11075-ror.md @@ -6,7 +6,7 @@ ROR (Research Organization Registry) has been added as an Author Identifier Type 6\. Update metadata blocks -These changes reflect incremental improvements made to the handling of core metadata fields. +These changes reflect incremental improvements made to the handling of core metadata fields. Expect the loading of the citation block to take several seconds because of its size (especially due to the number of languages). ```shell wget https://raw.githubusercontent.com/IQSS/dataverse/v6.6/scripts/api/data/metadatablocks/citation.tsv From 1fe95d97cfc087ca7f08a68383e3d6adcaa96fa8 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Tue, 21 Jan 2025 16:21:16 -0500 Subject: [PATCH 13/17] put ROR regex in line with existing pattern #11075 --- .../harvard/iq/dataverse/ExternalIdentifier.java | 6 ++++-- .../pidproviders/doi/XmlMetadataTemplate.java | 4 ++-- .../dataverse/DatasetFieldValueValidatorTest.java | 2 +- .../doi/datacite/XmlMetadataTemplateTest.java | 15 +++++++-------- 4 files changed, 14 insertions(+), 13 deletions(-) diff --git a/src/main/java/edu/harvard/iq/dataverse/ExternalIdentifier.java b/src/main/java/edu/harvard/iq/dataverse/ExternalIdentifier.java index 8c4fb6b1325..cf1dcfb3c15 100644 --- a/src/main/java/edu/harvard/iq/dataverse/ExternalIdentifier.java +++ b/src/main/java/edu/harvard/iq/dataverse/ExternalIdentifier.java @@ -13,8 +13,10 @@ public enum ExternalIdentifier { // note: DAI is missing from this list, because it doesn't have resolvable URL ResearcherID("ResearcherID", "https://publons.com/researcher/%s/", "^[A-Z\\d][A-Z\\d-]+[A-Z\\d]$"), ScopusID("ScopusID", "https://www.scopus.com/authid/detail.uri?authorId=%s", "^\\d*$"), - //Requiring ROR to be URL form as we use it where there is no id type field and matching any 9 digit number starting with 0 seems a bit aggressive - ROR("ROR", "https://ror.org/%s", "^(https:\\/\\/ror.org\\/)0[a-hj-km-np-tv-z|0-9]{6}[0-9]{2}$"); + // ROR regex from https://ror.readme.io/docs/identifier + ROR("ROR", "https://ror.org/%s", "^0[a-hj-km-np-tv-z|0-9]{6}[0-9]{2}$"), + // In some contexts, we check for the full ROR URL. + ROR_FULL_URL("ROR", "https://ror.org/%s", "^(https:\\/\\/ror.org\\/)0[a-hj-km-np-tv-z|0-9]{6}[0-9]{2}$"); private String name; private String template; diff --git a/src/main/java/edu/harvard/iq/dataverse/pidproviders/doi/XmlMetadataTemplate.java b/src/main/java/edu/harvard/iq/dataverse/pidproviders/doi/XmlMetadataTemplate.java index 8199b7d9c9f..eb1294a3db7 100644 --- a/src/main/java/edu/harvard/iq/dataverse/pidproviders/doi/XmlMetadataTemplate.java +++ b/src/main/java/edu/harvard/iq/dataverse/pidproviders/doi/XmlMetadataTemplate.java @@ -626,7 +626,7 @@ private void writeEntityElements(XMLStreamWriter xmlw, String elementName, Strin attributeMap.clear(); boolean isROR=false; String orgName = affiliation; - ExternalIdentifier externalIdentifier = ExternalIdentifier.ROR; + ExternalIdentifier externalIdentifier = ExternalIdentifier.ROR_FULL_URL; if (externalIdentifier.isValidIdentifier(orgName)) { isROR = true; JsonObject jo = getExternalVocabularyValue(orgName); @@ -1528,7 +1528,7 @@ private void writeFundingReferences(XMLStreamWriter xmlw, DvObject dvObject) thr fundingReferenceWritten = XmlWriterUtil.writeOpenTagIfNeeded(xmlw, "fundingReferences", fundingReferenceWritten); boolean isROR=false; String funderIdentifier = null; - ExternalIdentifier externalIdentifier = ExternalIdentifier.ROR; + ExternalIdentifier externalIdentifier = ExternalIdentifier.ROR_FULL_URL; if (externalIdentifier.isValidIdentifier(funder)) { isROR = true; JsonObject jo = getExternalVocabularyValue(funder); diff --git a/src/test/java/edu/harvard/iq/dataverse/DatasetFieldValueValidatorTest.java b/src/test/java/edu/harvard/iq/dataverse/DatasetFieldValueValidatorTest.java index da9496e899e..dc04bf7fe01 100644 --- a/src/test/java/edu/harvard/iq/dataverse/DatasetFieldValueValidatorTest.java +++ b/src/test/java/edu/harvard/iq/dataverse/DatasetFieldValueValidatorTest.java @@ -161,7 +161,7 @@ public void testIsValidAuthorIdentifierGnd() { public void testIsValidAuthorIdentifierRor() { DatasetFieldValueValidator validator = new DatasetFieldValueValidator(); Pattern pattern = ExternalIdentifier.valueOf("ROR").getPattern(); - assertTrue(validator.isValidAuthorIdentifier("https://ror.org/03vek6s52", pattern)); + assertTrue(validator.isValidAuthorIdentifier("03vek6s52", pattern)); assertFalse(validator.isValidAuthorIdentifier("junk", pattern)); } diff --git a/src/test/java/edu/harvard/iq/dataverse/pidproviders/doi/datacite/XmlMetadataTemplateTest.java b/src/test/java/edu/harvard/iq/dataverse/pidproviders/doi/datacite/XmlMetadataTemplateTest.java index 7faa9a0a6fa..2e0d4a12fe0 100644 --- a/src/test/java/edu/harvard/iq/dataverse/pidproviders/doi/datacite/XmlMetadataTemplateTest.java +++ b/src/test/java/edu/harvard/iq/dataverse/pidproviders/doi/datacite/XmlMetadataTemplateTest.java @@ -194,15 +194,14 @@ public void testDataCiteXMLCreation() throws IOException { assertEquals("https://orcid.org", XmlPath.from(xml).getString("resource.creators.creator[0].nameIdentifier.@schemeURI")); assertEquals("Bob", XmlPath.from(xml).getString("resource.creators.creator[1].creatorName")); assertEquals("Harvard University", XmlPath.from(xml).getString("resource.creators.creator[2].creatorName")); - // FIXME: nameIdentifierScheme, nameIdentifierScheme, and schemeURI should be populated - assertEquals("", XmlPath.from(xml).getString("resource.creators.creator[2].nameIdentifier")); - assertEquals(null, XmlPath.from(xml).getString("resource.creators.creator[2].nameIdentifier.@nameIdentifierScheme")); - assertEquals(null, XmlPath.from(xml).getString("resource.creators.creator[2].nameIdentifier.@schemeURI")); + assertEquals("https://ror.org/03vek6s52", XmlPath.from(xml).getString("resource.creators.creator[2].nameIdentifier")); + assertEquals("ROR", XmlPath.from(xml).getString("resource.creators.creator[2].nameIdentifier.@nameIdentifierScheme")); + assertEquals("https://ror.org", XmlPath.from(xml).getString("resource.creators.creator[2].nameIdentifier.@schemeURI")); assertEquals("Qualitative Data Repository", XmlPath.from(xml).getString("resource.creators.creator[3].creatorName")); - // FIXME: the URL below is not right - assertEquals("https://ror.org/https://ror.org/014trz974", XmlPath.from(xml).getString("resource.creators.creator[3].nameIdentifier")); - assertEquals("ROR", XmlPath.from(xml).getString("resource.creators.creator[3].nameIdentifier.@nameIdentifierScheme")); - assertEquals("https://ror.org", XmlPath.from(xml).getString("resource.creators.creator[3].nameIdentifier.@schemeURI")); + // The nameIdentifier fields below are not populated because the full ROR URL was entered. + assertEquals("", XmlPath.from(xml).getString("resource.creators.creator[3].nameIdentifier")); + assertEquals(null, XmlPath.from(xml).getString("resource.creators.creator[3].nameIdentifier.@nameIdentifierScheme")); + assertEquals(null, XmlPath.from(xml).getString("resource.creators.creator[3].nameIdentifier.@schemeURI")); assertEquals("Dataverse", XmlPath.from(xml).getString("resource.publisher")); } From baa84f578ecea52eb5f66aab57533f3749998030 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Wed, 22 Jan 2025 09:33:13 -0500 Subject: [PATCH 14/17] add unit test for ROR_FULL_URL #11075 --- .../iq/dataverse/DatasetFieldValueValidatorTest.java | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/src/test/java/edu/harvard/iq/dataverse/DatasetFieldValueValidatorTest.java b/src/test/java/edu/harvard/iq/dataverse/DatasetFieldValueValidatorTest.java index dc04bf7fe01..7320cf7acfe 100644 --- a/src/test/java/edu/harvard/iq/dataverse/DatasetFieldValueValidatorTest.java +++ b/src/test/java/edu/harvard/iq/dataverse/DatasetFieldValueValidatorTest.java @@ -165,6 +165,14 @@ public void testIsValidAuthorIdentifierRor() { assertFalse(validator.isValidAuthorIdentifier("junk", pattern)); } + @Test + public void testIsValidAuthorIdentifierRorFull() { + DatasetFieldValueValidator validator = new DatasetFieldValueValidator(); + Pattern pattern = ExternalIdentifier.valueOf("ROR_FULL_URL").getPattern(); + assertTrue(validator.isValidAuthorIdentifier("https://ror.org/03vek6s52", pattern)); + assertFalse(validator.isValidAuthorIdentifier("junk", pattern)); + } + final Validator validator = Validation.buildDefaultValidatorFactory().getValidator(); @ParameterizedTest From 308ea2ac3577cd0cdf4c3c0bd90d66aa6cf4885f Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Wed, 22 Jan 2025 09:36:26 -0500 Subject: [PATCH 15/17] use correct NIH #11075 --- .../java/edu/harvard/iq/dataverse/api/DatasetsIT.java | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/test/java/edu/harvard/iq/dataverse/api/DatasetsIT.java b/src/test/java/edu/harvard/iq/dataverse/api/DatasetsIT.java index 7f2569982f3..c1273fba9fc 100644 --- a/src/test/java/edu/harvard/iq/dataverse/api/DatasetsIT.java +++ b/src/test/java/edu/harvard/iq/dataverse/api/DatasetsIT.java @@ -4397,7 +4397,7 @@ public void testDataCiteExport() throws IOException { .add(Json.createObjectBuilder() .add("authorName", Json.createObjectBuilder() - .add("value", "https://ror.org/05h1kgg64") // NIH + .add("value", "https://ror.org/01cwqze88") // NIH .add("typeClass", "primitive") .add("multiple", false) .add("typeName", "grantNumberAgency") @@ -4464,13 +4464,13 @@ public void testDataCiteExport() throws IOException { boolean grantNumberAgencyRorEnabled = false; if (grantNumberAgencyRorEnabled) { exportDatasetAsDataCite.then().assertThat() - .body("resource.fundingReferences.fundingReference[0].funderName", equalTo("National Institute of Health")) + .body("resource.fundingReferences.fundingReference[0].funderName", equalTo("National Institutes of Health")) .body("resource.fundingReferences.fundingReference[0].funderIdentifier.@funderIdentifierType", equalTo("ROR")) .body("resource.fundingReferences.fundingReference[0].funderIdentifier.@schemeURI", equalTo("https://ror.org")) - .body("resource.fundingReferences.fundingReference[0].funderIdentifier", equalTo("https://ror.org/05h1kgg64")); + .body("resource.fundingReferences.fundingReference[0].funderIdentifier", equalTo("https://ror.org/01cwqze88")); } else { exportDatasetAsDataCite.then().assertThat() - .body("resource.fundingReferences.fundingReference[0].funderName", equalTo("https://ror.org/05h1kgg64")) + .body("resource.fundingReferences.fundingReference[0].funderName", equalTo("https://ror.org/01cwqze88")) .body("resource.fundingReferences.fundingReference[0].awardNumber", equalTo("12345")); } } From b9d0146b07c9c88e0b5b8759fe2cff10df19029d Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Wed, 22 Jan 2025 10:28:35 -0500 Subject: [PATCH 16/17] add to user guide how to enter an author identifier #11075 --- doc/sphinx-guides/source/user/dataset-management.rst | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/doc/sphinx-guides/source/user/dataset-management.rst b/doc/sphinx-guides/source/user/dataset-management.rst index b3a14554b40..6b2f48fb8d9 100755 --- a/doc/sphinx-guides/source/user/dataset-management.rst +++ b/doc/sphinx-guides/source/user/dataset-management.rst @@ -50,8 +50,10 @@ Adding a New Dataset #. Navigate to the Dataverse collection in which you want to add a dataset. #. Click on the "Add Data" button and select "New Dataset" in the dropdown menu. **Note:** If you are on the root Dataverse collection, your My Data page or click the "Add Data" link in the navbar, the dataset you create will be hosted in the root Dataverse collection. You can change this by selecting another Dataverse collection you have proper permissions to create datasets in, from the Host Dataverse collection dropdown in the create dataset form. This option to choose will not be available after you create the dataset. -#. To quickly get started, enter at minimum all the required fields with an asterisk (e.g., the Dataset Title, Author Name, - Description Text, Point of Contact Email, and Subject) to get a Data Citation with a DOI. +#. To quickly get started, enter at minimum all the required fields with an asterisk (e.g., the Dataset Title, Author Name, Description Text, Point of Contact Email, and Subject) to get a Data Citation with a DOI. + + #. When entering author identifiers, select the type from the dropdown (e.g. "ORCID") and under "Identifier" enter just the unique identifier (e.g. "0000-0002-1825-0097") rather than the full URL (e.g. "https://orcid.org/0000-0002-1825-0097"). + #. Scroll down to the "Files" section and click on "Select Files to Add" to add all the relevant files to your Dataset. You can also upload your files directly from your Dropbox. **Tip:** You can drag and drop or select multiple files at a time from your desktop directly into the upload widget. Your files will appear below the "Select Files to Add" button where you can add a From d94a7a243a4657a5486b8d052d0aa37fdff4631b Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Wed, 22 Jan 2025 10:33:15 -0500 Subject: [PATCH 17/17] explain how to add author identifier in release note #11075 --- doc/release-notes/11075-ror.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/release-notes/11075-ror.md b/doc/release-notes/11075-ror.md index dc8b68512d3..de32eebccdf 100644 --- a/doc/release-notes/11075-ror.md +++ b/doc/release-notes/11075-ror.md @@ -1,6 +1,6 @@ ### ROR (Research Organization Registry) as Author Identifier Type -ROR (Research Organization Registry) has been added as an Author Identifier Type (alongside ORCID, etc.) for when the author is an organization rather than a person. See , #11075, and #11118. +ROR (Research Organization Registry) has been added as an Author Identifier Type (alongside ORCID, etc.) for when the author is an organization rather than a person. As with all author identifiers, be sure to select the proper identifier type (e.g. "ROR") and enter just the unique identifier (e.g. "03vek6s52") rather than the full URL (e.g. "https://ror.org/03vek6s52"). Like ORCID, ROR will appear in the "Datacite" metadata export format. See also the [ROR](https://ror.org) website, a new [note](https://dataverse-guide--11118.org.readthedocs.build/en/11118/user/dataset-management.html#adding-a-new-dataset) in a guides about entering author identifiers, #11075, and #11118. ## Upgrade Instructions