Merge branch 'develop' into 10478-version-base-img

conf/solr/schema.xml

@@ -142,6 +142,7 @@
 
     <field name="dvName" type="text_en" stored="true" indexed="true" multiValued="false"/>
     <field name="dvAlias" type="text_en" stored="true" indexed="true" multiValued="false"/>
+    <field name="dvParentAlias" type="text_en" stored="true" indexed="true" multiValued="false"/>
     <field name="dvAffiliation" type="text_en" stored="true" indexed="true" multiValued="false"/>
     <field name="dvDescription" type="text_en" stored="true" indexed="true" multiValued="false"/>
 
@@ -205,6 +206,7 @@
     <field name="entityId" type="plong" stored="true" indexed="true" multiValued="false"/>
 
     <field name="datasetVersionId" type="plong" stored="true" indexed="true" multiValued="false"/>
+    <field name="datasetType" type="string" stored="true" indexed="true" multiValued="false"/>
 
     <!-- Added for Dataverse 4.0 alpha 1 to sort by name  -->
     <!-- https://redmine.hmdc.harvard.edu/issues/3482 -->
@@ -426,6 +428,7 @@
     <copyField source="dvAlias" dest="_text_" maxChars="3000"/>
     <copyField source="dvAffiliation" dest="_text_" maxChars="3000"/>
     <copyField source="dsPersistentId" dest="_text_" maxChars="3000"/>
+    <copyField source="datasetType" dest="_text_" maxChars="3000"/>
     <!-- copyField commands copy one field to another at the time a document
         is added to the index.  It's used either to index the same field differently,
         or to add multiple fields to the same field for easier/faster searching.  -->

doc/release-notes/10433-add-thumbnail-for-featured-dataverses.md

@@ -0,0 +1,5 @@
+Add the ability to configure a thumbnail logo that is displayed for a collection when the collection is configured as a featured collection. If present, this thumbnail logo is shown. Otherwise, the collection logo is shown. Configuration is done under the "Theme" for a collection.
+
+The HTML preview of the documentation can be found [here](https://dataverse-guide--10433.org.readthedocs.build/en/10433/user/dataverse-management.html#theme).
+
+For more information, see [#10291](https://github.com/IQSS/dataverse/issues/10291).

doc/release-notes/10517-datasetType.md

@@ -0,0 +1,10 @@
+### Initial Support for Dataset Types
+
+Out of the box, all datasets have the type "dataset" but superusers can add additional types. At this time the type can only be set at creation time via API. The types "dataset", "software", and "workflow" will be sent to DataCite when the dataset is published.
+
+For details see <https://dataverse-guide--10694.org.readthedocs.build/en/10694/user/dataset-management.html#dataset-types> and #10517. Please note that this feature is highly experimental and is expected to evolve.
+
+Upgrade instructions
+--------------------
+
+Update your Solr schema.xml file to pick up the "datasetType" additions and do a full reindex.

doc/release-notes/10583-dataset-unlink-functionality-same-permission-as-link.md

@@ -0,0 +1,2 @@
+New "Unlink Dataset" button has been added to the Dataset Page to allow a user to unlink a dataset from a collection that was previously linked with the "Link Dataset" button. The user must possess the same permissions needed to unlink the Dataset as they would to link the Dataset.
+The [existing API](https://guides.dataverse.org/en/6.3/admin/dataverses-datasets.html#unlink-a-dataset) for unlinking datasets has been updated to no longer require superuser access. The "Publish Dataset" permission is now enough.

doc/release-notes/10744-ro-crate-docs.md

@@ -0,0 +1,3 @@
+## RO-Crate Support (Metadata Export)
+
+Dataverse now supports [RO-Crate](https://www.researchobject.org/ro-crate/) in the sense that dataset metadata can be exported in that format. This functionality is not available out of the box but you can enable one or more RO-Crate exporters from the [list of external exporters](https://preview.guides.gdcc.io/en/develop/installation/advanced.html#inventory-of-external-exporters). See also #10744.

doc/release-notes/10797-update-current-version-bug-fix.md

@@ -0,0 +1,11 @@
+A significant bug in the superuser-only "Update-Current-Version" publication was found and fixed in this release. If the Update-Current-Version option was used when changes were made to the dataset Terms (rather than to dataset metadata), or if the PID provider service was down/returned an error, the update would fail and render the dataset unusable and require restoration from a backup. The fix in this release allows the update to succeed in both of these cases and redesigns the functionality such that any unknown issues should not make the dataset unusable (i.e. the error would be reported and the dataset would remain in its current state with the last-published version as it was and changes still in the draft version.)
+
+Users of earlier Dataverse releases are encouraged to alert their superusers to this issue. Those who wish to disable this functionality have two options:
+* Change the dataset.updateRelease entry in the Bundle.properties file (or local language version) to "Do Not Use" or similar (doesn't disable but alerts superusers to the issue), or
+* Edit the dataset.xhtml file to remove the lines
+
+<c:if test="#{dataverseSession.user.isSuperuser()}">
+  <f:selectItem rendered="#" itemLabel="#{bundle['dataset.updateRelease']}" itemValue="3" />
+</c:if>
+
+, delete the contents of the generated and osgi-cache directories in the Dataverse Payara domain, and restart the Payara server.

doc/release-notes/10800-add-dataverse-request-json-fix.md

@@ -0,0 +1 @@
+Fixed the "addDataverse" API endpoint (/dataverses/{id} POST) expected request JSON structure to parse facetIds as described in the docs.

doc/release-notes/10810-search-api-payload-extensions.md

@@ -0,0 +1,52 @@
+Search API (/api/search) response will now include new fields for the different entities.
+
+For Dataverse:
+
+- "affiliation" 
+- "parentDataverseName"
+- "parentDataverseIdentifier"
+- "image_url" (optional)
+
+```javascript
+"items": [
+    {
+        "name": "Darwin's Finches",
+        ...
+        "affiliation": "Dataverse.org",
+        "parentDataverseName": "Root",
+        "parentDataverseIdentifier": "root",
+        "image_url":"data:image/png;base64,iVBORw0..."
+(etc, etc)
+```
+
+For DataFile:
+
+- "releaseOrCreateDate"
+- "image_url" (optional)
+
+```javascript
+"items": [
+    {
+        "name": "test.txt",
+        ...
+        "releaseOrCreateDate": "2016-05-10T12:53:39Z",
+        "image_url":"data:image/png;base64,iVBORw0..."
+(etc, etc)
+```
+
+For Dataset:
+
+- "image_url" (optional)
+
+```javascript
+"items": [
+    {
+        ...
+        "image_url": "http://localhost:8080/api/datasets/2/logo"
+        ...
+(etc, etc)
+```
+
+The image_url field was already part of the SolrSearchResult JSON (and incorrectly appeared in Search API documentation), but it wasn’t returned by the API because it was appended only after the Solr query was executed in the SearchIncludeFragment of JSF. Now, the field is set in SearchServiceBean, ensuring it is always returned by the API when an image is available.
+
+The schema.xml file for Solr has been updated to include a new field called dvParentAlias for supporting the new response field "parentDataverseIdentifier". So for the next Dataverse released version, a Solr reindex will be necessary to apply the new schema.xml version.

doc/release-notes/10819-publish-thumbnail-bug.md

@@ -0,0 +1,6 @@
+The initial release of the Dataverse v6.3 introduced a bug where publishing would break the dataset thumbnail, which in turn broke the rendering of the parent Collection ("dataverse") page. This problem was fixed in the PR 10820.
+
+This bug fix will prevent this from happening in the future, but does not fix any existing broken links. To restore any broken thumbnails caused by this bug, you can call the http://localhost:8080/api/admin/clearThumbnailFailureFlag API, which will attempt to clear the flag on all files (regardless of whether caused by this bug or some other problem with the file) or the http://localhost:8080/api/admin/clearThumbnailFailureFlag/id to clear the flag for individual files. Calling the former, batch API is recommended. 
+
+Additionally, the same PR made it possible to turn off the feature that automatically selects of one of the image datafiles to serve as the thumbnail of the parent dataset. An admin can turn it off by raising the feature flag `<jvm-options>-Ddataverse.feature.disable-dataset-thumbnail-autoselect=true</jvm-options>`. When the feature is disabled, a user can still manually pick a thumbnail image, or upload a dedicated thumbnail image.
+

doc/release-notes/8129-harvesting.md

@@ -0,0 +1,18 @@
+### Remap oai_dc export and harvesting format fields: dc:type and dc:date
+
+The `oai_dc` export and harvesting format has had the following fields remapped:
+
+- dc:type was mapped to the field "Kind of Data". Now it is hard-coded to the word "Dataset".
+- dc:date was mapped to the field "Production Date" when available and otherwise to "Publication Date". Now it is mapped the field "Publication Date" or the field used for the citation date, if set (see [Set Citation Date Field Type for a Dataset](https://guides.dataverse.org/en/6.3/api/native-api.html#set-citation-date-field-type-for-a-dataset)).
+
+In order for these changes to be reflected in existing datasets, a [reexport all](https://guides.dataverse.org/en/6.3/admin/metadataexport.html#batch-exports-through-the-api) should be run.
+
+For more information, please see #8129 and #10737.
+
+### Backward incompatible changes 
+
+See the "Remap oai_dc export" section above.
+
+### Upgrade instructions
+
+In order for changes to the `oai_dc` metadata export format to be reflected in existing datasets, a [reexport all](https://guides.dataverse.org/en/6.3/admin/metadataexport.html#batch-exports-through-the-api) should be run.

doc/release-notes/8578-support-for-iso-639-3-languages.md

@@ -0,0 +1,17 @@
+The Controlled Vocabulary Values list for the metadata field Language in the Citation block has now been extended to include roughly 7920 ISO 639-3 values.
+- Some of the language entries in the pre-v.6.4 list correspond to "macro languages" in ISO-639-3 and admins/users may wish to update to use the corresponding individual language entries from ISO-639-3. As these cases are expected to be rare (they do not involve major world languages), finding them is not covered in the release notes. Anyone who desires help in this area is encouraged to reach out to the Dataverse community via any of the standard communication channels.
+- ISO 639-3 codes were downloaded from:
+```
+https://iso639-3.sil.org/code_tables/download_tables#Complete%20Code%20Tables:~:text=iso%2D639%2D3_Code_Tables_20240415.zip
+```
+- The file used for merging with the existing citation.tsv was iso-639-3.tab
+
+To be added to the 6.4 release instructions:
+
+### Additional Upgrade Steps
+6\. Update the Citation metadata block:
+
+```
+- `wget https://github.com/IQSS/dataverse/releases/download/v6.4/citation.tsv`
+- `curl http://localhost:8080/api/admin/datasetfield/load -X POST --data-binary @citation.tsv -H "Content-type: text/tab-separated-values"`
+```

doc/release-notes/9317-delete-saved-search.md

@@ -0,0 +1,4 @@
+### Saved search deletion
+
+Saved searches can now be removed using API `/api/admin/savedsearches/$id`. See PR #10198.
+This is reflected in the [Saved Search Native API section](https://dataverse-guide--10198.org.readthedocs.build/en/10198/api/native-api.html#saved-search) of the Guide.

doc/release-notes/permalink-base-urls.md

@@ -0,0 +1,10 @@
+The configuration setting `dataverse.pid.*.permalink.base-url`, which is used for PermaLinks, has been updated to
+support greater flexibility. Previously, the string "/citation?persistentId=" was automatically appended to the
+configured base URL. With this update, the base URL will now be used exactly as configured, without any automatic
+additions.
+
+**Upgrade instructions:**
+
+- If you currently use a PermaLink provider with a configured `base-url`: You must manually append
+   "/citation?persistentId=" to the existing base URL to maintain functionality.
+- If you use a PermaLink provider without a configured `base-url`: No changes are required.

doc/sphinx-guides/source/_static/api/dataset-create-software.json

@@ -0,0 +1,82 @@
+{
+  "datasetType": "software",
+  "datasetVersion": {
+    "license": {
+      "name": "CC0 1.0",
+      "uri": "http://creativecommons.org/publicdomain/zero/1.0"
+    },
+    "metadataBlocks": {
+      "citation": {
+        "fields": [
+          {
+            "value": "pyDataverse",
+            "typeClass": "primitive",
+            "multiple": false,
+            "typeName": "title"
+          },
+          {
+            "value": [
+              {
+                "authorName": {
+                  "value": "Range, Jan",
+                  "typeClass": "primitive",
+                  "multiple": false,
+                  "typeName": "authorName"
+                },
+                "authorAffiliation": {
+                  "value": "University of Stuttgart",
+                  "typeClass": "primitive",
+                  "multiple": false,
+                  "typeName": "authorAffiliation"
+                }
+              }
+            ],
+            "typeClass": "compound",
+            "multiple": true,
+            "typeName": "author"
+          },
+          {
+            "value": [ 
+                { "datasetContactEmail" : {
+                    "typeClass": "primitive",
+                    "multiple": false,
+                    "typeName": "datasetContactEmail",
+                    "value" : "[email protected]"
+                },
+                "datasetContactName" : {
+                    "typeClass": "primitive",
+                    "multiple": false,
+                    "typeName": "datasetContactName",
+                    "value": "Range, Jan"
+                }
+            }],
+            "typeClass": "compound",
+            "multiple": true,
+            "typeName": "datasetContact"
+          },
+          {
+            "value": [ {
+               "dsDescriptionValue":{
+                "value": "A Python module for Dataverse.",
+                "multiple":false,
+               "typeClass": "primitive",
+               "typeName": "dsDescriptionValue"
+            }}],
+            "typeClass": "compound",
+            "multiple": true,
+            "typeName": "dsDescription"
+          },
+          {
+            "value": [
+              "Computer and Information Science"
+            ],
+            "typeClass": "controlledVocabulary",
+            "multiple": true,
+            "typeName": "subject"
+          }
+        ],
+        "displayName": "Citation Metadata"
+      }
+    }
+  }
+}

doc/sphinx-guides/source/_static/api/dataset-create-software.jsonld

@@ -0,0 +1,16 @@
+{
+  "http://purl.org/dc/terms/title": "Darwin's Finches",
+  "http://purl.org/dc/terms/subject": "Medicine, Health and Life Sciences",
+  "http://purl.org/dc/terms/creator": {
+      "https://dataverse.org/schema/citation/authorName": "Finch, Fiona",
+      "https://dataverse.org/schema/citation/authorAffiliation": "Birds Inc."
+  },
+  "https://dataverse.org/schema/citation/datasetContact": {
+    "https://dataverse.org/schema/citation/datasetContactEmail": "[email protected]",
+    "https://dataverse.org/schema/citation/datasetContactName": "Finch, Fiona"
+  },
+  "https://dataverse.org/schema/citation/dsDescription": {
+    "https://dataverse.org/schema/citation/dsDescriptionValue": "Darwin's finches (also known as the Galápagos finches) are a group of about fifteen species of passerine birds."
+  },
+  "https://dataverse.org/schema/core#datasetType": "software"
+}

doc/sphinx-guides/source/admin/dataverses-datasets.rst

@@ -147,7 +147,7 @@ It returns a list in the following format:
 Unlink a Dataset
 ^^^^^^^^^^^^^^^^
 
-Removes a link between a dataset and a Dataverse collection. Only accessible to superusers. ::
+Removes a link between a dataset and a Dataverse collection. Accessible to users with Publish Dataset permissions. ::
 
     curl -H "X-Dataverse-key: $API_TOKEN" -X DELETE http://$SERVER/api/datasets/$linked-dataset-id/deleteLink/$linking-dataverse-alias