Summary

DataONE distinguishes three classes of objects that it will store and manage: data objects, science metadata objects, and resource map documents. Each of these are uniquely identifiable by their persistent identifier (PID), and each has associated SystemMetadata which describes the object type, size, access rules, etc.

• Data objects

  are treated as opaque blobs, and are retrievable via the get method given a persistent identifier (PID). Data objects can be represented in any format, but the repository encourages the use of non-proprietary, open formats such as CSV and netCDF.

• Science metadata objects

  are metadata documents such as EML, FGDC, ISO19115, and so forth that provide metadata describing some data object(s). These are represented in XML according to their respective schema.

• Resource Map objects

  are OAI-ORE documents that describe the aggregations of data and metadata into data packages. Individual data and metadata files can be uploaded to the repository, but to indicate that a set of files is part of an aggregated data package, you must provide a OAI-ORE resource map linking the objects.

  In addition to aggregation, Resource Maps can describe the origin of objects by asserting provenance relationships. These relationships will be displayed on the <%=title%>.

All API access is over HTTPS, and accessed via the <%=apiBaseUrl%>/ endpoint. The relative path prefix /v2/ indicates that we are currently using version 2 of the DataONE API.

The examples below show calls to the production <%=title%> data repository REST endpoint (<%=apiBaseUrl%>), but users should not create test data in the production environment. Instead, please use a test Metacat server to explore the API and create test data (e.g., <%=apiBaseUrl%>).

Quick Reference

Request Format

• GET, HEAD, and DELETE requests only pass parameters as part of the URL. The parameter values must be converted to UTF-8 and appropriately escaped for incorporating into the URL.
• Message bodies (e.g. for POST and PUT requests) are encoded using MIME Multipart, mixed (RFC2046). All information for creating the new object or resource is transmitted in the message body, which is encoded as a MIME multipart/mixed message. We use two types of content in MIME multipart/mixed messages: parameters and files. Parameters are to be used for all simple types (such as a String value). Files are to be used for all complex types (such as an XML structure) and for octet streams.

Response Format

Version 1.0 of the DataONE services only support XML serialization, and this format MUST be used when communicating with the <%=title%>. Request and response documents MUST also be encoded using UTF-8.

Authentication

Two mechanisms are supported for authentication:

• Authentication Tokens passed in the HTTP "Authorization" header
• Client-side SSL certificates (deprecated)

Using Authentication Tokens. In this preferred approach, users sign in to the <%=MetacatUI.appModel.get("repositoryName")%>, and copy an authentication token from their profile page which can then be included in HTTPS requests in the HTTP header "Authorization:" to establish their identity.

Users copy an authentication token from their profile page.

The Authentication Token is a long base64-encoded string of characters that encodes the user's credentials in a signed and validatable JWT token. Each language or tool will have is own mechanism for setting HTTP headers. For example, for curl an authenticated request can be made using the '-H' command line option to set the header, such as: $ export TOKEN='eyJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJodHRwOlwvXC9vcmNp...' $ curl -H "Authorization: Bearer $TOKEN" <%=apiBaseUrl%>/object The token expires in 18 hours, and so a new token will need to be retrieved periodically.

The authentication token should be carefully protected just as you would an account password, as it gives the holder full rights to the account. Do not save tokens in code, and don't check them into version control systems, or otherwise make them available to other people.

Using client-side certificates over SSL.

Deprecated. Client-side SSL certificates are deprecated because certain clients and browsers do not fully support them. Use JWT Authentication tokens as described above instead.

Each language or submission tool will have different mechanisms for setting the client certificate in the SSL session. For example, for Curl the certificate filename is passed in on the command line: curl -X POST --cert /tmp/x509up_u502 ....

The version of curl shipped by Apple on MacOS X 10.9 and later is broken and does not support providing PEM certificates via the command line. Instead, it uses certificates registered in the system keychain, as described on the curl mailing list. Thus, calls to the <%=title%> that require a certificate will fail on the standard Mac curl version, which can be fixed by replacing this with the MacPorts version of curl, or by using a certificate converted to PK12 format.

Authorization

Authorizing access for each object in the system is determined by the policies in its associated SystemMetadata document, which contains fields for rightsHolder, authoritativeMemberNode, and accessPolicy. Here’s an example SystemMetadata document showing these fields on a test server:

Each user in the DataONE network can be identified as a Subject string which links to that user's authenticated identity such as its ORCID identifier. These Subject strings can then be used to grant access to objects within the system.

Note how the object is owned by the ORCID on the rightsHolder field — that user can always do anything to the object. Additional rules are specified in accessPolicy. For each user, you can grant read, write, and changePermission permissions. Note that the special user public has been granted read, which makes the document globally readable by all users, even if they are not logged in. If you want to allow another account to make changes, you could add an allow rule to grant their ORCID the write permission in the SystemMetadata document, and then update that using the updateSystemMetadata API.