- Home
- Users & Science
- Support and Infrastructure
- Software
- The Data Portal
The Data Portal
The Data Portal stores experiment data and corresponding metadata produced at the ESRF. It provides access to all raw data files. For some experiments, processed data is also available. The portal relies on ICAT for storing the experiment metadata.
ICAT is an open source metadata management system designed for large facilities and provided by PaNdata (consortium of all photon and neutron sources in Europe, dedicated to sharing good practices of data management). It enables the collection, storage and searching of the generic metadata of experiments along with the corresponding raw data, therefore enabling their reuse at a later stage.
More information on the ICAT project can be found here.
Only the proposers and experiment team can access these data during an embargo period of three years after the experiment, after which time the data will be automatically made public for other users to see and use. This applies to non-proprietary research experiments only - see the ESRF Data Policy for more details and exceptions.
Automatically collected metadata for ESRF experiments is available via the Data Portal. Login with your User Portal credentials is only required for data under embargo.
Click on the menu "Experiments" and filter by "My data" to display the experiments where you are an experimental team member, or on the menu "Publications" to display the data that are public.
Look for the dataset you are interested in either by using the search tool or by using the proposal number and/or beamline number. Click on “Download” to retrieve the data in a zip file.
If the data is too large, the best option for downloading large datasets reliably is Globus: this is a service for downloading and transferring files. In order to use Globus to download files to your computer, you first need to install Globus Connect Personal. Once installed, start the software, and log in to create a local endpoint.
A scientist cannot access the data from a past experiment unless they were originally listed as a participant on the A-form for that experiment. If this is not the case, the scientist must contact the main proposer of the proposal and request that access be granted. Once access is approved, the new user will be able to access both the experimental data and the electronic logbook, with the same permissions as other members of the experiment session. Their role will be listed as collaborator.
How-to add a collaborator documentation (login required).
According to the ESRF Data Policy and ESRF Publication Guidelines, every time you publish your research analysis issued from raw data measured at ESRF, you must mention the ESRF beamline on which the data were measured and include the data DOI in your manuscript in order for other scientists to reuse your data once it becomes openly accessible.
The DOI for an experiment session can be found both in the Data Portal and in the User Portal
- In the Data Portal
Click on My Data, to see all the experiment sessions you have been involved in.
Use the search box to find a specific session keyword or sort your sessions by proposal, beamline or date.
Once you have found the session you are looking for, you will see the DOI link for that session in the right column of the table. Click on this link to go directly to the DOI landing page.
- In the User Portal
Under the tab "Proposals/Experiments":
Select "A Form": a table with the list of your session(s) is proposed.
Click on the DOI icon ("Further Actions" in the right of column of the table) to get the DOI number corresponding to the selected session: this will also lead you to the DOI page with all the information on the session.
For many publications, the data referred to in the publication are a subset of one or more experiment sessions, or include data from more than one session. For this reason, researchers may prefer to manually create a DOI for a specific collection of datasets within one or more experiment sessions.
As a member of an experiment session, you can create a bespoke DOI for the specific data used in a publication. This DOI refers exclusively to the selected datasets and can be minted directly from the Data Portal. When minted, the DOI immediately makes the associated datasets publicly accessible. You also have the flexibility to include data from multiple experiment sessions when creating the DOI. If you need to reference the DOI before the publication is finalized, you can choose to reserve it in advance for future use.
As experimental team member, you can access the Data Portal documentation here (login required).