NXxpcs

Status:

application definition, extends NXobject

Description:

X-ray Photon Correlation Spectroscopy (XPCS) data (results).

The purpose of NXxpcs is to document and communicate an accepted vernacular for various XPCS results data in order to support development of community software tools. The definition presented here only represents a starting point and contains fields that a common software tool should support for community acceptance.

Additional fields may be added to XPCS results file (either formally or informally). It is expected that this XPCS data will be part of multi-modal data set that could involve e.g., NXcanSAS or 1D and/or 2D data.

Symbols:

The symbol(s) listed here will be used below to coordinate datasets with the same shape.

nP: Number of points

Groups cited:

NXbeam, NXdata, NXdetector, NXentry, NXinstrument, NXnote, NXpositioner, NXprocess, NXsample

Structure:

entry: (required) NXentry

definition: (required) NX_CHAR

Official NeXus NXDL schema to which this file conforms

Obligatory value: NXxpcs

entry_identifier: (required) NX_CHAR

Locally unique identifier for the experiment (a.k.a. run or scan).

  • For bluesky users, this is the run’s “scan_id”.

  • For SPEC users, this is the scan number (SCAN_N).

entry_identifier_uuid: (optional) NX_CHAR

(optional) UUID identifier for this entry.

See the UUID standard (or wikipedia) for more information.

  • For bluesky users, this is the run’s “uid” and is expected for that application.

  • Typically, SPEC users will not use this field without further engineering.

scan_number: (required) NX_INT

DEPRECATED: Use the entry_identifier field.

Scan number (must be an integer).

NOTE: Link to collection_identifier.

start_time: (required) NX_DATE_TIME

Starting time of experiment, such as “2021-02-11 11:22:33.445566Z”.

end_time: (optional) NX_DATE_TIME

Ending time of experiment, such as “2021-02-11 11:23:45Z”.

data: (required) NXdata

The results data captured here are most commonly required for high throughput, equilibrium dynamics experiments. Data (results) describing on-equilibrium dynamics consume more memory resources so these data are separated.

frame_sum: (optional) NX_NUMBER {units=NX_COUNT}

Two-dimensional summation along the frames stack.

sum of intensity v. time (in the units of “frames”)

frame_average: (optional) NX_NUMBER {units=NX_COUNT}

Two-dimensional average along the frames stack.

average intensity v. time (in the units of “frames”)

g2: (optional) NX_NUMBER {units=NX_DIMENSIONLESS}

normalized intensity auto-correlation function, see Lumma, Rev. Sci. Instr. (2000), Eq 1

\[g_2(\boldsymbol Q,t) = \frac{ \langle I(\boldsymbol Q,t\prime) I(\boldsymbol Q,t\prime + t) \rangle }{ \langle I(\boldsymbol Q,t\prime)\rangle^2 }; t > 0\]

Typically, \(g_2\) is a quantity calculated for a group of pixels representing a specific region of reciprocal space. These groupings, or bins, are generically described as \(q\). Some open-source XPCS libraries refer to these bins as “rois”, which are not to be confused with EPICS AreaDetector ROI. See usage guidelines for q_lists and roi_maps within a mask. [1]

In short, \(g_2\) should be ordered according to the roi_map value. In principle, any format is acceptable if the data and its axes are self-describing as per NeXus recommendations. However, the data is preferred in one of the following two formats:

  • iterable list of linked files (or keys) for each \(g_2\) with 1 file (key) per \(q\), where q is called by the nth roi_map value

  • 2D array [2] with shape (\(g_2\), \(q\)), where q is represented by the nth roi_map value, not the value q value

Note it is expected that “g2” and all quantities following it will be of the same length.

Other formats are acceptable with sufficient axes description.

See references below for related implementation information:

@storage_mode: (required) NX_CHAR

storage_mode describes the format of the data to be loaded

We encourage the documentation of other formats not represented here.

  • one array representing entire data set (“one_array”)

  • data exchange format with each key representing one q by its corresponding roi_map value (“data_exchange_keys”)

Any of these values: one_array | data_exchange_keys | other

g2_derr: (optional) NX_NUMBER {units=NX_DIMENSIONLESS}

error values for the \(g_2\) values.

The derivation of the error is left up to the implemented code. Symmetric error will be expected (\(\pm\) error). The data should be in the same format as g2.

@storage_mode: (required) NX_CHAR

Any of these values: one_array | data_exchange_keys | other

G2_unnormalized: (optional) NX_NUMBER {units=NX_ANY}

unnormalized intensity auto-correlation function.

Specifically, g2 without the denominator. The data should be in the same format as g2.

@storage_mode: (required) NX_CHAR

Any of these values: one_array | data_exchange_keys | other

delay_difference: (optional) NX_INT {units=NX_COUNT}

delay_difference (also known as delay or lag step)

This is quantized difference so that the “step” between two consecutive frames is one frame (or step = dt = 1 frame)

It is the “quantized” delay time corresponding to the g2 values.

The unit of delay_differences is NX_INT for units of frames (i.e., integers) preferred, refer to NXdetector for conversion to time units.

@storage_mode: (required) NX_CHAR

Any of these values: one_array | data_exchange_keys | other

twotime: (optional) NXdata

The data (results) in this section are based on the two-time intensity correlation function derived from a time series of scattering images.

two_time_corr_func: (optional) NX_NUMBER {units=NX_ANY}

two-time correlation of speckle intensity for a given q-bin or roi (represented by the nth roi_map value)

See Fluerasu, Phys Rev E (2007), Eq 1 and Sutton, Optics Express (2003) for an early description applied to X-ray scattering:

\[C(\boldsymbol Q, t_1, t_2) = \frac{ \langle I(\boldsymbol Q, t_1)I(\boldsymbol Q, t_2)\rangle }{ \langle I(\boldsymbol Q,t_1)\rangle \langle I(\boldsymbol Q,t_2)\rangle }\]

in which time is quantized by frames. In principle, any data format is acceptable if the data and its axes are self-describing as per NeXus recommendations. However, the data is preferred in one of the following two formats:

  • iterable list of linked files (or keys) for each q-bin called by the nth roi_map value. data for each bin is a 2D array

  • 3D array with shape (frames, frames, q) or (q, frames, frames), where \(q\) is represented by the nth roi_map value, not the value q value

The computation of this result can be customized. These customizations can affect subsequently derived results (below). The following attributes will be used to manage the customization.

  • Other normalization methods may be applied, but the method will not be specified in this definition. Some of these normalization methods result in a baseline value of 0, not 1.

  • The various software libraries use different programming languages. Therefore, we need to specify the time = 0 origin location of the 2D array for each \(q\).

  • A method to reduce data storage needs is to only record half of the 2D array by populating array elements above or below the array diagonal.

@storage_mode: (required) NX_CHAR

storage_mode describes the format of the data to be loaded

We encourage the documention of other formats represented here.

Any of these values:

  • one_array_q_first

  • one_array_q_last

  • data_exchange_keys

  • other

@baseline_reference: (required) NX_INT

baseline is the expected value of a full decorrelation

The baseline is a constant value added to the functional form of the auto-correlation function. This value is required.

Any of these values: 0 | 1

@time_origin_location: (required) NX_CHAR

time_origin_location is the location of the origin

Any of these values: upper_left | lower_left

@populated_elements: (required) NX_CHAR

populated_elements describe the elements of the 2D array that are populated with data

Any of these values: all | upper_half | lower_half

g2_from_two_time_corr_func: (optional) NX_NUMBER {units=NX_DIMENSIONLESS}

frame weighted average along the diagonal direction in two_time_corr_func

The data format and description should be consistent with that found in “/NXxpcs/entry/data/g2”

  • iterable list of linked files for each \(g_2\) with 1 file per \(q\)

  • 2D array with shape (\(g_2\), \(q\))

Note that delay_difference is not included here because it is derived from the shape of extracted \(g_2\) because all frames are considered, which is not necessarily the case for \(g_2\).

The computation of this result can be customized. The customization can affect the fitting required to extract quantitative results. The following attributes will be used to manage the customization.

@storage_mode: (required) NX_CHAR

Any of these values:

  • one_array_q_first

  • one_array_q_last

  • data_exchange_keys

  • other

@baseline_reference: (required) NX_INT

Any of these values: 0 | 1

@first_point_for_fit: (required) NX_INT

first_point_for_fit describes if the first point should or should not be used in fitting the functional form of the dynamics to extract quantitative time-scale information.

The first_point_for_fit is True (“1”) or False (“0”). This value is required.

Any of these values: 0 | 1

g2_err_from_two_time_corr_func: (optional) NX_NUMBER {units=NX_DIMENSIONLESS}

error values for the \(g_2\) values.

The derivation of the error is left up to the implemented code. Symmetric error will be expected (\(\pm\) error).

@storage_mode: (required) NX_CHAR

Any of these values:

  • one_array_q_first

  • one_array_q_last

  • data_exchange_keys

  • other

g2_from_two_time_corr_func_partials: (optional) NX_NUMBER {units=NX_DIMENSIONLESS}

subset of frame weighted average along the diagonal direction in two_time_corr_func

Time slicing along the diagonal can be very sophisticated. This entry currently assumes equal frame-binning. The data formats are highly dependent on the implantation of various analysis libraries. In principle, any data format is acceptable if the data and its axes are self describing as per NeXus recommendations. However, the data is preferred in one of the following two formats:

  • iterable list of linked files (or keys) for each partial \(g_2\) of each q-bin represented by the roi_map value

  • 3D array with shape (\(g_2\), \(q\), nth_partial)

Note that delay_difference is not included here because it is derived from the shape of extracted \(g_2\).

@storage_mode: (required) NX_CHAR

Any of these values: one_array | data_exchange_keys | other

@baseline_reference: (required) NX_INT

Any of these values: 0 | 1

g2_err_from_two_time_corr_func_partials: (optional) NX_NUMBER {units=NX_DIMENSIONLESS}

error values for the \(g_2\) values.

The derivation of the error is left up to the implemented code. Symmetric error will be expected (\(\pm\) error).

instrument: (required) NXinstrument

XPCS instrument Metadata.

Objects can be entered here directly or linked from other objects in the NeXus file (such as within /entry/instrument).

incident_beam: (required) NXbeam

incident_energy: (required) NX_FLOAT {units=NX_ENERGY}

Incident beam line energy (either keV or eV).

incident_energy_spread: (optional) NX_FLOAT {units=NX_ENERGY}

Spread of incident beam line energy (either keV or eV). This quantity is otherwise known as the energy resolution, which is related to the longitudinal coherence length.

incident_polarization_type: (optional) NX_CHAR

Terse description of the incident beam polarization.

The value can be plain text, such as vertical, C+, circular left.

extent: (optional) NX_FLOAT {units=NX_LENGTH}

Size (2-D) of the beam at this position.

DETECTOR: (required) NXdetector

XPCS data is typically produced by area detector (likely EPICS AreaDetector) as a stack of 2D images. Sometimes this data is represented in different ways (sparse arrays or photon event list), but this detail is left to the analysis software. Therefore, we only include requirements based on full array data.

We note that the image origin (pixel coordinates (0,0)) are found at the top left of a single 2D image array. This is the standard expected by Coherent X-ray Imaging Data Bank. [3] See CXI version 1.6 and Maia, Nature Methods (2012). This seems to be consistent with matplotlib and the practiced implementation of EPICS AreaDetector. However, some exceptions may exists in the CXI documentation (See Fig 11 vs Fig 12).

Additionally, not all NXdetector dependencies are inherited from AreaDetector or other control systems. frame_time is used to convert delay_difference to seconds. frame_time field could be missing from AreaDetector or may either be acquire_period or acquire_time, depending on the detector model and the local implementation.

description: (optional) NX_CHAR

Detector name.

distance: (optional) NX_NUMBER {units=NX_LENGTH}

Distance between sample and detector.

count_time: (required) NX_NUMBER {units=NX_TIME}

Exposure time of frames, s.

frame_time: (required) NX_NUMBER {units=NX_TIME}

Exposure period (time between frame starts) of frames, s

beam_center_x: (required) NX_NUMBER {units=NX_LENGTH}

Position of beam center, x axis, in detector’s coordinates.

beam_center_y: (required) NX_NUMBER {units=NX_LENGTH}

Position of beam center, y axis, in detector’s coordinates.

x_pixel_size: (optional) NX_NUMBER {units=NX_LENGTH}

Length of pixel in x direction.

y_pixel_size: (optional) NX_NUMBER {units=NX_LENGTH}

Length of pixel in y direction.

masks: (optional) NXnote

Data masks or mappings to regions of interest (roi) for specific \(Q\) values

Fields in this masks group describe regions of interest in the data by either a mask to select pixels or to associate a map of rois with a (one-dimensional) list of values.

“roi_maps” provide for representation of pixel binning that are arbitrary and irregular, which is geometry scattering agnostic and most flexible. The maps work as a labeled array for N rois.

“Dynamic” represents quantities directly related to XPCS and NXxcps/entry/data and NXxpcs/entry/two_time.

“Static” refers to finer binning used for computation not strictly used for the final XPCS results. Implementation of _static_ binning is left for individual libraries to document. We encourage usage of NXcanSAS to represent standard SAXS results or development of new NeXus definitions for GI-SAXS or other reciprocal space intensity mapping.

dynamic_roi_map: (required) NX_NUMBER {units=NX_DIMENSIONLESS}

roi index array or labeled array

The values of this mask index (or map to) the \(Q\) value from the the dynamic_q_list field. Not that the value of 0 represents in-action. XPCS computations are performed on all pixels with a value > 0.

The units attribute should be set to "au" indicating arbitrary units.

dynamic_q_list: (optional) NX_NUMBER {units=NX_PER_LENGTH}

1-D list of \(Q\) values, one for each roi index value.

List order is determined by the index value of the associated roi map starting at 1.

The only requirement for the list is that it may be iterable. Some expected formats are:

  • iterable list of floats (i.e., \(Q(r)\))

  • iterable list of tuples (i.e., \(Q(r)\), \(\varphi\)), but preferable use the seperate \(\varphi\) field below

  • iterable list of tuples (e.g., (H, K, L); (qx, qy, qz); (horizontal_pixel, vertical_pixel))

  • iterable list of integers (for Nth roi_map value) or strings

This format is chosen because results plotting packages are not common and simple I/O is required by end user. The lists can be accessed as lists, arrays or via keys

dynamic_phi_list: (optional) NX_NUMBER {units=NX_PER_LENGTH}

Array of \(\varphi\) value for each pixel.

List order is determined by the index value of the associated roi map starting at 1.

static_roi_map: (optional) NX_NUMBER {units=NX_DIMENSIONLESS}

roi index array.

The values of this mask index the \(|Q|\) value from the the static_q_list field.

The units attribute should be set to "au" indicating arbitrary units.

static_q_list: (optional) NX_NUMBER {units=NX_PER_LENGTH}

1-D list of \(|Q|\) values, 1 for each roi.

sample: (optional) NXsample

temperature_set: (optional) NX_NUMBER {units=NX_TEMPERATURE}

Sample temperature setpoint, (C or K).

temperature: (optional) NX_NUMBER {units=NX_TEMPERATURE}

Sample temperature actual, (C or K).

position_x: (optional) NXpositioner

position_y: (optional) NXpositioner

position_z: (optional) NXpositioner

NOTE: (optional) NXnote

Any other notes.

NAME: The NeXus convention, to use all upper case to indicate the name (here NOTE), is left to the file writer. In our case, follow the suggested name pattern and sequence: note_1, note_2, note_3, … Start with note_1 if the first one, otherwise pick the next number in this sequence.

PROCESS: (required) NXprocess

Describe the computation process that produced these results.

Hypertext Anchors

List of hypertext anchors for all groups, fields, attributes, and links defined in this class.

NXDL Source:

https://github.com/FAIRmat-Experimental/nexus_definitions/tree/fairmat/contributed_definitions/NXxpcs.nxdl.xml