NXxpcs¶
Status:
application definition, extends NXobject
Description:
X-ray Photon Correlation Spectroscopy (XPCS) data (results).
The purpose of
NXxpcs
is to document and communicate an accepted vernacular for various XPCS results data in order to support development of community software tools. The definition presented here only represents a starting point and contains fields that a common software tool should support for community acceptance.Additional fields may be added to XPCS results file (either formally or informally). It is expected that this XPCS data will be part of multi-modal data set that could involve e.g., NXcanSAS or 1D and/or 2D data.
Symbols:
The symbol(s) listed here will be used below to coordinate datasets with the same shape.
nP: Number of points
- Groups cited:
NXbeam, NXdata, NXdetector, NXentry, NXinstrument, NXnote, NXpositioner, NXprocess, NXsample
Structure:
entry: (required) NXentry
definition: (required) NX_CHAR
Official NeXus NXDL schema to which this file conforms
Obligatory value:
NXxpcs
entry_identifier: (required) NX_CHAR
Unique identifier for the experiment.
entry_identifier_uuid: (optional) NX_CHAR
(optional) UUID identifier for this entry.
scan_number: (required) NX_INT
Scan number (must be an integer).
NOTE: Link to collection_identifier.
start_time: (required) NX_DATE_TIME
Starting time of experiment, such as “2021-02-11 11:22:33.445566Z”.
end_time: (optional) NX_DATE_TIME
Ending time of experiment, such as “2021-02-11 11:23:45Z”.
data: (required) NXdata
The results data captured here are most commonly required for high throughput, equilibrium dynamics experiments. Data (results) describing on-equilibrium dynamics consume more memory resources so these data are separated.
frame_sum: (optional) NX_NUMBER {units=NX_COUNT}
Two-dimensional summation along the frames stack.
sum of intensity v. time (in the units of “frames”)
frame_average: (optional) NX_NUMBER {units=NX_COUNT}
Two-dimensional average along the frames stack.
average intensity v. time (in the units of “frames”)
g2: (optional) NX_NUMBER {units=NX_DIMENSIONLESS}
normalized intensity auto-correlation function, see Lumma, Rev. Sci. Instr. (2000), Eq 1
\[g_2(\boldsymbol Q,t) = \frac{ \langle I(\boldsymbol Q,t\prime) I(\boldsymbol Q,t\prime + t) \rangle }{ \langle I(\boldsymbol Q,t\prime)\rangle^2 }; t > 0\]Typically, \(g_2\) is a quantity calculated for a group of pixels representing a specific region of reciprocal space. These groupings, or bins, are generically described as \(q\). Some open-source XPCS libraries refer to these bins as “rois”, which are not to be confused with EPICS AreaDetector ROI. See usage guidelines for q_lists and roi_maps within a mask. [1]
In short, \(g_2\) should be ordered according to the roi_map value. In principle, any format is acceptable if the data and its axes are self-describing as per NeXus recommendations. However, the data is preferred in one of the following two formats:
iterable list of linked files (or keys) for each \(g_2\) with 1 file (key) per \(q\), where q is called by the nth roi_map value
2D array [2] with shape (\(g_2\), \(q\)), where q is represented by the nth roi_map value, not the value q value
Note it is expected that “g2” and all quantities following it will be of the same length.
Other formats are acceptable with sufficient axes description.
See references below for related implementation information:
@storage_mode: (required) NX_CHAR
storage_mode describes the format of the data to be loaded
We encourage the documentation of other formats not represented here.
one array representing entire data set (“one_array”)
data exchange format with each key representing one
q
by its corresponding roi_map value (“data_exchange_keys”)Any of these values:
one_array
|data_exchange_keys
|other
g2_derr: (optional) NX_NUMBER {units=NX_DIMENSIONLESS}
error values for the \(g_2\) values.
The derivation of the error is left up to the implemented code. Symmetric error will be expected (\(\pm\) error). The data should be in the same format as
g2
.@storage_mode: (required) NX_CHAR
Any of these values:
one_array
|data_exchange_keys
|other
G2_unnormalized: (optional) NX_NUMBER {units=NX_DIMENSIONLESS}
unnormalized intensity auto-correlation function.
Specifically,
g2
without the denominator. The data should be in the same format asg2
.@storage_mode: (required) NX_CHAR
Any of these values:
one_array
|data_exchange_keys
|other
delay_difference: (optional) NX_INT {units=NX_INT}
delay_difference (also known as delay or lag step)
This is quantized difference so that the “step” between two consecutive frames is one frame (or step
= dt = 1 frame
)It is the “quantized” delay time corresponding to the
g2
values.The unit of delay_differences is
NX_INT
for units of frames (i.e., integers) preferred, refer to NXdetector for conversion to time units.@storage_mode: (required) NX_CHAR
Any of these values:
one_array
|data_exchange_keys
|other
twotime: (optional) NXdata
The data (results) in this section are based on the two-time intensity correlation function derived from a time series of scattering images.
two_time_corr_func: (optional) NX_NUMBER {units=NX_ANY}
two-time correlation of speckle intensity for a given q-bin or roi (represented by the nth roi_map value)
See Fluerasu, Phys Rev E (2007), Eq 1 and Sutton, Optics Express (2003) for an early description applied to X-ray scattering:
\[C(\boldsymbol Q, t_1, t_2) = \frac{ \langle I(\boldsymbol Q, t_1)I(\boldsymbol Q, t_2)\rangle }{ \langle I(\boldsymbol Q,t_1)\rangle \langle I(\boldsymbol Q,t_2)\rangle }\]in which time is quantized by frames. In principle, any data format is acceptable if the data and its axes are self-describing as per NeXus recommendations. However, the data is preferred in one of the following two formats:
iterable list of linked files (or keys) for each q-bin called by the nth roi_map value. data for each bin is a 2D array
3D array with shape (frames, frames, q) or (q, frames, frames), where \(q\) is represented by the nth roi_map value, not the value q value
The computation of this result can be customized. These customizations can affect subsequently derived results (below). The following attributes will be used to manage the customization.
Other normalization methods may be applied, but the method will not be specified in this definition. Some of these normalization methods result in a baseline value of
0
, not1
.The various software libraries use different programming languages. Therefore, we need to specify the
time = 0
origin location of the 2D array for each \(q\).A method to reduce data storage needs is to only record half of the 2D array by populating array elements above or below the array diagonal.
@storage_mode: (required) NX_CHAR
storage_mode describes the format of the data to be loaded
We encourage the documention of other formats represented here.
Any of these values:
one_array_q_first
one_array_q_last
data_exchange_keys
other
@baseline_reference: (required) NX_INT
baseline is the expected value of a full decorrelation
The baseline is a constant value added to the functional form of the auto-correlation function. This value is required.
Any of these values:
0
|1
@time_origin_location: (required) NX_CHAR
time_origin_location is the location of the origin
Any of these values:
upper_left
|lower_left
@populated_elements: (required) NX_CHAR
populated_elements describe the elements of the 2D array that are populated with data
Any of these values:
all
|upper_half
|lower_half
g2_from_two_time_corr_func: (optional) NX_NUMBER {units=NX_DIMENSIONLESS}
frame weighted average along the diagonal direction in
two_time_corr_func
The data format and description should be consistent with that found in “/NXxpcs/entry/data/g2”
iterable list of linked files for each \(g_2\) with 1 file per \(q\)
2D array with shape (\(g_2\), \(q\))
Note that delay_difference is not included here because it is derived from the shape of extracted \(g_2\) because all frames are considered, which is not necessarily the case for \(g_2\).
The computation of this result can be customized. The customization can affect the fitting required to extract quantitative results. The following attributes will be used to manage the customization.
@storage_mode: (required) NX_CHAR
Any of these values:
one_array_q_first
one_array_q_last
data_exchange_keys
other
@baseline_reference: (required) NX_INT
Any of these values:
0
|1
@first_point_for_fit: (required) NX_INT
first_point_for_fit describes if the first point should or should not be used in fitting the functional form of the dynamics to extract quantitative time-scale information.
The first_point_for_fit is True (“1”) or False (“0”). This value is required.
Any of these values:
0
|1
g2_err_from_two_time_corr_func: (optional) NX_NUMBER {units=NX_DIMENSIONLESS}
error values for the \(g_2\) values.
The derivation of the error is left up to the implemented code. Symmetric error will be expected (\(\pm\) error).
@storage_mode: (required) NX_CHAR
Any of these values:
one_array_q_first
one_array_q_last
data_exchange_keys
other
g2_from_two_time_corr_func_partials: (optional) NX_NUMBER {units=NX_DIMENSIONLESS}
subset of frame weighted average along the diagonal direction in
two_time_corr_func
Time slicing along the diagonal can be very sophisticated. This entry currently assumes equal frame-binning. The data formats are highly dependent on the implantation of various analysis libraries. In principle, any data format is acceptable if the data and its axes are self describing as per NeXus recommendations. However, the data is preferred in one of the following two formats:
iterable list of linked files (or keys) for each partial \(g_2\) of each q-bin represented by the roi_map value
3D array with shape (\(g_2\), \(q\), nth_partial)
Note that delay_difference is not included here because it is derived from the shape of extracted \(g_2\).
@storage_mode: (required) NX_CHAR
Any of these values:
one_array
|data_exchange_keys
|other
@baseline_reference: (required) NX_INT
Any of these values:
0
|1
g2_err_from_two_time_corr_func_partials: (optional) NX_NUMBER {units=NX_DIMENSIONLESS}
error values for the \(g_2\) values.
The derivation of the error is left up to the implemented code. Symmetric error will be expected (\(\pm\) error).
instrument: (required) NXinstrument
XPCS instrument Metadata.
Objects can be entered here directly or linked from other objects in the NeXus file (such as within
/entry/instrument
).incident_beam: (required) NXbeam
incident_energy: (required) NX_FLOAT {units=NX_ENERGY}
Incident beam line energy (either keV or eV).
incident_energy_spread: (optional) NX_FLOAT {units=NX_ENERGY}
Spread of incident beam line energy (either keV or eV). This quantity is otherwise known as the energy resolution, which is related to the longitudinal coherence length.
incident_polarization_type: (optional) NX_CHAR
Terse description of the incident beam polarization.
The value can be plain text, such as
vertical
,C+
,circular left
.extent: (optional) NX_FLOAT {units=NX_LENGTH}
Size (2-D) of the beam at this position.
DETECTOR: (required) NXdetector
XPCS data is typically produced by area detector (likely EPICS AreaDetector) as a stack of 2D images. Sometimes this data is represented in different ways (sparse arrays or photon event list), but this detail is left to the analysis software. Therefore, we only include requirements based on full array data.
We note that the image origin (pixel coordinates (0,0)) are found at the top left of a single 2D image array. This is the standard expected by Coherent X-ray Imaging Data Bank. [3] See CXI version 1.6 and Maia, Nature Methods (2012). This seems to be consistent with matplotlib and the practiced implementation of EPICS AreaDetector. However, some exceptions may exists in the CXI documentation (See Fig 11 vs Fig 12).
Additionally, not all NXdetector dependencies are inherited from AreaDetector or other control systems.
frame_time
is used to convertdelay_difference
to seconds.frame_time
field could be missing from AreaDetector or may either be acquire_period or acquire_time, depending on the detector model and the local implementation.description: (optional) NX_CHAR
Detector name.
distance: (optional) NX_NUMBER {units=NX_LENGTH}
Distance between sample and detector.
count_time: (required) NX_NUMBER {units=NX_TIME}
Exposure time of frames, s.
frame_time: (required) NX_NUMBER {units=NX_TIME}
Exposure period (time between frame starts) of frames, s
beam_center_x: (required) NX_NUMBER {units=NX_LENGTH}
Position of beam center, x axis, in detector’s coordinates.
beam_center_y: (required) NX_NUMBER {units=NX_LENGTH}
Position of beam center, y axis, in detector’s coordinates.
x_pixel_size: (optional) NX_NUMBER {units=NX_LENGTH}
Length of pixel in x direction.
y_pixel_size: (optional) NX_NUMBER {units=NX_LENGTH}
Length of pixel in y direction.
masks: (optional) NXnote
Data masks or mappings to regions of interest (roi) for specific \(Q\) values
Fields in this
masks
group describe regions of interest in the data by either a mask to select pixels or to associate a map of rois with a (one-dimensional) list of values.“roi_maps” provide for representation of pixel binning that are arbitrary and irregular, which is geometry scattering agnostic and most flexible. The maps work as a labeled array for N rois.
“Dynamic” represents quantities directly related to XPCS and NXxcps/entry/data and NXxpcs/entry/two_time.
“Static” refers to finer binning used for computation not strictly used for the final XPCS results. Implementation of _static_ binning is left for individual libraries to document. We encourage usage of NXcanSAS to represent standard SAXS results or development of new NeXus definitions for GI-SAXS or other reciprocal space intensity mapping.
dynamic_roi_map: (required) NX_NUMBER {units=NX_DIMENSIONLESS}
roi index array or labeled array
The values of this mask index (or map to) the \(Q\) value from the the
dynamic_q_list
field. Not that the value of0
represents in-action. XPCS computations are performed on all pixels with a value > 0.The
units
attribute should be set to"au"
indicating arbitrary units.dynamic_q_list: (optional) NX_NUMBER {units=NX_PER_LENGTH}
1-D list of \(Q\) values, one for each roi index value.
List order is determined by the index value of the associated roi map starting at
1
.The only requirement for the list is that it may be iterable. Some expected formats are:
iterable list of floats (i.e., \(Q(r)\))
iterable list of tuples (i.e., \(Q(r)\), \(\varphi\)), but preferable use the seperate \(\varphi\) field below
iterable list of tuples (e.g., (H, K, L); (qx, qy, qz); (horizontal_pixel, vertical_pixel))
iterable list of integers (for Nth roi_map value) or strings
This format is chosen because results plotting packages are not common and simple I/O is required by end user. The lists can be accessed as lists, arrays or via keys
dynamic_phi_list: (optional) NX_NUMBER {units=NX_PER_LENGTH}
Array of \(\varphi\) value for each pixel.
List order is determined by the index value of the associated roi map starting at
1
.static_roi_map: (optional) NX_NUMBER {units=NX_DIMENSIONLESS}
roi index array.
The values of this mask index the \(|Q|\) value from the the
static_q_list
field.The
units
attribute should be set to"au"
indicating arbitrary units.static_q_list: (optional) NX_NUMBER {units=NX_PER_LENGTH}
1-D list of \(|Q|\) values, 1 for each roi.
sample: (optional) NXsample
temperature_set: (optional) NX_NUMBER {units=NX_TEMPERATURE}
Sample temperature setpoint, (C or K).
temperature: (optional) NX_NUMBER {units=NX_TEMPERATURE}
Sample temperature actual, (C or K).
position_x: (optional) NXpositioner
position_y: (optional) NXpositioner
position_z: (optional) NXpositioner
NOTE: (optional) NXnote
Any other notes.
NAME: The NeXus convention, to use all upper case to indicate the name (here
NOTE
), is left to the file writer. In our case, follow the suggested name pattern and sequence: note_1, note_2, note_3, … Start withnote_1
if the first one, otherwise pick the next number in this sequence.PROCESS: (required) NXprocess
Describe the computation process that produced these results.
Hypertext Anchors¶
List of hypertext anchors for all groups, fields, attributes, and links defined in this class.
/NXxpcs/entry/instrument/incident_beam/incident_energy-field
/NXxpcs/entry/instrument/incident_beam/incident_energy_spread-field
/NXxpcs/entry/instrument/incident_beam/incident_polarization_type-field
/NXxpcs/entry/twotime/g2_err_from_two_time_corr_func@storage_mode-attribute
/NXxpcs/entry/twotime/g2_err_from_two_time_corr_func_partials-field
/NXxpcs/entry/twotime/g2_from_two_time_corr_func@baseline_reference-attribute
/NXxpcs/entry/twotime/g2_from_two_time_corr_func@first_point_for_fit-attribute
/NXxpcs/entry/twotime/g2_from_two_time_corr_func@storage_mode-attribute
/NXxpcs/entry/twotime/g2_from_two_time_corr_func_partials-field
/NXxpcs/entry/twotime/g2_from_two_time_corr_func_partials@baseline_reference-attribute
/NXxpcs/entry/twotime/g2_from_two_time_corr_func_partials@storage_mode-attribute
/NXxpcs/entry/twotime/two_time_corr_func@baseline_reference-attribute
/NXxpcs/entry/twotime/two_time_corr_func@populated_elements-attribute
/NXxpcs/entry/twotime/two_time_corr_func@storage_mode-attribute
/NXxpcs/entry/twotime/two_time_corr_func@time_origin_location-attribute