.. _sec-vader: VADER: Trending Analysis for Code/Data Validation ================================================= *S. Hart, J. Clarity, L. Fassino* VADER (Validation Analysis Data Evaluation Resource) is a new module in SCALE 6.3 derived from the legacy USLSTATS program. VADER is a tool that allows the determination of bias and bias uncertainty for criticality safety computational methods. The older USLSTATS program, written in Java, existed outside of SCALE and provided tools to calculate only the confidence band with administrative margin (sometimes called USL-1) and the single-sided uniform width closed interval (USL-2). For normality testing it offered only a crude chi-squared normality test that had no user-configurable options and presented a simple pass/no-pass functionality. VADER has been written completely in C++ using modern coding practices and standards. It has been moved to be part of the SCALE code package and is run through the normal SCALE executables. To support the VADER methods, large updates were made to the statistical capabilities of SCALE. Interfaces to known statistical distributions were added and many of the statistical calculations required for VADER were made available to all SCALE sequences. In addition to being able to process legacy USLSTATS inputs, VADER adds support for the single-sided tolerance band method (from NUREG/CR-6698 :cite:`VADER-dean-nureg-cr-6698`) as well as two non-trending methods: the "historical" nonparametric and parametric methods, also as described in NUREG/CR-6698 :cite:`VADER-dean-nureg-cr-6698`. To support determination of the appropriate method for a given dataset, VADER also includes the t-test for trend significance and the Anderson-Darling and chi-squared tests for normality. VADER has also been integrated into the Fulcrum GUI used by SCALE. This allows VADER to benefit from the auto-complete, templating, and input validation features offered by Fulcrum. In addition, Fulcrum can plot the results given by VADER, and the plot files have been expanded to include the new methods available in VADER. The methods outlined herein are consistent with the descriptions available in "Determination of Bias and Bias Uncertainty for Criticality Safety Computational Methods" :cite:`VADER-clarity-nureg-bias`, which provides complete detail on the origins and limitations for these methods. Some guidance on the usage of these methods and tests is provided below, but the user is advised to reference :cite:`VADER-clarity-nureg-bias` for more information. .. _sec-vader.input: Input ----- The VADER input is based on the SON input syntax common in other modernized SCALE codes such as :ref:`ORIGEN ` and :ref:`ORIGAMI `. The SON input format consists of key/value pairs where the value can be either a single item, an array of items, or other SON blocks. The VADER input generally has one level of depth. There are no nested SON blocks. Input data is primarily structured using the SCALE array format. An overview of top level items is provided in :numref:`tab-vader.input_summary_table`, which are elaborated upon in the :ref:`detailed input ` section. .. table:: Summary of top level options for VADER :align: center :name: tab-vader.input_summary_table ===================================== ============ ========= **Keyword** **Required** **Description** ------------------------------------- ------------ --------- :ref:`data ` Yes Data entered in triplets in the form ``X Y unc.``. ------------------------------------- ------------ --------- :ref:`title ` No Title to give the case. ------------------------------------- ------------ --------- :ref:`parameters ` No Global parameters that apply to each test or trend. ------------------------------------- ------------ --------- :ref:`trend_values ` No User-specified trend values to evaluate in selected method(s). ------------------------------------- ------------ --------- :ref:`methods ` Yes Selected methods to apply to the data. ------------------------------------- ------------ --------- :ref:`tests ` No Statistical tests to apply to the data. ===================================== ============ ========= .. |data| replace:: ``data`` .. _sec-vader.legacy_input: Legacy Input ~~~~~~~~~~~~ For backwards compatibility VADER supports reading the original USLSTATS format. This format is restricted to USL methods USL1 and USL2. Results should match those obtained using the Java-based USLSTATS program with some exceptions detailed below. * The extrapolation parameter has been updated to be more rigorous. Only USL1 can do extrapolation and, if turned off, values outside of the range will be labeled in the table as `N/A`. USL2 will always give `N/A` for values outside of the range. * The USL1 method one will always use the maximum band width inside the x-range, :math:`W`, inside of the range and then the larger of :math:`W` or :math:`w(x)` outside of the range (if extrapolating). This differs from legacy USLSTATS that used `w(x)` across the entire range if extrapolating. * The administrative margin parameter in legacy USLSTATS uses a default value of 0.03, which differs from the VADER default value of 0.05. .. _sec-vader.detailed: Detailed Input -------------- .. _sec-vader.data: ``data`` ~~~~~~~~ The ``data`` input array is a 1-dimensional array consisting of data triplets. There is no punctuation between the triplets. The data is entered in the order :math:`X_1 \, Y_1 \, \sigma_1 \; X_2 \, Y_2 \, \sigma_2 \; ... \; X_N \, Y_N \, \sigma_N`. Where ============== ============================== *X* The trended value, usually something like enrichment or moderator density. -------------- ------------------------------ *Y* The y value, usually something like C/E or eigenvalue. -------------- ------------------------------ :math:`\sigma` The standard deviation of *Y*. ============== ============================== VADER will emit an error if the number of entries in this array is not divisible by three. .. code-block:: scale :caption: Snippet of ``data`` input data=[2.4 0.9976 0.00122 2.6 0.9978 0.00022 ... 1.9 1.0011 0.00213 1.99 1.013 0.00012 ] .. _sec-vader.title: ``title`` ~~~~~~~~~ The ``title`` entry allows the user to specify a title for the case. The title is cosmetic and doesn't affect any of the results. The title is echoed to the output in several places and is included on some of the graphs generated by VADER. .. code-block:: scale :caption: ``title`` title="My Very First VADER Case" .. _sec-vader.param: ``parameters`` ~~~~~~~~~~~~~~ The ``parameters`` section can exist outside of any test or method. If a parameter is listed in this global section, then it will apply to any method or test that has that parameter available instead of the default value. Values in this global ``parameters`` section can be overridden in the specific method or test block for which the user wishes to override the global parameter. For a list of all parameters, their default values, and which tests and methods they are applied to, see :numref:`sec-vader.available_parameters`. .. code-block:: scale :caption: ``parameters`` array parameters {extrapolate=yes admin_margin=0.03 } .. _sec-vader.trend: ``trend_values`` ~~~~~~~~~~~~~~~~ This array allows the user to insert values at which they wish the trend *Y* (typically the upper subcritical limit) values to be calculated. Extrapolation outside of the data range is only available if the method supports it and extrapolation is enabled. .. code-block:: scale :caption: ``trend_values`` array trend_values=[-0.1 0.2 2.4 10.0 ] .. _sec-vader.methods: ``methods`` ~~~~~~~~~~~ This is where the user specifies the methods to enable and, optionally, gives method-specific parameters. Each method is followed by an array of the parameters the user wishes to set. Any omitted parameters are initialized with the default value (see :numref:`vader.parameters_table`). If the user wishes to enable a method with all default values, then an empty array can be given. .. code-block:: scale :caption: `methods` block with some default methods and some specific parameters methods { USL1 {} % uses default parameters CR6698 {weighted=yes admin_margin=0.03 chi_p=0.97 } parametric {confidence=0.9 } } .. _sec-vader.tests: ``tests`` ~~~~~~~~~ This block is similar to the `methods` :ref:`block `, but it allows the user to specify which tests they wish to enable. These tests are run on the data and generally either pass or fail and give a few statistical values as their result. Most of the tests implemented for VADER for now are normality tests. .. code-block:: scale :caption: `tests` block tests { chisq {} % default parameters t-test {confidence=0.98 } } .. _sec-vader.available_methods: Available Methods ----------------- The sections below indicate the input name to be used to activate the available method. The origins and use cases for these methods are provided in :cite:`VADER-clarity-nureg-bias`, which should be referenced to assist in determining which method is suitable for a given dataset. The implementation of these methods is described with respect to calculating the bias, bias uncertainty, and the upper subcritical limit (USL) based on a dataset consisting of some independent variable, :math:`x_i`, with its associated :math:`k_{norm,i}` and uncertainty, :math:`\sigma_i`. The normalized :math:`k_{eff}` (:math:`k_{norm}`) is the ratio of an experiment's calculated :math:`k_{eff}` (:math:`k_{calc}`) to its expected :math:`k_{eff}` (:math:`k_{exp}`), as shown in :eq:`knorm`. Its respective combined uncertainty, :math:`\sigma_i`, is the squared sum of the associated relative variances for :math:`k_{calc}` and :math:`k_{exp}`, scaled to convert to absolute uncertainty by :math:`k_{norm}`, as shown in :eq:`knorm-unc`. .. math:: k_{norm} = \frac{k_{calc}}{k_{exp}} :label: knorm .. math:: \sigma = k_{norm} \sqrt{\left(\frac{\sigma_{exp}}{k_{exp}}\right)^{2} + \left(\frac{\sigma_{calc}}{k_{calc}}\right)^{2}} :label: knorm-unc Typically, :math:`\sigma_{calc}` is the stochastic uncertainty associated with a Monte Carlo calculation of :math:`k_{eff}`, and it may be referred to as :math:`\sigma_{MC}` instead. Additionally, the combined uncertainty expression in :eq:`knorm-unc` may be reduced to the square root of the sum of variances :math:`\sigma_{calc}^2` and :math:`\sigma_{exp}^2` as an approximation, as the difference between the absolute and relative uncertainties are typically very small given that the values for :math:`k` in this context are expected to be near 1. This approximation yields the more commonly seen form in :eq:`knorm-unc-approx`. .. math:: \sigma = \sqrt{\sigma_{exp}^{2} + \sigma_{calc}^{2}} :label: knorm-unc-approx Each method calculates the USL according to :eq:`usl`, where the bias, :math:`\beta`, is always less than or equal to zero; positive biases are not credited in the USL calculation for any method. .. math:: USL = 1 + \beta - \sigma_{\beta} - M_A - M_D \text{ for } \beta \leq 0 :label: usl In the above calculation, :math:`\sigma_{\beta}` is the bias uncertainty, :math:`M_A` is the administrative margin, and :math:`M_D` is the margin for other deficiencies. .. note:: The bias is not explicitly reported in the output; the adjusted :math:`k(x)` value, ``Adj. k(x)``, is reported instead. The ``Adj. k(x)`` value is equivalent to :math:`1+\beta(x)`, and it is never greater than 1 since the bias must always be less than or equal to zero. Additionally, the nonparametric margin (NPM) in the ``nonparametric`` method is the only instance where :math:`M_D` exists and may be greater than zero, but it is not reported separately. Instead, the reported bias uncertainty incorporates the NPM when it is greater than zero, which is noted in the output when true. If the :math:`k_{norm}` data has a significant linear trend with respect to the independent parameter, then trending methods ``CR6698``, ``USL1``, and ``USL2`` may be used. If there is no significant linear trend in the dataset but the data has a normal distribution, then the non-trending ``parametric`` method may be used. Otherwise, the ``nonparametric`` method, a non-trending, distribution-free method, may be used. See :ref:`sec-vader.available_tests` for information on testing for trends and the normal distribution. .. _sec-vader.cr6698_method: ``CR6698`` -- Single-Sided Lower Tolerance Band ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The ``CR6698`` method uses the single-sided lower tolerance band method to calculate the USL. In this method, the linear fit of the bias is defined by :eq:`bias-fit`, with a bias uncertainty defined by :eq:`cr6698-bias-unc`. .. math:: :label: bias-fit \beta(x) = k_{fit}(x)-1 .. math:: :label: cr6698-bias-unc \sigma_{\beta}(x) = S_p \left( \sqrt{2 F_{\alpha}^{(2,N-2)} \left[ \frac{1}{N} + \frac{(x-\bar{x})^2}{S_{xx}} \right] } + z_p \sqrt{\frac{N-2}{\chi^{2}_{1-\frac{\alpha}{2},N-2}}} \right) The linear fit for :math:`k_{norm}` as a function of :math:`x` is defined by :eq:`kfit`, where :math:`\beta_0` and :math:`\beta_1` are the intercept and slope of the linear fit defined in :eq:`fit-b0` and :eq:`fit-b1`, respectively. The calculation of bias uncertainty relies on several statistical parameters as well, which are detailed below. .. math:: :label: kfit k_{fit}(x) = \beta_0 + \beta_1 x .. math:: :label: fit-b0 \beta_0 = \bar{k}_{norm} - \beta_1 \bar{x} .. math:: :label: fit-b1 \beta_1 = \frac{S_{kx}}{S_{xx}} :eq:`fit-b0` requires calculation of the average :math:`k_{norm}` and :math:`x` values, which may be calculated using either uncertainty weighted or unweighted methods. The unweighted expression for :math:`\bar{k}_{norm}` is given by :eq:`kbar-unweighted`, and the weighted calculation is given by :eq:`kbar-weighted`. .. math:: :label: kbar-unweighted \bar{k}_{norm} = \frac{\sum_{i=1}^{N} k_{norm_i}}{N} .. math:: :label: kbar-weighted \bar{k}_{norm} = \frac{\sum_{i=1}^{N} \frac{k_{norm_i}}{\sigma_i^2}}{\sum_{i=1}^{N} \frac{1}{\sigma_i^2}} In this equation, :math:`N` is the number of points in the dataset, and :math:`k_{norm_i}` and :math:`\sigma_i` are the normalized :math:`k_{eff}` and the associated combined uncertainty, given by :eq:`knorm` and :eq:`knorm-unc`, respectively. Likewise, the unweighted calculation for :math:`\bar{x}` is given by :eq:`xbar-unweighted`, and the weighted calculation is given by :eq:`xbar-weighted`. .. math:: :label: xbar-unweighted \bar{x} = \frac{\sum_{i=1}^{N} x_{i}}{N} .. math:: :label: xbar-weighted \bar{x} = \frac{\sum_{i=1}^{N} \frac{x_{i}}{\sigma_i^2}}{\sum_{i=1}^{N} \frac{1}{\sigma_i^2}} .. note:: Methods with both unweighted and weighted variations use either the unweighted or weighted expressions consistently throughout the calculation. In :eq:`fit-b1` as well as in :eq:`cr6698-bias-unc`, :math:`S_{xx}`, the sum of squared deviations about the mean for the independent variable, is calculated using the unweighted method with :eq:`sxx-unweighted` and for the weighted method with :eq:`sxx-weighted`. .. math:: :label: sxx-unweighted S_{xx} = \sum^{N}_{i=1} (x_i - \bar{x})^{2} .. math:: :label: sxx-weighted S_{xx} = \frac{ \sum^{N}_{i=1} \left( \frac{1}{\sigma_{i}^2}(x_{i} - \bar{x})^{2} \right)} { \frac{1}{N} \sum^{N}_{i=1} \frac{1}{\sigma_{i}^{2}}} Similarly, the :math:`S_{kx}` calculation is the product sum of deviations of :math:`x` and :math:`k_{norm}` about their respective means, given by :eq:`skx-unweighted` for the unweighted method and :eq:`skx-weighted` for the weighted method. .. math:: :label: skx-unweighted S_{kx} = \sum^{N}_{i=1} (k_{norm_i} - \bar{k}_{norm}) (x_{i} - \bar{x}) .. math:: :label: skx-weighted S_{kx} = \frac{ \sum^{N}_{i=1} \left( \frac{1}{\sigma^{2}_{i}} (x_{i} - \bar{x}) (k_{norm_i}-\bar{k}_{norm}) \right)} { \frac{1}{N} \sum^{N}_{i=1} \frac{1}{\sigma^{2}_{i}} } The calculation of the bias uncertainty per :eq:`cr6698-bias-unc` requires the calculation of several other parameters: :math:`S_p`, the square root of the pooled variance; :math:`F_{\alpha}^{(2,N-2)}`, the inverse of the F probability distribution corresponding to a degree of fit 2 and :math:`N-2` degrees of freedom for a significance of :math:`\alpha`; :math:`z_p`, the inverse of the standard normal cumulative distribution containing fraction :math:`p` of the distribution; and :math:`\chi^{2}_{1-\frac{\alpha}{2},N-2}`, the inverse of the right-tailed probability of the chi-squared distribution. :eq:`fit-sp` describes the calculation for :math:`S_p` of the linear fit. It combines the variance of the fit, :math:`\sigma_{fit}^2`, and the average total variance, :math:`\bar{\sigma}^2`, which is sometimes referred to as the "within variance." The average total variance is given by :eq:`avg-tot-var-unweighted` for the unweighted method and by :eq:`avg-tot-var-weighted` for the weighted method. The unweighted calculation of :math:`\sigma_{fit}^2` is given by :eq:`fit-unc-unweighted` for the unweighted method and by :eq:`fit-unc-weighted` for the weighted method. .. math:: :label: fit-sp S_p = \sqrt{\sigma_{fit}^2 + \bar{\sigma}^2} .. math:: :label: avg-tot-var-unweighted \bar{\sigma}^2 = \frac{\sum_{i=1}^{N}\sigma_{i}^{2}}{N} .. math:: :label: avg-tot-var-weighted \bar{\sigma}^2 = \frac{N}{\sum_{i=1}^{N}\frac{1}{\sigma_{i}^{2}}} .. math:: :label: fit-unc-unweighted \sigma_{fit} = \sqrt{ \frac{\sum^{N}_{i=1} e^{2}_{i}} {N-2}} .. math:: :label: fit-unc-weighted \sigma_{fit} = \sqrt{ \frac{ \frac{1}{N-2} \sum^{N}_{i=1} \frac{1}{\sigma_{i}^{2}} e_{i}^{2} } { \frac{1}{N} \sum^{N}_{i=1} \frac{1}{\sigma_{i}^{2}} }} The calculation of :math:`\sigma_{fit}^2` requires calculating the residual error, :math:`e_{i}`, for each datapoint with respect to the fit, as defined by :eq:`fit-error`. .. math:: :label: fit-error e_{i} = k_{norm_i} - k_{fit}(x_{i}) .. note:: The calculation of the linear fit--related parameters are necessary for the other trending methods, ``USL1`` and ``USL2``, as well. These methods only use the unweighted expressions. .. _sec-vader.usl1_method: ``USL1`` -- Confidence Band with Administrative Margin ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The ``USL1`` method uses the confidence band with administrative margin method to calculate the USL. The bias is calculated according to :eq:`bias-fit` using the unweighted method described in :ref:`sec-vader.cr6698_method`, and the bias uncertainty is calculated using :eq:`usl1-bias-unc`. The calculated bias uncertainty for a given x-value depends on whether it is inside or outside the input ``xrange``. For values inside ``xrange``, the bias uncertainty evaluated at :math:`x_{min}` and :math:`x_{max}`, and the maximum is returned. This is represented by :eq:`usl1-bias-unc-const`. Outside of `xrange`, :math:`\sigma_{\beta}(x)` is returned, unless either :math:`\sigma_{\beta}(x_{min})` or :math:`\sigma_{\beta}(x_{max})` is higher, in which case the larger value will be returned. .. math:: :label: usl1-bias-unc \sigma_{\beta}(x) = t_{1-\alpha,N-2} S_p \sqrt{1 + \frac{1}{N} + \frac{(x-\bar{x})^2}{S_{xx}}} In this equation, :math:`S_p`, :math:`\bar{x}`, and :math:`S_{xx}` correspond to unweighted calculations in :eq:`fit-sp`, :eq:`xbar-unweighted`, and :eq:`sxx-unweighted`, respectively. The value :math:`t_{1-\alpha,N-2}` is the inverse t-distribution corresponding to the desired confidence with :math:`N-2` degrees of freedom. .. math:: :label: usl1-bias-unc-const \sigma_{\beta} = \max(\sigma_{\beta}(x_{min}),\sigma_{\beta}(x_{max})) .. _sec-vader.usl2_method: ``USL2`` -- Single-Sided Uniform Width Closed Interval Approach ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The ``USL2`` method uses the single-sided, uniform-width, closed-interval approach to calculate the USL. The bias is calculated according to :eq:`bias-fit` using the unweighted method described in :ref:`sec-vader.cr6698_method`, and the bias uncertainty for this method is calculated using :eq:`usl2-bias-unc`. .. math:: :label: usl2-bias-unc \sigma_{\beta} = C_{\alpha/p} \, S_p Here, the square root of the pooled variance, :math:`S_p`, is as calculated per :eq:`fit-sp` using unweighted methods, and the parameter :math:`C_{\alpha/p}` is defined in :eq:`usl2-cap`. .. math:: :label: usl2-cap C_{\alpha/p} = C^{*} + t_{P,N-2} \sqrt{ \frac{N-2} {\chi_{\alpha,N-2}^2} } In this equation, :math:`C^{*}`, as defined in :eq:`usl2-cstar`, depends on a series of calculations, outlined in :eq:`usl2-a` for :math:`A`, :eq:`usl2-g` for :math:`g`, :eq:`usl2-h` for :math:`h`, and :eq:`usl2-d` for :math:`D`. .. math:: :label: usl2-cstar C^{*} = \begin{cases} D \, g & \text{ if } 0.5 \leq A \leq 1.5, \\ D \, h & \text{ for other values of } A \end{cases} .. math:: :label: usl2-a A=\frac{g}{h} .. math:: :label: usl2-g g = \sqrt{\frac{1}{N} + \frac{(x_{min}-\bar{x})}{S_{xx}}} .. math:: :label: usl2-h h = \sqrt{\frac{1}{N} + \frac{(x_{max}-\bar{x})^2}{S_{xx}}} .. math:: :label: usl2-d \int_{-D}^{D} \int_{-AD}^{AD} \frac{1}{2 \pi \sqrt{ 1 - \rho^2 }} \left[ 1 + \frac{u^2 - 2 \, \rho \, u \, v + v^2 } {(N-2) (1-\rho^2)} \right] ^{\frac{-N}{2}} \, du \, dv \leq \alpha The parameter :math:`D` is calculated iteratively to the desired level of confidence. The parameter :math:`\rho` is defined in :eq:`usl2-rho`, using the unweighted calculations referenced above. .. math:: :label: usl2-rho \rho = \frac{1}{g \, h} \left[ \frac{1}{N} + \frac{(x_{min}-\bar{x})(x_{max}-\bar{x})}{S_{xx}} \right] .. _sec-vader.parm_method: ``parametric`` -- Historical Parametric Method ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The historical parametric method is suitable for data which has no statistically significant trend that is appropriately or conservatively represented by a normal distribution. This method uses the lower tolerance limit (LTL) approach to develop a bias and bias uncertainty. The bias and bias uncertainty are calculated by :eq:`nontrending-bias` and :eq:`parametric-bias-unc`, respectively. .. math:: :label: nontrending-bias \beta = \bar{k}_{norm}-1 .. math:: :label: parametric-bias-unc \sigma_{\beta} = KS_p In :eq:`nontrending-bias`, :math:`\bar{k}_{norm}` is the either the unweighted or uncertainty-weighted average of :math:`k_{norm}` in a dataset calculated by :eq:`kbar-unweighted` or :eq:`kbar-weighted`, respectively. The parameters required to calculate the bias uncertainty per :eq:`parametric-bias-unc` are the single-sided lower tolerance factor, :math:`K`, and the square root of the pooled variance, :math:`S_p`. The square root of the pooled variance calculation is detailed in :eq:`parametric-sp`, and it comprises the variance associated with the statistical scatter of datapoints about the mean, :math:`\sigma^2_{\bar{k}_{norm}}`, and the average total variance associated with the individual :math:`k_{norm}` values, :math:`\bar{\sigma}^2`. .. math:: :label: parametric-sp S_p = \sqrt{\sigma^2_{\bar{k}_{norm}} + \bar{\sigma}^2} The equations for the variance associated with scatter about the mean are provided for the unweighted method in :eq:`kbar-var-unweighted` and weighted method in :eq:`kbar-var-weighted`. .. math:: :label: kbar-var-unweighted \sigma^2_{\bar{k}_{norm}} = \frac{\sum_{i=1}^{N}(k_{norm_i} - \bar{k}_{norm})^2}{N - 1} .. math:: :label: kbar-var-weighted \sigma^2_{\bar{k}_{norm}} = \frac{ \frac{1}{N-1} \sum_{i=1}^{N} \frac{1}{\sigma_i^2} (k_{norm_i} - \bar{k}_{norm})^2} {\frac{1}{N} \sum_{i=1}^{N} \frac{1}{\sigma_i^2}} The "within variance" or "average total variance," which will be referred to as the average total variance hereinafter, described by :eq:`avg-tot-var-unweighted` for the unweighted method and by :eq:`avg-tot-var-weighted` for the weighted method, combines the uncertainties associated with the input dataset. The appropriate implementation of the single-sided lower tolerance factor, :math:`K`, using :eq:`parametric-k`, relies on the assumption of a normal distribution being applicable or conservative. .. math:: :label: parametric-k K = \frac{t_{\gamma,N-1,\delta}}{\sqrt{N}} In this equation, :math:`t_{\gamma,N-1,\delta}` is the inverse of the cumulative noncentral t-distribution corresponding to the desired confidence level, :math:`\gamma`, with a non-centrality parameter, :math:`\delta`, calculated per :eq:`parametric-delta`, using :math:`z_p`, the z-score corresponding to the desired proportion of the population, :math:`p`. .. math:: :label: parametric-delta \delta = z_p \sqrt{N} .. _sec-vader.nonparm_method: ``nonparametric`` -- Historical Non-Parametric Method ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The historical non-parametric method is suitable for calculating bias and bias uncertainty without relying on statistical methods applicable only for normal distributions; no assumptions are made regarding the shape of the distribution for the dataset. For this method, the bias is calculated in the same manner as that of the parametric method, per :eq:`nontrending-bias`, with :math:`\bar{k}_{norm}` calculated using either :eq:`kbar-unweighted` for the unweighted method or :eq:`kbar-weighted` for the weighted method. The bias uncertainty is calculated per :eq:`nonparametric-bias-unc` below. This method requires identifying :math:`k_{norm_r}` and :math:`\sigma_{k_{norm_r}}` corresponding to the lower rank value for the desired confidence level as well as calculating the applicable nonparametric margin, :math:`NPM`. .. math:: :label: nonparametric-bias-unc \sigma_{\beta} = \begin{cases} \bar{k}_{norm} - k_{norm_r} + \sigma_{k_{norm_r}} + NPM & \text{if } \bar{k}_{norm} \leq 1, \\ 1 - k_{norm_r} + \sigma_{k_{norm_r}} + NPM & \text{if } \bar{k}_{norm} > 1, \\ \end{cases} The :math:`NPM` is applied (:math:`NPM > 0`)for cases with less than the required number of points to justify a lower rank of one, determined by the degree of confidence calculated by :eq:`nonparametric-c`. .. math:: :label: nonparametric-c C = 1 - \sum_{j=0}^{m-1} \frac{N!}{j!(N-j)!}(1-q)^j q^{N-j} In this equation, :math:`q` is the desired proportion of the population (typically 0.95), :math:`N` is the number of points in the dataset, and :math:`m` is the rank of the data point under consideration, where the lowest :math:`k_{norm_i}` value corresponds to a rank of 1 (:math:`m=1`), the second-lowest corresponding to a rank of 2 (:math:`m=2`), and the highest value corresponds to rank :math:`N` (:math:`m=N`). If the sample size is insufficient for the desired confidence level at the lowest rank (:math:`m=1`)---for example, having less than 59 points for a desired confidence level of 95% using the lowest rank---then a nonparametric margin would be necessary to justify the lack of statistical confidence. Historically, :math:`NPM` values based on the calculated degree of confidence for 95% of the population are as follows: .. table:: NPM required given calculated confidence :align: center :name: vader.npm_tables =================================================== ==================== **Degree of Confidence for 95% of the population** **NPM** --------------------------------------------------- -------------------- :math:`>` 90% 0.00 --------------------------------------------------- -------------------- :math:`>` 80% 0.01 --------------------------------------------------- -------------------- :math:`>` 70% 0.02 --------------------------------------------------- -------------------- :math:`>` 60% 0.03 --------------------------------------------------- -------------------- :math:`>` 50% 0.04 --------------------------------------------------- -------------------- :math:`>` 40% 0.05 --------------------------------------------------- -------------------- :math:`\leq` 40% Need additional data =================================================== ==================== .. _sec-vader.available_tests: Available Tests --------------- The sections below briefly describe the available tests. Two tests for normality are included: ``chisq``, the :math:`\chi^2` normality test, and ``anderson_darling``, the Anderson--Darling normality test using the modified test statistic, a function of the sample size. One trend significance test, ``t-test``, is available as well. The t-test for trend significance is the recommended first step for evaluating a data set, as the existence of a significant linear trend necessitates the use of trending methods for calculating bias, bias uncertainty, and the corresponding USL. For data with no significant trend, the normality tests can be used to assess which nontrending method is suitable. For normal distributions, this is the ``parametric`` method; otherwise, the ``nonparametric`` method is suitable. .. _sec-vader.trendt_test: ``t-test`` -- Trend Significance t-test ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Performs the t-test for trend significance for datasets with at least 5 points using either uncertainty-weighted or unweighted methods to determine whether a significant trend exists. The method calculates :math:`t_{fit}` per :eq:`t_fit` using parameters calculated with either the weighted or unweighted method outlined in :ref:`sec-vader.cr6698_method`. .. math:: :label: t_fit t_{fit} = \frac{|\beta_1|}{ \frac{ \sigma_{fit} }{ \sqrt{S_{xx}} } } The calculated test statistic is compared to the :math:`t`-statistic, :math:`t_{n-2,\frac{\alpha}{2}}`, with :math:`n-2` degrees of freedom and a confidence level of :math:`1-\alpha`. If :math:`t_{fit} > t_{n-2,\frac{\alpha}{2}}`, then the null hypothesis is rejected, and it is concluded that :math:`\beta_1 \neq 0`. If :math:`t_{fit} \leq t_{n-2,\frac{\alpha}{2}}`, the null hypothesis is not rejected, and it is concluded that :math:`\beta_1 = 0`. .. _sec-vader.chisq_test: ``chisq`` -- :math:`\chi^2` Normality Test ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Performs the :math:`\chi^2` goodness-of-fit test for normality per the description in "Testing for Normality" by Thode :cite:`VADER-thode-normality-2002` for datasets with at least 5 points. The test returns the :math:`\chi^2` test statistic for the number of bins defined by the ``num_bins``, which is compared to the reference :math:`\chi^2` statistic corresponding to the input confidence in ``confidence``. Setting ``num_bins`` appropriately directly influences the accuracy of the test, so the user is advised to research the test further to better understand the implications and conditions for a poorly defined number of bins. .. _sec-vader.anderson_darling_test: ``anderson_darling`` -- Anderson--Darling Normality Test ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Performs the Anderson--Darling goodness-of-fit test; this calculates the test statistics :math:`A^{2}` and :math:`A^{*2}`, which are used to generate the corresponding p-value. The set confidence level is then assessed with respect to the calculated p-value to either accept or reject the null hypothesis. The implementation of this method uses the modified statistic described in "Goodness-of-Fit Techniques" by D'Agostino and Stephens :cite:`VADER-dagostino-goodness-of-fit-1986`, where the test statistic :math:`A^{2}` is modified from :eq:`a_squared` as a function of sample size (:math:`n`) per :eq:`astarsq`. .. math:: :label: a_squared A^{2} = -n - \frac{1}{n} \sum^{n}_{i=1} \left[ (2i -1) * \log(Z_{(i)}) + (2n+1-2i) * \log(1-Z_{(i)}) \right] In :eq:`a_squared`, the :math:`log` functions are base :math:`e`, and the values :math:`Z_{(i)}` are the corresponding cumulative probability for each datapoint :math:`x_i` arranged in ascending order, calculated using the sample mean in :eq:`sample_mean` and sample uncertainty in :eq:`sample_uncertainty`. .. math:: :label: sample_mean \bar{x} = \frac{\sum_{i=1}^{n}}{n} .. math:: :label: sample_uncertainty \sigma = \sqrt{\frac{\sum_{i=1}^{n}(x_i -\bar{x})^2}{n-1}} .. math:: :label: astarsq A^{*2} = A^{2} (1.0 + \frac{0.75}{n} + \frac{2.25}{n^{2}}) .. _sec-vader.available_parameters: Available Parameters -------------------- :numref:`vader.parameters_table` shows all of the available parameters, their default value, and the methods and tests to which they apply. Any parameter can be put into the global ``parameters`` :ref:`block `. Only parameters that apply to a specific method or test can be put into that method or tests ``parameters`` block. Following the table is a more detailed description of each parameter. .. raw:: latex \clearpage .. table:: Available Parameters for VADER :align: center :name: vader.parameters_table ================================================ ================= ===================== **Keyword** **Default Value** **Available In** ------------------------------------------------ ----------------- --------------------- :term:`tsunami` No ``USL1`` ``USL2`` ------------------------------------------------ ----------------- --------------------- :term:`extrapolate` Yes ``USL1`` ------------------------------------------------ ----------------- --------------------- :term:`fit_confidence` 0.95 ``USL1`` ``USL2`` ------------------------------------------------ ----------------- --------------------- :term:`admin_margin` 0.05 ``USL1`` ``USL2`` ``CR6698`` ``nonparametric`` ``parametric`` ------------------------------------------------ ----------------- --------------------- :term:`xrange` All X Values ``USL1`` ``USL2`` ``CR6698`` ``nonparametric`` ``parametric`` ------------------------------------------------ ----------------- --------------------- :term:`proportion_confidence` 0.95 ``USL2`` ------------------------------------------------ ----------------- --------------------- :term:`proportion` 0.95 ``USL2`` ------------------------------------------------ ----------------- --------------------- :term:`f_p` 0.95 ``CR6698`` ------------------------------------------------ ----------------- --------------------- :term:`normal_p` 0.95 ``CR6698`` ------------------------------------------------ ----------------- --------------------- :term:`chi_p` 0.95 ``CR6698`` ------------------------------------------------ ----------------- --------------------- :term:`weighted` No ``CR6698`` ``nonparametric`` ``parametric`` ``t-test`` ------------------------------------------------ ----------------- --------------------- :term:`confidence` 0.95 ``nonparametric`` ``parametric`` ``chisq`` ``t-test`` ``anderson_darling`` ------------------------------------------------ ----------------- --------------------- :term:`proportion_of_pop` 0.95 ``nonparametric`` ``parametric`` ------------------------------------------------ ----------------- --------------------- :term:`use_npm` Yes ``nonparametric`` ------------------------------------------------ ----------------- --------------------- :term:`num_bins` 5 ``chisq`` ================================================ ================= ===================== .. glossary:: ``tsunami`` This enables a TSUNAMI-mode calculation for VADER, requiring that all *X* values be between -1.0 and 1.0 for the method calculation. This also minimally tailors the output printing for the given constraints on the data. ``extrapolate`` This allows extrapolation past the data values entered by the user. This is generally only applicable when the user has input values in the :ref:`trend_values ` block. ``fit_confidence`` Confidence in the statistical result. Sometimes referred to as :math:`1 - \gamma`. In USL1 this is used to calculated the t-statistic quantile. ``admin_margin`` Administrative margin subtracted from the result. This can be used to add an extra layer of confidence that the value reported is below criticality. ``xrange`` Allows the user to tailor *X* values for each method or test or change the values for all tests without having to change the `data` :ref:`block `. Data is entered as an array of two values. By default all of the *X* values are used. ``proportion_confidence`` In USL2 this controls the confidence on the proportion that is in the confidence band. ``proportion`` In USL2 this controls the proportion of the data points that will be inside the confidence band. ``f_p`` The percentile used for the F-distribution in the single-sided lower tolerance band method. ``normal_p`` The percentile used for the normal distribution in the single-sided lower tolerance band method. ``chi_p`` The percentile used for the :math:`\chi^2` distribution in the single-sided lower tolerance band method. ``weighted`` Whether to use the uncertainty-weighted calculations. ``confidence`` Statistical confidence that the proportion calculated to be above the cutoff is correct. ``proportion_of_pop`` Proportion of the population that will be above the cutoff value in the non-parametric method. ``use_npm`` Use (add) non-parametric margin to the non-parametric method. ``num_bins`` Number of bins to use for the calculation of the :math:`\chi^2` test statistic. Example Trending Bias Assessment -------------------------------- This section illustrates the bias assessment workflow using VADER. Consider the following VADER input for the U-233 critical experiments and their corresponding energy of the average lethargy of neutrons causing fission (EALF) data sourced from Example 1 in the Appendix of NUREG/CR-7311 :cite:`VADER-clarity-nureg-bias`. Note that in this example, the EALF is not expected to be linear, but the natural logarithm of the EALF is expected to be linear. Thus, ln(EALF) is used for the trending parameter *X*, which corresponds to the U-233 C/E *Y* values. It is assumed in this input that a trending analysis will be applicable for this data set, but that assumption should be confirmed by evaluating trend significance. In this example, two trending methods are used: the *CR6698* method and the *USL1* method. One test is used: the t-test for trend significance. Three ln(EALF) trend values of interest are also specified: -2.996, 0.0, and 3.912, which correspond to EALF values of 0.01, 1.0, and 50.0 eV. Default parameters are used for each method and test. .. code-block:: scale :name: VADER_trending_example :caption: VADER Trending Data Example Input =vader title="U-233 Critical Experiments" data = [ ' Example 1 Data - U-233 Critical Experiments ' ln(EALF) C/E uncertainty of C/E 0.43178241643 1.00082 0.00250 -0.24590053844 1.00227 0.00241 -0.76787072676 1.00058 0.00250 1.92570744174 0.98522 0.00818 2.08069076108 0.98053 0.00834 2.15408508468 0.98126 0.00648 1.31372366829 0.99217 0.00605 2.22137503757 0.98462 0.00807 1.45628673294 0.98553 0.00601 2.26695791535 0.98174 0.00579 1.51512723296 0.98015 0.00549 1.99470031322 0.97978 0.00666 2.31253542385 0.97862 0.00519 2.04898233420 0.98018 0.00559 1.49514876603 0.98123 0.00893 1.62727783056 0.98207 0.00697 1.69927861643 0.98000 0.00735 0.93609335917 0.98848 0.00544 1.76644166124 0.97843 0.00558 1.80828877118 0.97523 0.00810 1.10194007876 0.97965 0.00549 1.85002837735 0.97295 0.00487 1.87333945622 0.97805 0.00479 1.55180879960 0.99000 0.00465 0.68813463874 0.99236 0.00804 0.82417544297 0.98533 0.00798 0.87546873735 0.98907 0.00643 0.94000725849 0.98357 0.00600 0.97832612279 0.97727 0.00958 0.99325177301 0.99085 0.00704 1.03318448335 0.97571 0.00517 0.73236789371 0.99382 0.00457 0.83290912294 0.98998 0.00515 0.59332684528 0.97444 0.00273 0.37843643572 0.97763 0.00518 0.26236426447 0.98645 0.00671 0.31481073984 0.98620 0.00543 0.35065687161 0.97657 0.00967 0.38526240079 0.97311 0.00652 0.40546510811 0.96863 0.00484 -3.23398946268 1.00141 0.00311 -3.21887582487 1.00067 0.00330 -3.20398721237 1.00012 0.00330 -3.18931702263 1.00103 0.00331 -3.17485893945 1.00025 0.00330 -1.74869997977 1.00187 0.00868 -2.00991547903 0.98990 0.00858 -2.25379492882 1.00592 0.00872 -2.46510402249 1.00252 0.00869 -2.60233974668 1.00738 0.00873 -2.71810053696 0.99400 0.00861 -2.77258872224 0.98375 0.00853 -2.85597033118 0.99783 0.00865 -2.95843648881 0.98647 0.00855 -2.99373427089 0.99937 0.00866 -3.06401111431 1.00803 0.00874 -1.29828348380 0.98872 0.00857 -0.71743987313 0.98852 0.00857 -1.97328134585 0.99754 0.00864 -2.34758245892 1.00412 0.00870 -2.91139112512 1.00640 0.00872 -1.15518264016 1.00214 0.00873 -1.04982212450 1.01754 0.01541 -1.09064411902 0.99854 0.00869 -0.23825718912 1.00206 0.01259 0.05826890812 1.00933 0.01233 -2.04022082853 1.02084 0.00888 -2.48051630149 1.01405 0.00882 -2.67220054821 1.01003 0.00878 -2.78062089394 1.00984 0.00878 -3.07477548089 1.00751 0.00877 -1.76026080217 0.99887 0.00876 -2.01740615076 1.00241 0.00859 -1.29828348380 0.99177 0.00879 -0.71539278951 0.98154 0.00869 -0.95191790952 0.98867 0.00887 -0.72154665508 1.00145 0.01049 -0.95972028980 0.99817 0.01036 -1.98050159382 1.00466 0.01023 -2.78223509929 1.00160 0.00401 -2.90955457731 1.00456 0.00492 -3.29144651770 1.00089 0.00290 -3.27280416689 0.99933 0.00441 -3.28341434601 1.00118 0.00401 -3.29413830937 1.00163 0.00381 -3.30497852392 0.99981 0.00380 0.22314355131 0.99022 0.00505 -1.75446368448 1.00010 0.00280 -1.79576749063 1.00004 0.00250 -1.91054300522 1.00998 0.00233 -2.22562405186 1.00247 0.00151 -2.39250980052 1.00412 0.00713 -2.51825662947 1.00523 0.00111 -2.91323105204 1.00126 0.00381 -2.91692109313 0.99854 0.00479 -1.85150947363 1.00548 0.00735 -1.85150947363 1.00557 0.00705 -1.84516024596 1.00562 0.00695 -1.83885107676 1.00595 0.00735 -1.83258146375 1.00653 0.00675 -1.89047544217 1.00647 0.00504 -1.88387475814 1.00629 0.00544 -1.88387475814 1.00681 0.00504 -1.87731735759 1.00707 0.00454 -1.87731735759 1.00769 0.00464 -1.89711998489 1.00529 0.00543 -1.88387475814 1.00621 0.00504 -1.89047544217 1.00372 0.00623 -1.87731735759 1.00642 0.00514 -2.25379492882 1.02100 0.00787 -2.42136862898 0.99338 0.00686 -2.44992568089 0.99596 0.00518 -2.51949809456 1.00020 0.00200 -2.51701670371 0.99634 0.00887 -2.76303450943 0.99820 0.00559 -2.86998106825 1.00238 0.00341 0.09531017980 0.98999 0.00743 0.21511137962 0.98558 0.00690 -0.32711614170 0.98933 0.00406 -0.22941316433 0.98653 0.00691 0.12221763272 0.98994 0.00505 -0.36672527979 0.99315 0.00745 -0.26526847761 0.99395 0.00686 -0.21443161071 0.99177 0.00684 -0.76356964486 0.99644 0.00359 -0.16605458433 0.98968 0.00594 -0.14156356432 0.98839 0.00425 -0.68319684971 0.99653 0.00289 -0.10758521068 0.97430 0.00546 -0.09211528891 0.97484 0.00507 -1.24132859087 0.99483 0.00786 -1.15518264016 0.99794 0.00699 -1.10866262452 0.99612 0.00618 -1.07002483182 0.99407 0.00547 -1.04696905552 0.99084 0.00505 -1.49165487678 0.99585 0.00229 -2.06356819252 0.99396 0.00656 -2.03255795578 0.99871 0.00629 -2.01740615076 0.99682 0.00578 -2.00248050054 0.99532 0.00508 -1.98777435315 0.99465 0.00478 -1.98050159382 0.99381 0.00547 -1.23443201181 1.00439 0.00372 -1.23100147671 1.00577 0.00443 -1.23100147671 1.00458 0.00362 -1.23443201181 1.00600 0.00353 -1.23443201181 0.99620 0.00339 -1.23443201181 0.99530 0.00338 -1.23443201181 0.99461 0.00278 -1.23443201181 0.99569 0.00269 -1.24479479885 1.00389 0.00301 -1.24479479885 1.00456 0.00412 -1.24479479885 1.00485 0.00473 -1.93102153656 1.00505 0.00362 -1.93102153656 1.00482 0.00262 -1.93102153656 1.00582 0.00272 -1.92414865727 1.00965 0.00313 -1.91732269220 0.99429 0.00279 -1.92414865727 0.99588 0.00359 -1.92414865727 0.99486 0.00348 -1.93794197941 1.00902 0.00283 -1.93794197941 1.00882 0.00343 -1.93794197941 1.00895 0.00313 -1.93794197941 1.00839 0.00242 -2.50592601701 1.00106 0.00401 -2.50592601701 1.00679 0.00343 -2.50715225874 1.00384 0.00372 -2.50838000597 0.99960 0.00371 -2.50838000597 1.00033 0.00311 -2.50838000597 0.99923 0.00320 -2.85771097566 1.01071 0.00344 -2.85771097566 1.01307 0.00325 -2.85771097566 1.01283 0.00396 -2.16282315062 1.00454 0.00322 -2.18925640769 1.00051 0.00250 -2.21640739675 1.00519 0.00352 -2.48650793115 1.00581 0.00403 -2.50715225874 1.00240 0.00291 -2.91139112512 1.00021 0.00290 -2.89317568523 0.99990 0.00370 ] trend_values=[ -2.996 0.0 3.912 ] parameters{extrapolate=no} methods{ CR6698{} USL1{} } tests{ t-test{} } end Before using any results of a method, confirm that the method is suitable by investigating the results of the applicable test. In this case, the t-test evaluates the null hypothesis that no trend exists, and the output for this example states "Null Hypothesis Rejected," meaning there is a significant trend in the data. Based on this result, it is appropriate to proceed to the trending methods' output. The following output should be seen for the t-test. .. code-block:: :name: vader_trending_example_test_output :caption: VADER Example 1 Trend Significance Test Output Test (1 of 1): Trend Significance t-test ======================================== Null Hypothesis: Data have a slope of zero (no trend). Test: Null Hypothesis Rejected ------------ Test Results ------------ Test statistic: 1.9734 t_fit value (null hypothesis rejected if above test statistic): 14.419 From this output, it can be confirmed that the null hypothesis was rejected, meaning there is a statistically significant trend based on this t-test. Because there is a statistically significant trend, it is likely appropriate to proceed with the use of trending methods for the USL calculation. The first method output shown is for the CR6698 single-sided lower tolerance band method. Information on the dataset and parameters is printed at the start of the output. Notably, a default administrative margin of 0.05 is applied across all methods and is incorporated in the USL calculation. For this example, extrapolation is disabled. .. code-block:: :name: vader_trending_example_cr6698_output :caption: VADER Trending Example CR6698 Method Output Method (1 of 2): CR6698 ======================= Trend Name: Single-Sided Lower Tolerance Band (from NUREG/CR-6698) Number of data points: 180 Data Set Minimum X value: -3.305 Data Set Maximum X value: 2.3125 Data Set unweighted X average: -1.1994 Data Set unweighted Y average: 0.99661 ----------------- Method Parameters ----------------- F-Distribution Percentile: 0.95 Symmetric percentile for normal distribution: 0.95 Chi-square percentile: 0.95 Administrative Margin: 0.05 Minimum X Value: -3.30498(DATA_MIN) Maximum X Value: 2.31254(DATA_MAX) -------------- Method Results -------------- Pooled standard deviation: 0.00945737 S_xx: 425.815 Fit Coeff 0 (Intercept): 0.99064 Fit Coeff 1 (Slope): -0.00497895 k = 1.0 intercept: -1.87998 ------------ Trend Values ------------ Equation: Not Implemented Trend Value k(x) Adj. k(x) Bias Unc.(x) Adm. Margin USL -3.30498 1.00709 1.00000 0.02031 0.05000 0.92969 -2.99600 1.00556 1.00000 0.02003 0.05000 0.92997 -2.50248 1.00310 1.00000 0.01964 0.05000 0.93036 -1.87998 1.00000 1.00000 0.01926 0.05000 0.93074 -1.69997 0.99910 0.99910 0.01919 0.05000 0.92992 -0.89747 0.99511 0.99511 0.01913 0.05000 0.92598 -0.09497 0.99111 0.99111 0.01950 0.05000 0.92161 0.00000 0.99064 0.99064 0.01957 0.05000 0.92107 0.70753 0.98712 0.98712 0.02013 0.05000 0.91699 1.51003 0.98312 0.98312 0.02088 0.05000 0.91224 2.31254 0.97913 0.97913 0.02170 0.05000 0.90743 3.91200 0.97116 0.97116 0.02340 0.05000 0.89776 NOTE: N/A is printed in the table if extrapolation is off and a value outside of the range was desired. The "Method Parameters" section should be used to verify expectations for the data set and parameters before proceeding to the results. In the "Method Results" section, the output data includes the pooled standard deviation (:math:`S_p`), the sum of squared deviations from the mean for *X* values (:math:`S_{xx}`), the linear fit intercept (:math:`\beta_0`), the linear fit slope (:math:`\beta_1`), and the fit trend value, where :math:`k=1`. The "Trend Values" output section will produce the USL as a function of trend values spanning the data *X* values, and the values specified explicitly in *trend_values* are included in the output table as well. For each trend value, the following output values are printed: the trended k-eff :math:`k(x)` based on the linear regression; the adjusted :math:`k(x)`, which caps at 1.0; the method-specific bias uncertainty of the trend :math:`\sigma(x)`; administrative margin; and USL. Note that extrapolation has not been enabled globally; here, the extrapolate parameter applies only for the USL1 method. Therefore, it is expected to see extrapolated trend values in the output of the CR6698 method but not in the USL1 method. In the USL1 method, any trend value outside of the range of the *X* values in the data set will return "N/A." The output of the USL-1 method is included below. .. code-block:: :name: vader_trending_example_usl1_output :caption: VADER Tending Example USL-1 Method Output Method (2 of 2): USL1 ===================== Trend Name: USL Method 1 (Confidence Band with Administrative Margin) Number of data points: 180 Data Set Minimum X value: -3.305 Data Set Maximum X value: 2.3125 Data Set unweighted X average: -1.1994 Data Set unweighted Y average: 0.99661 ----------------- Method Parameters ----------------- TSUNAMI-Mode: No Extrapolation On: No Confidence [%]: 0.95 Administrative Margin: 0.05 Minimum X Value: -3.30498(DATA_MIN) Maximum X Value: 2.31254(DATA_MAX) -------------- Method Results -------------- Pooled std. dev.: 0.00945737 Within variance: 3.86665e-05 Variance of fit: 5.07754e-05 W: 0.015905 S_xx: 425.815 Fit Coeff. 0 (Intercept): 0.99064 Fit Coeff. 1 (Slope):: -0.00497895 ------------ Trend Values ------------ Equation: 0.924735 + (-0.004979)*X (X > -1.879985) 0.934095 (X <= -1.879985) Trend Value k(x) Adj. k(x) Bias Unc.(x) Adm. Margin USL -3.30498 1.00709 1.00000 0.01590 0.05000 0.93410 -2.99600 1.00556 1.00000 0.01590 0.05000 0.93410 -2.50248 1.00310 1.00000 0.01590 0.05000 0.93410 -1.87998 1.00000 1.00000 0.01590 0.05000 0.93410 -1.69997 0.99910 0.99910 0.01590 0.05000 0.93320 -0.89747 0.99511 0.99511 0.01590 0.05000 0.92920 -0.09497 0.99111 0.99111 0.01590 0.05000 0.92521 0.00000 0.99064 0.99064 0.01590 0.05000 0.92473 0.70753 0.98712 0.98712 0.01590 0.05000 0.92121 1.51003 0.98312 0.98312 0.01590 0.05000 0.91722 2.31254 0.97913 0.97913 0.01590 0.05000 0.91322 3.91200 N/A N/A N/A N/A N/A NOTE: N/A is printed in the table if extrapolation is off and a value outside of the range was desired. The "Method Parameters" output section confirms that the extrapolation parameter is both part of the method and is disabled. This is reflected in the "Trend Values" output section, which prints "N/A" for trend values at 50 eV (ln(EALF) = 3.912), which is a value outside of the *X* value range, as indicated by the minimum and maximum *X* values printed near the start of the method output. In the USL-1 method, the bias uncertainty as a function of *x* is constant across the values in-range. This is a conservatism in the USL-1 methodology that uses the maximum of :math:`\sigma_{\beta}(x)` evaluated over the *X* range. When extrapolation is enabled, the trended value for bias uncertainty is used outside of the *X* range. This would correspond to a bias uncertainty of 0.01615 at the trend value corresponding to 50 eV. .. only:: html .. rubric:: References .. bibliography:: zSCALE.bib :cited: :keyprefix: VADER- :labelprefix: VADER-