Conditional Inference: Towards a Hierarchy of Statistical Evidence

04/09/2021
by   Ying Jin, et al.
0

Statistical uncertainty has many sources. P-values and confidence intervals usually quantify the overall uncertainty, which may include variation due to sampling and uncertainty due to measurement error, among others. Practitioners might be interested in quantifying only one source of uncertainty. For example, one might be interested in the uncertainty of a regression coefficient of a fixed set of subjects, which corresponds to quantifying the uncertainty due to measurement error and ignoring the variation induced by sampling. In causal inference it is common to infer treatment effects for a certain set of subjects, only accounting for uncertainty due to random treatment assignment. Motivated by these examples, we consider conditional estimation and conditional inference for parameters in parametric and semi-parametric models, where we condition on observed characteristics of a population. We derive a theory of conditional inference, including methods to obtain conditionally valid p-values and confidence intervals. Conditional p- values can be used to construct a hierarchy of statistical evidence that may help clarify the generalizability of a statistical finding. We show that a naive method allows to gauge the generalizability of a finding, with rigorous control of the family-wise error rate. In addition, the proposed approach allows to conduct transfer learning of conditional parameters, with rigorous conditional guarantees. The performance of the proposed approach is evaluated on simulated and real-world data.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/22/2019

Regression Analysis of Unmeasured Confounding

When studying the causal effect of x on y, researchers may conduct regre...
research
07/21/2020

Quantifying Performance Changes with Effect Size Confidence Intervals

Measuring performance quantifying a performance change are core eval...
research
05/02/2019

Sparsity Double Robust Inference of Average Treatment Effects

Many popular methods for building confidence intervals on causal effects...
research
11/02/2019

Correcting for attenuation due to measurement error

I present a frequentist method for quantifying uncertainty when correcti...
research
09/08/2023

Confidence in Causal Inference under Structure Uncertainty in Linear Causal Models with Equal Variances

Inferring the effect of interventions within complex systems is a fundam...
research
05/07/2021

The r-value: evaluating stability with respect to distributional shifts

Common statistical measures of uncertainty like p-values and confidence ...
research
05/18/2018

Method G: Uncertainty Quantification for Distributed Data Problems using Generalized Fiducial Inference

It is not unusual for a data analyst to encounter data sets distributed ...

Please sign up or login with your details

Forgot password? Click here to reset