Error parsing markdown: Unexpected character `^` (U+005E) before attribute name, expected a character that can start an attribute name, such as a letter, `$`, or `_`; whitespace before attributes; or the end of the tag.
You can fix the errors in source mode and switch to rich text mode when you are ready.

Error parsing markdown: Unexpected character `(` (U+0028) before attribute name, expected a character that can start an attribute name, such as a letter, `$`, or `_`; whitespace before attributes; or the end of the tag.
You can fix the errors in source mode and switch to rich text mode when you are ready.

# Leveraging Unlabeled Data to Predict Out-of-Distribution Performance 

Saurabh Garg*<br />Carnegie Mellon University<br />sgarg2@andrew.cmu.edu Sivaraman Balakrishnan<br />Carnegie Mellon University<br />sbalakri@andrew.cmu.edu Zachary C. Lipton<br />Carnegie Mellon University<br />zlipton@andrew.cmu.edu

Proof of Proposition 1. Consider a binary classification problem. Assume  \mathcal{P}  be the set of possible target conditional distribution of labels given  p_{s}(x, y)  and  p_{t}(x) .
The forward direction is simple. If  \mathcal{P}=\left\{p_{t}(y \mid x)\right\}  is singleton given  p_{s}(x, y)  and  p_{t}(x) , then the error of any classifier  f  on the target domain is identified and is given by
  
\mathcal{E}_{\mathcal{D}^{T}}(f)=\mathbb{E}_{x \sim p_{t}(x), y \sim p_{t}(y \mid x)}\left[\mathbb{I}\left[\underset{j \in \mathcal{Y}}{\arg \max } f_{j}(x) \neq y\right]\right]
  
For the reverse direction assume that given <p_{t}(x)  and  p_{s}(x, y) , we have two possible distributions  \mathcal{D}^{T}  and  \mathcal{D}^{T^{\prime}}  with  p_{t}(y \mid x), p_{t}^{\prime}(y \mid x) \in \mathcal{P}  such that on some  x  with  p_{t}(x)>0 , we have  <p_{t}(y \mid x) \neq p_{t}^{\prime}(y \mid x) . Consider  \mathcal{X}_{M}=\left\{x \in \mathcal{X} \mid p_{t}(x)>0\right.  and  \left.p_{t}(y=1 \mid x) \neq p_{t}^{\prime}(y=1 \mid x)\right\}  be the set of all input covariates where the two distributions differ. We will now choose a classifier  f  such that the error on the two distributions differ. On a subset  \mathcal{X}_{M}^{1}=\left\{x \in \mathcal{X} \mid p_{t}(x)>0\right.  and  \left.p_{t}(y=1 \mid x)>p_{t}^{\prime}(y=1 \mid x)\right\} , assume  f(x)=0  and on a subset  \mathcal{X}_{M}^{2}=\left\{x \in \mathcal{X} \mid p_{t}(x)>0\right.  and  \left. <p_{t}(y=1 \mid x)<p_{t}^{\prime}(y=1 \mid x)\right\} , assume  f(x)=1 . We will show that the error of  f  on distribution with  p_{t}(y \mid x)  is strictly greater than the error of  f  on distribution with  <p_{t}^{\prime}(y \mid x) . Formally