Estimators for Sequential and Simultaneous Nested NPIV

In this section, we analyze the closed-form or approximate solutions under different function classes for the following estimators:

Sequential Nested NPIV:

Given observations \((A_i, B_i, C_i)\) in tr, an initial estimator \(\hat{g}\) which may be estimated in tr, and hyperparameter values \((\lambda, \mu)\), estimate

\[\hat{h} = \arg\min_{h \in \mathcal{H}} \left[ \sup_{f \in \mathcal{F}} \left( 2 \cdot \text{loss}(f, \hat{g}, h) - \text{penalty}(f, \lambda) \right) + \text{penalty}(h, \mu) \right]\]

where \(\text{penalty}(f, \lambda) = \mathbb{E}_m\{f(C)^2\} + \lambda \cdot \|f\|^2_{\mathcal{F}}\) and \(\text{penalty}(h, \mu) = \mu \cdot \|h\|^2_{\mathcal{H}}\).

Sequential Nested NPIV: Ridge:

Given observations \((A_i, B_i, C_i)\) in tr, an initial estimator \(\hat{g}\) which may be estimated in tr, and a hyperparameter \(\mu\), estimate

\[\hat{h} = \arg\min_{h \in \mathcal{H}} \left[ \sup_{f \in \mathcal{F}} \left( 2 \cdot \text{loss}(f, \hat{g}, h) - \text{penalty}(f) \right) + \text{penalty}(h, \mu) \right]\]

where \(\text{penalty}(f) = \mathbb{E}_m\{f(C)^2\}\) and \(\text{penalty}(h, \mu) = \mu \cdot \mathbb{E}_m\{h(B)^2\}\).

Simultaneous Nested NPIV:

Given observations \((A_i, B_i, C_i, C_i')\) in tr, and hyperparameter values \((\mu', \mu)\), estimate

\[\begin{split}(\hat{g}, \hat{h}) = \arg\min_{g \in \mathcal{G}, h \in \mathcal{H}} \left[ \sup_{f' \in \mathcal{F}} \left( 2 \cdot \text{loss}(f', Y, g) - \text{penalty}(f') \right) + \text{penalty}(g, \mu') \right. \\ \left. + \sup_{f \in \mathcal{F}} \left( 2 \cdot \text{loss}(f, g, h) - \text{penalty}(f) \right) + \text{penalty}(h, \mu) \right]\end{split}\]

using analogous \(\text{penalty}\) notation to the Sequential estimators.