We consider the classical problem of heteroscedastic linear regression, where
we are given $n$ samples $(\mathbf{x}_i, y_i) \in \mathbb{R}^d \times
\mathbb{R}$ obtained from $y_i = \langle \mathbf{w}^{*}, \mathbf{x}_i \rangle +
\epsilon_i \cdot \langle \mathbf{f}^{*}, \mathbf{x}_i \rangle$, where
$\mathbf{x}_i \sim N(0,\mathbf{I})$, $\epsilon_i \sim N(0,1)$, and our task is
to estimate $\mathbf{w}^{*}$. In addition to the classical applications of
heteroscedastic models in fields such as statistics, econometrics, time series
analysis etc., it is also particularly relevant in machine learning when data
is collected from multiple sources of varying but apriori unknown quality,
e.g., large model training. Our work shows that we can estimate
$\mathbf{w}^{*}$ in squared norm up to an error of
$\tilde{O}\left(\|\mathbf{f}^{*}\|^2 \cdot \left(\frac{1}{n} +
\left(\frac{d}{n}\right)^2\right)\right)$ and prove a matching lower bound (up
to logarithmic factors). Our result substantially improves upon the previous
best known upper bound of $\tilde{O}\left(\|\mathbf{f}^{*}\|^2\cdot
\frac{d}{n}\right)$. Our upper bound result is based on a novel analysis of a
simple, classical heuristic going back to at least Davidian and Carroll (1987)
and constitutes the first non-asymptotic convergence guarantee for this
approach. As a byproduct, our analysis also provides improved rates of
estimation for both linear regression and phase retrieval with multiplicative
noise, which maybe of independent interest. The lower bound result relies on a
careful application of LeCam's two point method, adapted to work with heavy
tailed random variables where the relevant mutual information quantities are
infinite (precluding a direct application of LeCam's method), and could also be
of broader interest.

本文研究异方差线性回归问题，提出一种新的方法估计真实权重向量 w，并对该方法提供了收敛保证及误差估计。