Skip to main content
MathOverflow

Questions tagged [linear-regression]

Ask Question

The tag has no summary.

36 questions
Newest Active Bountied Unanswered
Filter by
Sorted by
Tagged with
1 vote
0 answers
69 views

Let's say we find some regression equation $\ell$ (best fit / linear / whatever words you need to put here) for a sample $D$, subset of population $P$. This equation/model can be thought of as a ...
cheyne's user avatar
  • 1,726
0 votes
1 answer
208 views

$Y_i$ are independent random variables following a normal law of mean $m_i = Ax_i + B$ and variance $V.$ Let's take a sample $y_i \sim Y_i.$ I determine $a$ and $b,$ the weigthed least squares ...
loqueelviento's user avatar
  • 73
0 votes
0 answers
55 views

Let's say we are considering the following model: $$ (\beta^{\star},f^{\star}) := \arg\min_{\beta,f \in \mathcal{F}} \mathbb{E}[\left(Z_i - f(X_i, E_i) - \beta^\top \boldsymbol{\tau}_{i,E_i}\right)^2|...
Frédéric Chopin's user avatar
  • 1
1 vote
0 answers
69 views

Imagine we are interested in the following problem: $$ \min_{w} \left( w^T V w + \lambda \|w\| \right) \\ \text{s.t. } w^T R \geq c $$ Where 𝑤 is an $N \times 1$ vector, $V$ is an $N \times N$ ...
Azmy Rajab's user avatar
  • 111
0 votes
1 answer
165 views

I am reading a scientific article in which matrices are handled (which I do not use often). We consider a matrix $X\in\mathbb R^{n\times p}$ and a vector $y\in\mathbb R^n$. The authors show that the ...
Zach's user avatar
  • 101
0 votes
0 answers
135 views

In chemical analysis, the instrument's signal are plotted as a function of chemical concentration. In general, higher the concentration higher is the response and the relationship is linear. At ...
ACR's user avatar
  • 923
2 votes
1 answer
165 views

$\DeclareMathOperator\Cov{Cov}$Backround of my Question Let $Y$ be the response variable, $\mathbb{X}$ be the explanatory variables. The ultimate goal of prediction is finding a function $f^{*}$ that ...
Cheng-Yu's user avatar
  • 23
1 vote
0 answers
90 views

I'm looking for algorithms to solve a special quadratic programming problem, but I don't know its name or related keywords. Can anyone give me some clues? The problem reads \begin{equation} {\min}_x \...
Duo Zhang's user avatar
  • 11
4 votes
1 answer
203 views

For $i=1,...,n$, let $b_i$ be a scalar and $A_i$ be an $k\times l$ matrix. Is there a closed form solution for the following problem assuming $n>k+l$? $$\min_{x\in \mathbb{R}^k ,y\in \mathbb{R}^l} \...
dff's user avatar
  • 240
2 votes
0 answers
102 views

Let $f(x):\mathbb{R}^K\Longrightarrow \mathbb{R}^L$ denote a multivariate continuously differentiable function. All the partial derivatives of $f$ (all its Jacobian elements) are bounded from above ...
Yarden Levy's user avatar
  • 21
0 votes
0 answers
52 views

I am thinking about the possibility of making a parameter in my regression, let's say the $\lambda$ in a ridge regression, somehow, inside a range : $\lambda \in [0,1]$. Do you have any ideas how I ...
SUMQXDT's user avatar
  • 1
1 vote
0 answers
112 views

I have a function expressed as the ratio of two exponential series with certain parameters $$\frac{\sum\limits_{j=1}^{i-1} \frac{e^{-ar_jt}}{\prod_{l=1\\l \ne j}^{i-1} (b^j-b^l)}}{\sum\limits_{j=1}^{i}...
Ahmed Khan's user avatar
  • 35
1 vote
0 answers
69 views

Suppose the observation $(X_1, Y_1), \ldots, (X_n, Y_n)$ satisfies the following semi-parametric model $$Y_t = m(X_t, \alpha) + \sigma(X_t, \beta) U_t,$$ where $U_t$ is independent with $X_t$ with ...
香结丁's user avatar
  • 341
0 votes
1 answer
67 views

Let $Y_t \in \mathbb{R}$ be a response variable and $X_t$ a $d$-dimensional explanatory variable. Assume we observe the process that $(X_1, Y_1), \cdots, (X_n, Y_n)$. \begin{equation} Y_{t} = \mu(...
香结丁's user avatar
  • 341
1 vote
1 answer
795 views

I'm interested in bounding the tail probabilities of a quadratic form $x^t A x$ where $x\in \mathbb{R}^n$ is a sub-Gaussian vector with independent entries. $A\in \mathbb{R}^{n\times n}$ is a matrix. ...
Puzzler's user avatar
  • 31
1 vote
3 answers
140 views

I am interested in doing RKHS regression with missing response variables. Given input-output pairs $(x_i,y_i)$, I want to estimate a function $f(\cdot)$ as follows \begin{equation}f(x)\approx u(x)=\...
MthQ's user avatar
  • 41
1 vote
0 answers
211 views

I have a large linear regression where all the independent variables are logical (ie TRUE/FALSE) and sparse. The data has roughly 10,000 variables and 10 million observations, on average around 20 ...
quarague's user avatar
  • 697
1 vote
1 answer
155 views

A few months ago I asked this question on Mathematics Stack Exchange but it has received little attention. Perhaps the question is more applicable here. Let $p_k$ denote the $k$th prime such that $...
TheSimpliFire's user avatar
  • 1,379
8 votes
3 answers
459 views

I'm studying the difference between regularization in RKHS regression and linear regression, but I have a hard time grasping the crucial difference between the two. Given input-output pairs $(x_i,y_i)...
MthQ's user avatar
  • 41
1 vote
2 answers
556 views

Any rank k matrix $Y\in\mathbb{R}^{m\times n}$ can be written as: $$ Y = UV'$$ Where $U\in \mathbb{R}^{m\times k}, V\in \mathbb{R}^{n\times k}$. This factorization is not unique since for any ...
Patrick's user avatar
  • 111
4 votes
2 answers
1k views

Let $R$, $C$, and $X$ be independent random variables defined on $(0,\infty)$ and $$Y=\underbrace{R\, X}_{Z}+C.$$ We are given the joint probability distribution of $X$ and $Y$, $P_{XY}(x,y)$ and ...
stochastic's user avatar
  • 371
3 votes
1 answer
159 views

In the SCAD paper by Fan and Li (2001), there exist two forms of penalized least squares as follows: $$\frac{1}{2}\left \| y-X\beta \right \|^2+\lambda \sum_{j=1}^{d}p_j (\left | \beta _j \right |),$$ ...
Tim Xu's user avatar
  • 31
2 votes
0 answers
67 views

Let $\{\mathbf{x}_i, y_i \}$ be a set of binary-labeled samples ($\mathbf{x}_i \in \mathbb{R}^d, y_i \in \{a,b\}, a,b\in\mathbb{R}$). Let $\{ \mathbf{x}'_i, y_i \}$ be also such a set. Define $\mathbf{...
le4m's user avatar
  • 183
2 votes
1 answer
84 views

I want to solve the folowing problem B*M=V, where B is the unknown of size 3x3, M of size 3xN and V of size 3xN. The difficulty is, that B has to be unitary. N is in the range of 500. All matrices ...
yar's user avatar
  • 231
1 vote
0 answers
113 views

I'm an undergraduate student who is green in statistics. I have a problem in the chose of objective function when estimating the parameters. Let $Y = \beta^TX + \epsilon $ be the standard liner ...
R. Qian's user avatar
  • 11
9 votes
1 answer
22k views

I need to clarify some idea I have in my mind about linear and non-linear regressions. Whatever I know about this topic comes from the book of Taylor "Introduction to error analysis": a set ...
Stefano Fedele's user avatar
  • 203
1 vote
0 answers
82 views

This is possibly a reference request. Let $G$ : $\mathbb{R}^p \to \mathbb{R}^q$ be a continuous injective/bijective function. Let $\mu$(we may also assume this to be a non degenerate Gaussian) be ...
Madhuresh's user avatar
  • 157
2 votes
0 answers
174 views

On page 79 (or page 5) of this this paper the gradient of the SSE of the Geodesic model is described explicitly. My question is how are these equitations derived in detail; where can I find the ...
AB_IM's user avatar
  • 4,842
3 votes
3 answers
3k views

I have a matrix which is similar to Vandermonde matrix except that the entries are monomials of degree $d$ polynomial in 2 variables. Each row has the following form: $X_{i}= [1, x_{i}, y_{i}, x_{i}^...
TravisJ's user avatar
  • 133
2 votes
0 answers
341 views

Background: Let $X \in \mathbb{R}^{N\times K}$ and $y \in \mathbb{R}^{N\times 1}$ be data for a regression problem. The aim is to find $\beta \in \mathbb{R}^{K\times 1}$ such that $X\beta \approx y$ ...
svangen's user avatar
  • 326
1 vote
1 answer
191 views

Let $E\subset{\mathbb R}^n$ be a set of the type $I_1\times \dots \times I_n$, where $I_k$ are real intervals, and $X$ be and $n\times p$ real matrix. Suppose also that $rank(X)=p$ and $n>p$. Is ...
Linger's user avatar
  • 13
-1 votes
1 answer
755 views

I'm trying to fit a quadratic $a_0 + a_1x + a_2x^2$ by Polynomial Regression: $$ \begin{pmatrix} n & \Sigma x_i & \Sigma x_i\\ \Sigma x_i & \Sigma x_i^2 & \Sigma x_i^3\\ \Sigma ...
Ed King's user avatar
  • 101
1 vote
0 answers
317 views

I was looking at numpy's lstsq to find a least squares solution of an equation system when the following occurred to me: Given the points (0,0), (3,4), (4,3), if I ...
nh2's user avatar
  • 111
4 votes
1 answer
253 views

I have a theoretical question about regression models. Let's say I measured multiple responses from $n$ subjects and these responses are correlated with each other. For example, let's say I measured ...
user1701545's user avatar
  • 163
1 vote
1 answer
3k views

Suppose we want to find coefficients $b$ in $\underset{b}{\operatorname{argmin}} \displaystyle\sum\limits_{i=1}^n | y_{i}-b_{1}x_{i}-b_{0}\mid$. If we rewrite this problem in terms of linear ...
Math_manul's user avatar
  • 199
10 votes
0 answers
397 views

Scheffé's method for identifying statistically significant contrasts is widely known. A contrast among the means $\mu_i$, $i=1,\ldots,r$ of $r$ populations is a linear combination $\sum_{i=1}^r c_i \...
Michael Hardy's user avatar
  • 12.9k