Skip to main content
MathOverflow

Questions tagged [least-squares]

Ask Question

The tag has no summary.

46 questions
Newest Active Bountied Unanswered
Filter by
Sorted by
Tagged with
4 votes
1 answer
231 views

Given $n < m < n^2$, fat rank-$m$ matrix ${\bf A} \in {\Bbb R}^{m \times n^2}$ (that has full row rank) and vector ${\bf y} \in {\Bbb R}^m$, $$\begin{align} \underset{{\bf X} \in {\Bbb R}^{n \...
usergh's user avatar
  • 83
0 votes
0 answers
84 views

Suppose we are given a list of $N$ positive definite quadratic forms $X^TQ_k X$ (where $k\in[1,N]$ and $Q_k\in\mathbb{R}^{p\times p}$ $\forall k$), and a positive vector $V$ of same length $N$ i.e. $V=...
Ernest F's user avatar
  • 1
0 votes
1 answer
196 views

I've tried, unsuccessfully, to either solve or find a solution to something along lines of: find $\bar{a}$, $\bar{b}$ nearby to some initial guess that satisfies $\bar{c} = \bar{a} \times \bar{b}$. ...
wrjohns's user avatar
  • 101
1 vote
1 answer
237 views

I have the task of finding a Chebyshev approximation for a time-series; I want to check different types of functions, e.g. polynomials, rational functions, harmonics, etc. I know that the Remez ...
Manfred Weis's user avatar
  • 14k
2 votes
1 answer
180 views

Consider the following quantity $$X^T (XX^T + \mathrm{Id})^{-1} X,$$ where $X \in \mathbb{R}^{m\times n}$ is a iid random matrix with 0 mean and finite variance. The empiric covariance matrix ${X^T X}$...
Goulifet's user avatar
  • 2,602
0 votes
0 answers
135 views

In chemical analysis, the instrument's signal are plotted as a function of chemical concentration. In general, higher the concentration higher is the response and the relationship is linear. At ...
ACR's user avatar
  • 923
1 vote
0 answers
61 views

Let $M \in \mathbb{R}^{m \times n}$. Let $S \in \mathbb{R}^{m \times N_t}, U \in \mathbb{R}^{n \times N_t}$, with $ N_t \gg m,n$. Moreover, $\epsilon = S - M U$, with $\epsilon$ zero mean white noise ...
baptiste's user avatar
  • 123
1 vote
0 answers
51 views

I am currently working on a problem involving the minimization of the $\chi^2$ deviation between a model matrix ($C_\text{model}$) and a measured matrix ($C_\text{measured}$). by finding the best-fit ...
Elaf Salah's user avatar
  • 11
1 vote
0 answers
292 views

$$ \begin{align} f_i &= \operatorname*{argmin}_f \| Af - d \| ^2_{l2} + λ \| P_X(f) - L_r(P_X(f_{i-1})) \|_F^2 \\[8pt] &=M_4^{-1}(A^Hd+\lambda P_X^*(L_r(P_X(f_{i-1})))) \end{align} $$ where $$ ...
Mark Hayes's user avatar
  • 11
1 vote
1 answer
224 views

Given the wide matrices ${\bf A} \in {\Bbb R}^{n \times m}$ and ${\bf B} \in {\Bbb R}^{p \times m} $, where $m > n > p$, form an overdetermined linear system in ${\bf X} \in {\Bbb R}^{p \times n}...
RedOct's user avatar
  • 11
0 votes
1 answer
229 views

I am trying to understand why I am getting an almost singular matrix in a problem I have. The problem is a simple as $$ \min_{X \in \mathbb{R}^{m,n}} \left\lVert AX - B \right\rVert_F^2 $$ Obvioulsy ...
user8469759's user avatar
  • 129
1 vote
1 answer
222 views

Let $f(x)$ be some unknown continuous square-integrable function defined on the interval $[0,1]$. Suppose we have $i=1,...,n$ samples $f_i$ of $f$ of the following form: $$f_i(x)=a_i*f(x+b_i)+c_i$$ ...
dff's user avatar
  • 240
0 votes
1 answer
155 views

An issue from 3D tessellated geometry: Find the direction vector of a plane that minimizes the silhouette of a set of triangles. To say it another way, find the direction vector that is most ...
mattica's user avatar
  • 103
1 vote
0 answers
90 views

I'm looking for algorithms to solve a special quadratic programming problem, but I don't know its name or related keywords. Can anyone give me some clues? The problem reads \begin{equation} {\min}_x \...
Duo Zhang's user avatar
  • 11
1 vote
3 answers
501 views

Given a matrix $L\in \mathbb{R}^{3 \times 3}$, I'm looking for a method to find the closest (in a least squares sense) product of a non-uniform scaling matrix and a rotation matrix: $$ \min_{s\in\...
Alec Jacobson's user avatar
  • 733
4 votes
1 answer
203 views

For $i=1,...,n$, let $b_i$ be a scalar and $A_i$ be an $k\times l$ matrix. Is there a closed form solution for the following problem assuming $n>k+l$? $$\min_{x\in \mathbb{R}^k ,y\in \mathbb{R}^l} \...
dff's user avatar
  • 240
2 votes
0 answers
102 views

Let $f(x):\mathbb{R}^K\Longrightarrow \mathbb{R}^L$ denote a multivariate continuously differentiable function. All the partial derivatives of $f$ (all its Jacobian elements) are bounded from above ...
Yarden Levy's user avatar
  • 21
1 vote
0 answers
283 views

Given symmetric and positive definite $n \times n$ (real) matrices $A_1, \dots, A_m$ and $b_1, \dots, b_m \in {\Bbb R}^{n}$, I am trying to find the solution with the least sum of squared errors of ...
abc's user avatar
  • 11
2 votes
0 answers
139 views

Given a system of normal equations $A^TAx=A^Tb$ where $A^TA$ is $n\times n$, what I call the $i$th subsystem is the linear system of size $n-1\times n-1$ where the $i$th column of $A$ has been removed....
Ke. Fel's user avatar
  • 29
1 vote
0 answers
55 views

For each $n\in \mathbb{N}\cup\{0\}$, let $x_n(t)=\frac{t^n}{n!}$ for all $t\in [0,1]$. As the functions $X_N=(x_0 ,\ldots, x_N)$ are linearly independent, the matrix $ \int_0^1 X_N(t)^\top X_N(t)\,\...
John's user avatar
  • 537
20 votes
4 answers
6k views

Given a linear system $Ax=b$, the pseudoinverse of $A$ is found as the matrix $A^+$ such that $x=A^+ b$ where $x$ solves the least squares problem $\min \| Ax - b \|^2 $ and $x \perp \mathcal{N}(A)$. ...
Herman Jaramillo's user avatar
  • 521
3 votes
1 answer
470 views

Define the piecewise-linear function $\psi(t):=\max(t,0)$ for all $t \in \mathbb R$. Let $d,n,k \to \infty$ at the same rate (i.e $n \asymp k \asymp d$). Let $y_1,\ldots,y_n \in \{-1,1\}$ uniformly ...
dohmatob's user avatar
  • 7,043
1 vote
2 answers
252 views

Problem setting : $ \underset{x}{\text{min}} \|Ax-b\|$, where $A \in \mathcal{R}^{m \times n}, m\gg n $, full rank. L1 loss is used for robust estimation using IRLS. The corresponding equation to ...
lalit's user avatar
  • 21
1 vote
1 answer
223 views

I'm trying to algorithmically solve the Time Delay of Arrival problem as part of some mathematics research. The problem is as follows: Given the location of three receivers in a plane (A, B, and C), ...
K_M's user avatar
  • 111
2 votes
2 answers
157 views

Given a finite set of points $(x_1, y_1), (x_2, y_2), \ldots, (x_n, y_n)$ in the plane, Linear Regression tells us how to find the straight line "$y=a+bx$" best approximating the given points, in the ...
Ruy's user avatar
  • 2,313
1 vote
3 answers
140 views

I am interested in doing RKHS regression with missing response variables. Given input-output pairs $(x_i,y_i)$, I want to estimate a function $f(\cdot)$ as follows \begin{equation}f(x)\approx u(x)=\...
MthQ's user avatar
  • 41
2 votes
1 answer
331 views

Consider column vectors $\boldsymbol{z}_i$, $\quad i=1,\dots,n$. Each $\boldsymbol{z}_i$ has $j$ elements and can be expressed as $\boldsymbol{z}_i = \begin{bmatrix} \boldsymbol{x}_i \\ \boldsymbol{y}...
Lincoln Hannah's user avatar
  • 21
8 votes
3 answers
1k views

Given tall matrices $A$ and $Y$ and the following overdetermined linear system in square matrix $X$ $$AX=Y$$ is there an explicit formula for the least-squares solution if $X$ is constrained to be ...
Museful's user avatar
  • 233
0 votes
1 answer
1k views

From the algorithm, we can see that it tries different damping factor until it gets a good one by the error. Is the damping factor related to the eigenvalues of the Hessian matrix?
zskalibur's user avatar
  • 1
2 votes
0 answers
59 views

Given a finite set $P$ of points in $\mathbb{E}^3$ , one can calculate an approximating plane either as the solution of a Total Least Squares problem or by interpreting the problem physically, ...
Manfred Weis's user avatar
  • 14k
1 vote
1 answer
217 views

It is well known that $E[X|X+Y]$ is Gaussian if both $X$ and $Y$ are, and the result can be derived using standard density arguments. However, how can one prove it by only resulting to optimization ...
AB_IM's user avatar
  • 4,842
1 vote
0 answers
159 views

Good morning, I have the a set of data $(\sigma,D,\alpha_0)_i$, $i=1...n$ data. I want to determine two parameters $K_{IC}$, $C_f$ in the basic equation given as $K_{IC} = \sigma \sqrt{D} k_0(\...
gama's user avatar
  • 11
2 votes
2 answers
680 views

I'm working on a real-time implementation of Lucas-Kanade for optical flow. However, the SVD decomposition to do achieve the least square method to reduce the error seems to take too much time. A ...
Miguel Rueda's user avatar
  • 121
8 votes
3 answers
459 views

I'm studying the difference between regularization in RKHS regression and linear regression, but I have a hard time grasping the crucial difference between the two. Given input-output pairs $(x_i,y_i)...
MthQ's user avatar
  • 41
5 votes
2 answers
404 views

For the Sylvester matrix equation $AX+XB=C$, I want to find the least-squares solution $X$ via $$\begin{array}{ll} \text{minimize} & \| AX + XB - C \|_{\text{F}}^2\\ \text{subject to} & \mbox{...
dave2d's user avatar
  • 191
4 votes
2 answers
633 views

I am trying to find the least-squares solution $X$ of the following matrix equation $$AXB+CXD=E$$ Of course, I know that this equation can be written in the form $$(B^T \otimes A+D^T \otimes C) \...
dave2d's user avatar
  • 191
3 votes
2 answers
445 views

I have an optimization problem of the following form $$\text{minimize} \,\|Qa-b\|_2 \quad \text{ subject to } Q \succeq 0$$ where $a,b \in \mathbb{R}^n$ are given and the $n \times n$ square matrix ...
user402940's user avatar
  • 153
2 votes
1 answer
215 views

We have an unknown $m\times n$ matrix $X=(x_{ij})_{i=1,j=1}^{m,n}$. Assume we are given measurements of the differences $$x_{i,j+1}-x_{i,j}$$ and $$x_{i+1,j}-x_{i,j}$$ for all $(i,j)\in \{1,\...
user100927's user avatar
  • 398
9 votes
1 answer
22k views

I need to clarify some idea I have in my mind about linear and non-linear regressions. Whatever I know about this topic comes from the book of Taylor "Introduction to error analysis": a set ...
Stefano Fedele's user avatar
  • 203
0 votes
0 answers
107 views

In their classic paper "Ideal spatial adaptation by wavelet shrinkage" (http://biomet.oxfordjournals.org/content/81/3/425.short?rss=1&ssource=mfr), Donoho and Johnstone make the following ...
user32849's user avatar
  • 121
1 vote
1 answer
168 views

Motivation Then the usual stochastic filtering problem says that: $$ \operatorname{argmin}_{Z \in L^2(\mathscr{G}_t)}\,\mathbb{E}[(Y_t-Z_t)^2], $$ where $\mathscr{G}_t$ is the $\sigma$-algebra ...
AB_IM's user avatar
  • 4,842
1 vote
1 answer
231 views

If $a_{m\times 1}$ and $Q_{m\times n}$ ($m<n $) are known, and we know every element of $b$ is between $[-1\ \ 1]$, how to determine $b$ to minimize $\|a+Qb\|_2$?
Peng Zhao's user avatar
  • 29
2 votes
1 answer
412 views

I have the following matrix equation $$AX=B$$ given $8 \times 3$ matrices $A$ and $B$. $X$ is a $3 \times 3$ diagonal matrix whose main diagonal contains the $3$ unknowns. Whenever I solve for $X$ ...
user90091's user avatar
  • 21
8 votes
2 answers
347 views

The Sylvester equation $AX+XB=C$ has been studied quite a lot and there are known algorithms for solving it. But has the situation where (an over-determined) system of equations $A_{i}X+XB_{i}=C_{i}$ ...
Felix Goldberg's user avatar
  • 7,092
0 votes
2 answers
375 views

I want to quickly solve the following linear least-squares problem $$\min_{x \in \mathbb{R}^n} \left\| A x - b \right\|_2^2$$ with a special sparse structure where each row in $A$ has only up to $4$ ...
sellibitze's user avatar
  • 101
3 votes
1 answer
3k views

I would like to solve the following optimization problem in $k$-vector $w_i$ $$ \min_{w_i} \quad \left\|P_i - X \mbox{diag} (w_i) Y^T \right\|_F^2 $$ where $P_i$ is a $6 \times 6$ matrix, $X$ and $Y$ ...
Jackson's user avatar
  • 33