The classical universal approximation theorem says that functions of the form $f(x)=\sum_{i=1}^N\alpha_i\sigma(w_i^Tx_i+b_i)$ can approximate any continuous function from a compact subset of $\mathbb{R}^n$ to $\mathbb{R}$, where $\sigma$ is the sigmoid function. My question is: can this class of functions approximate any nonnegative continuous function on a compact subset when $\alpha_i\geq0,\forall i$? And what about the situation where $\alpha_i\geq0,\forall i$, $\sum_{i=1}^N\alpha_i=1$ and $f(x)$ is between $0$ and $1$?
1
-
1$\begingroup$ Cross-posted: math.stackexchange.com/q/5066737/14578, mathoverflow.net/q/492904/37212, cs.stackexchange.com/q/171951/755. Please do not post the same question on multiple sites. $\endgroup$D.W.– D.W.2025-05-20 06:44:33 +00:00Commented May 20 at 6:44
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
In dimension one, it would imply that any non-negative function $f$, say on the interval $[0,1]$, can be approximated by a sum of a non-negative increasing function $f_+$ and a non-negative decreasing function $f_-$, uniformly up to within $\epsilon$. But this is obviously false. For example, if $f(1)=0$, then $f_+(1)\leq \epsilon$, but since $f_+$ is increasing and non-negative, this implies $0\leq f_+\leq \epsilon$. Therefore for all $x<y$ one has $$ f(y)\leq f_-(y)+f_+(y)+\epsilon\leq f_-(x)+2\epsilon\leq f(x)+3\epsilon. $$ But not all continuous non-negative functions satisfying $f(1)=0$ have this property, e.g., any function with $f(0)=0$ and $f(1/2)=4\epsilon$ fails it.
-
$\begingroup$ Sorry I can't understand this. Why dose it imply that $f$ could be approximated by a sum of two functions? Is it possible that $f$ could be approximated by a sum of $m$ functions? $\endgroup$Mango– Mango2025-05-20 07:10:11 +00:00Commented May 20 at 7:10