Bourbaki and I Broke Up

Bourbaki and I Broke Up 2026-08-17T13:59:14+00:00 https://hadrien-montanelli.github.io/ When planets dance https://hadrien-montanelli.github.io/2017-10-09.html 2017-10-09T00:00:00+00:00

My second paper as a Ph.D. student, in collaboration with my friend and former colleague at Oxford Nikola Gushterov, was about the choreographies of the $n$-body problem. These are very simple periodic solutions of the $n$-body problem, in which the bodies share a common orbit and are uniformly spread along it; see, e.g., these animations. The trivial ones are circles, and these were found by Lagrange in 1772. Fore more than two centuries, this was the full story. Everything changed at the end of the 20th century, when Moore (numerically in 1993), and Chenciner and Montgomery (theoretically in 2000), discovered the first non-trivial choreography: the figure-eight of the three-body problem. And then the magic happened: in the early 2000s, Simò found many new choreographies using numerical optimization of the so-called action. In this post, I explain the idea of Simò, who coined the term choreographies, the $n$ bodies being "seen to dance in a somewhat complicated way." Let $z_j(t)\in\mathbb{C}$, $0\leq j\leq n-1$, denote the positions of $n$ bodies with unit mass in the complex plane. The $n$-body problem describes the motion of these bodies under the action of Newton's law of gravitation, through the nonlinear coupled system of ODEs $$ z_j^{''}(t) - \sum_{\underset{i\neq j}{i=0}}^{n-1} \frac{z_i(t) - z_j(t)}{\big\vert z_i(t) - z_j(t) \big\vert^3} = 0, \;\, 0\leq j\leq n-1. $$ Since choreographies share a single orbit and are uniformly spread along it, these can be written as $$ z_j(t) = q\Big(t + \frac{2\pi j}{n}\Big), \;\, 0\leq j\leq n-1, $$ for some $2\pi$-periodic function $q:[0,2\pi]\rightarrow\mathbb{C}$. It has been well known since Poincaré that the principle of least action, first introduced by Maupertuis in 1744, can be used to characterize choreographies: these are minima of the action, defined as the integral over one period of the kinetic minus the potential energy, $$ A = \int_0^{2\pi} \big(K(t) - U(t)\big)\,dt, $$ with kinetic energy $$ K(t) = \frac{1}{2}\sum_{j=0}^{n-1} \Big\vert q'\Big(t + \frac{2\pi j}{n}\Big) \Big\vert^2 $$ and potential energy $$ U(t) = -\sum_{j=0}^{n-1}\sum_{i=0}^{j-1} \Big\vert q\Big(t + \frac{2\pi i}{n}\Big) - q\Big(t + \frac{2\pi j}{n}\Big) \Big\vert^{-1}. $$ Note that the action $A$ depends on $q(t)$ via $U(t)$ and on $q'(t)$ via $K(t)$. Since the integral of $K(t)$ does not depend on $j$ and the integral of $U(t)$ only depends on $i-j$, the action can be rewritten $$ \begin{align} A = \frac{n}{2}\int_0^{2\pi} \big\vert q'(t) \big\vert^2 dt + \frac{n}{2}\sum_{j=1}^{n-1} \int_0^{2\pi} \Big\vert q(t) - q\Big(t + \frac{2\pi j}{n}\Big) \Big\vert^{-1}dt. \end{align} $$ Choreographies correspond to functions $q(t)$ which minimize $A$. Since these are closed curves in the complex plane, these can be represented by Fourier series. The action $A$ becomes a function of Fourier coefficients, can be computed with the exponentially accurate trapezoidal rule, and an optimization algorithm (over a finite number of Fourier coefficients) can be used to found the minima. In our paper, we have recomputed the choreographies found by Simò to higher accuracy, and extended his work to spaces of constant positive curvature—check it out!

Solving PDEs on the sphere https://hadrien-montanelli.github.io/2017-10-26.html 2017-10-26T00:00:00+00:00

My latest paper, with my former colleague Yuji Nakatsukasa, has just been accepted in SISC. In this post, I review the main ideas of our work. We are interested in computing smooth solutions of stiff PDEs on the unit sphere of the form $$ u_t = \alpha\Delta u + \mathcal{N}(u), \;\; u(0,x,y,z)=u_0(x,y,z), $$ where $u(t,x,y,z)$ is a function of time $t$ and Cartesian coordinates $(x,y,z)$ with $x^2 + y^2 + z^2=1$. The function $u$ can be real or complex and the PDE can be a single equation, as well as a system of equations. A large number of PDEs of interest in science and engineering take this form. Examples include the Allen–Cahn equation $u_t = \epsilon\Delta u + u - u^3$, the nonlinear Schrödinger equation $u_t=i\Delta u + iu|u|^2$, the Ginzburg–Landau equation (the picture at the top is a solution), and all reaction-diffusion equations. Our algorithms are based on a variant of the double Fourier sphere method in coefficient space with multiplication matrices that differ from the usual ones, and implicit-explicit time-stepping schemes. Operating in coefficient space with these new matrices allows one to use a sparse direct solver, avoids the coordinate singularity, and maintains smoothness at the poles, while implicit-explicit schemes circumvent severe restrictions on the time-steps due to stiffness. A comparison is made against exponential integrators and it is found that implicit-explicit schemes perform best. Implementations in MATLAB and Chebfun make it possible to compute the solution of many PDEs to high accuracy in a very convenient fashion—check out the spinsphere code. I hope to use this code for investigating pattern formation on the sphere with Philippe Trinh, Michael Ward, and Stanislav Shvartsman. I'm visiting Stanislav at Princeton in a couple of weeks—looking forward to it!

Deep networks and the curse of dimensionality https://hadrien-montanelli.github.io/2017-12-22.html 2017-12-22T00:00:00+00:00

I started working on deep learning models when I began my research at Columbia University in September 2017. Coming from the approximation theory world, I wanted to learn about their approximation properties. A few months down the road Qiang Du and I have proven a new theorem concerning the approximation of multivariate functions by deep ReLU networks. Deep learning has been successfully applied to many fields including computer vision, speech recognition, and natural language processing. It is based on approximations by deep networks, as opposed to shallow networks. The latter are neural networks with a single hidden layer and correspond to approximations $f_N$ of multivariate functions $f:\mathbb{R}^d\rightarrow\mathbb{R}$ of the form $$ f_N(\boldsymbol{x}) = \sum_{i=1}^N \alpha_i \sigma(\boldsymbol{w}_i^T\boldsymbol{x} + \theta_i), $$ with $\alpha_i,\theta_i\in\mathbb{R},\, \boldsymbol{x},\boldsymbol{w}_i\in\mathbb{R}^d$, and for some activation function $\sigma:\mathbb{R}\rightarrow\mathbb{R}$. The former are neural networks with one or more hidden layers, where each unit of each layer performs an operation of the form $\sigma(\boldsymbol{w}\cdot\boldsymbol{x} + \theta)$. The depth of a network is the number of hidden layers, and the size is the total number of units. Shallow networks have depth $1$ and their size is the number $N$ in the expansion above, while deep networks usually have depth $\gg 1$. Deep ReLU networks use the activation function $\sigma(x) = \max(0,x)$. One of the most important theoretical problems is to determine why and when deep (but not shallow) networks can lessen or break the curse of dimensionality. A possible way of addressing this problem is to focus on a particular set of functions that have a very special (compositional or polynomial) structure and to show that for this particular set deep networks perform extremely well. We have followed a different route. We have considered a generic set of functions and proved new error estimates for which the curse of dimensionality is lessened by establishing a connection with sparse grids. For a real-valued function $f$ in $\mathbb{R}^d$ whose smoothness is characterized by some integer $m$ (typically the order of integrable derivatives), and for some prescribed error $\epsilon>0$, one tries to show that there exists a neural network $f_N$ of size $N$ that satisfies $$ \Vert f - f_N \Vert \leq \epsilon \;\; \text{with} \;\; N=\mathcal{O}(\epsilon^{-\frac{d}{m}}), $$ for some norm $\Vert\cdot\Vert$. For deep networks, one also wants to find the asymptotic behavior of the depth as a function of $\epsilon$. Results of this form are standard approximation results that suffer from the curse of dimensionality. For small dimensions $d$, the size $N$ of the network increases at a reasonable rate as $\epsilon$ goes to zero. However, $N$ grows geometrically with $d$. The picture is different for deep networks. My theorem shows that functions in the so-called Korobov spaces $X^{2,p}(\Omega)$ of mixed derivatives of order two can be represented with error $\epsilon$ by deep networks of depth $\mathcal{O}(\vert\log_2\epsilon\vert\log_2d)$ and size $$ N=\mathcal{O}(\epsilon^{-\frac{1}{2}}\vert\log_2\epsilon\vert^{\frac{3}{2}(d-1)+1}(d-1)). $$ The curse of dimensionality is not totally overcome but is lessened since $d$ only affects logarithmic factors $\vert\log_2\epsilon\vert$.

Gibbs phenomenon and Cesàro mean https://hadrien-montanelli.github.io/2018-01-04.html 2018-01-04T00:00:00+00:00

In this post, I talk about a fascinating subject: the convergence of Fourier series and the Gibbs phenomenon. Periodic continuous functions $f$ of bounded variation have a unique and uniformly convergent Fourier series, that is, the partial sums $$ f_n(\theta) = \sum_{k=0}^{n}a_k\cos(k\theta) + \sum_{k=1}^{n}b_k\sin(k\theta), $$ with Fourier coefficients $$ a_k = \frac{1}{\pi}\int_0^{2\pi}f(\theta)\cos(k\theta)d\theta, $$ and $$ b_k = \frac{1}{\pi}\int_0^{2\pi} f(\theta) \sin(k\theta)d\theta, $$ converge uniformly to $f$. (Note that $1/\pi$ is changed to $1/(2\pi)$ for $a_0$.) We write $$ f(\theta) = \sum_{k=0}^{\infty}a_k\cos(k\theta) + \sum_{k=1}^{\infty}b_k\sin(k\theta). $$ For functions $f$ that are merely of bounded variation, the convergence is only pointwise: at every point $\theta$, the partial sums converge to $\frac{1}{2}[f(\theta^-) + f(\theta^+)]$; in particular, these converge to $f(\theta)$ at every point of continuity. What happens at points of discontinuity? A classical example is the square wave $$ f(\theta) = \frac{\pi}{4}\mathrm{sign}(\pi - \theta), \quad \theta\in[0,2\pi]. $$ This function is of bounded variation and discontinuous at $0$, $\pi$ and $2\pi$. Its Fourier coefficients can be computed analytically and are given by $$ a_k = 0, \quad b_{2k} = 0, \quad b_{2k+1} = \frac{1}{2k+1}. $$ When we compute the partial sums for $n=16,32,64,128$ we obtain the picture at the top. This is the so-called Gibbs phenomenon, which involves both the fact that the partial sums overshoot at a discontinuity and that this overshoot does not disappear as more terms are added to the sums. The overshoot at $0$, $\pi$ and $2\pi$ is known exactly and is given by $$ \frac{1}{2}\int_0^{\pi}\frac{\sin(\theta)}{\theta}d\theta - \frac{\pi}{4} = \frac{\pi}{2}\times(0.0895\ldots), $$ or about $9\%$ of the jump. Note that, as $n$ increases, the position of the overshoot moves closer to the discontinuity. The Gibbs phenomenon is a very standard result that most (applied) mathematicians know. What is perhaps less known is a simple cure using the $(C,1)$ Cesàro mean of the partial sums, which is simply the arithmetic mean, $$ \sigma_n(\theta) = \frac{1}{n+1}\sum_{k=0}^{n}f_k(\theta). $$ For integrable functions $f$, one can show that $\sigma_n(\theta)$ converges to $\frac{1}{2}[f(\theta^-) + f(\theta^+)]$; in particular, this converges to $f(\theta)$ at every point of continuity, and for continuous functions the convergence is uniform—this is Fejér's thereom. Moreover, if $m\leq f(\theta)\leq M$ then $m\leq \sigma_n(\theta)\leq M$. Therefore, the $(C,1)$ Cesàro mean does not show the Gibbs phenomenon, as the following figure illustrates: With Mikael Slevinsky, we are using the $(C,2)$ Cesàro mean for spherical harmonic expansions on the sphere to post-process numerical solutions of nonlocal PDEs—the paper is coming soon!

Solving nonlocal equations on the sphere https://hadrien-montanelli.github.io/2018-01-25.html 2018-01-25T00:00:00+00:00

I have just submitted a paper about nonlocal equations on the sphere with my colleagues Mikael Slevinsky and Qiang Du. Here are the main ideas of our work. We proposed a fast spectral method for computing solutions of nonlocal models of the form $$ u_t = \epsilon^2\mathcal{L}_\delta u + \mathcal{N}(u), \;\; u(t=0,\boldsymbol{x}) = u_0(\boldsymbol{x}), $$ where $u(t,\boldsymbol{x})$ is a function of time $t\ge0$ and position $\boldsymbol{x}$ on the unit sphere $\mathbb{S}^2\subset\mathbb{R}^3$, $\mathcal{N}$ is a nonlinear operator with constant coefficients (e.g., $\mathcal{N}(u)=u-u^3$), and $\mathcal{L}_{\delta}$ is a nonlocal Laplace–Beltrami operator, $$ \mathcal{L}_\delta u(\boldsymbol{x}) = \int_{\mathbb{S}^2} \rho_\delta(\vert\boldsymbol{x}-\boldsymbol{y}\vert) [u(\boldsymbol{y}) - u(\boldsymbol{x})]\,d\Omega(\boldsymbol{y}). $$ In the definition above, $d\Omega(\boldsymbol{y})$ denotes the standard measure on $\mathbb{S}^2$ and $\rho_\delta$ is a suitably defined nonlocal kernel with horizon \(0 There has been substantial work on the numerical approximation of this type of equations in Euclidean geometries. However, no study has been attempted so far to investigate similar discretizations on the sphere. It is of practical interest to study the extension to non-Euclidean geometries (the sphere being a representative example) for the modeling of anomalous diffusion, pattern formation, and image analysis. Our algorithms are based on the diagonalizability of nonlocal diffusion operators in the basis of spherical harmonics, the computation of their eigenvalues to high relative accuracy using quadrature and asymptotic formulas, and a fast spherical harmonic transform. The picture at the top shows the numerical solutions of the local (first row) and nonlocal (second row) Allen–Cahn equation. If you want to know more about our algorithms, check our paper out!

Spherical caps in cell polarization https://hadrien-montanelli.github.io/2018-02-27.html 2018-02-27T00:00:00+00:00

I have just submitted a paper about the modeling of embryogenesis with the Shvartsman group at Princeton University. Let me review the main ideas of our work. We proposed a new model for the formation of polarized patterns observed in early embryogenesis of C. elegans where mass conservation is the driving force. In our model, a species $V$ moves freely in a cell $\Omega$ and reversibly adheres to the cell membrane $\partial\Omega$. The membrane-bound species $U$ diffuses on the surface and recruits more of itself to the membrane. For a sphere of radius $r$, $U$ and $V$ satisfy the following equations, $$ \left\{ \begin{array}{l} U_t = \frac{D_U}{r^2}\Delta U + k_b\big(U_0 + H(U - \Gamma)\big)V - k_d U \;\text{on $\partial\Omega$}, \\[1.5pt] V_t = D_V\Delta V\;\text{in $\Omega$}, \\[1.5pt] D_V\nabla V\cdot n = -k_b\big(U_0 + H(U - \Gamma)V\big) + k_d U, \end{array} \right. $$ for some constants $D_U$, $D_V$, $k_b$, $k_d$, $\Gamma$ and $U_0$. Upon nondimensionalization by $u=k_dU/k_bV_0$, we obtain a single PDE $$ u_\tau = \delta^2\Delta u + \big(1 - \alpha\bar{u}\big)\big(\beta+H(u - \gamma)\big) - u. $$ Using the algorithms presented in a previous blog post, we showed that for arbitrary initial conditions, the only non-uniform long-term behavior was a single axisymmetric spherical cap: We also showed that, starting from a constant initial condition, transient convection is enough to start the polarization process: These findings reinforce the idea that symmetry breaking is crucial for the development of all living organisms.

Computer-assisted proofs for PDEs https://hadrien-montanelli.github.io/2018-12-05.html 2018-12-05T00:00:00+00:00

A couple of weeks ago, I spent a few days at McGill University with Jean-Philippe Lessard. He introduced me to the world of computer-assisted proofs, which I briefly describe in this post. Theory Suppose we want to find steady-state solutions of nonlinear time-dependent PDEs of the form $$ u_t = F(u), $$ for some smooth operator $F:U\mapsto U'$ between two Banach spaces $U$ and $U'$. This amounts to solve the nonlinear equation $F(u) = 0$. We typically use a spectral method and interpret $u$ as an infinite vector of spectral coefficients and $F$ as an operator acting on those coefficients. For finite $N$-dimensional problems $F_N(u_N)=0$ one would typically use Newton's method where one iterates the map $$ T_N(u_N) = u_N - DF_N^{-1}(u_N)F_N(u_N). $$ When the Jacobian $DF_N^{-1}(u_N)$ is invertible, $T_N$ is a contraction and it allows us to prove that the numerical solution $\bar{u}_N$ is close to the true solution $u_N$. (Typically, $u_N$ would be the vector of coefficients in a spectral expansion.) In an infinite-dimensional setting, Newton's method reads $$ T(u) = u - DF^{-1}(u)F(u). $$ However, working with the inverse $DF^{-1}(u)$ of the Fréchet derivative might be hard. Instead, one typically considers an approximation $A$ (independent of $u$) of $DF^{-1}(u)$ and looks at the fixed point problem $$ T(u) = u - AF(u). $$ The main challenge is to choose $A$ such that $T$ is a contraction on some neighborhood of the unknown true solution. The idea behind computer-assisted proofs goes like this. First, compute a numerical solution $\bar{u}_N$ of the discretized problem $F_N(u_N)=0$ in $N$ dimensions. Second, reinterpret $\bar{u}_N$ as an element $\bar{u}$ of the infinite-dimensional space by padding the vector of coefficients with an infinite number of zero. We expect the true solution $u$ to be close to $\bar{u}$ so we will choose $A$ to be an approximation of $DF^{-1}(\bar{u})$. The following theorem justifies this approach. Theorem (Radii polynomial approach in Banach spaces). Let $U$ and $U'$ be Banach spaces. Denote the norm on $U$ by $\Vert\cdot\Vert_U$. Consider bounded linear operators $A^{\dagger}\in L(U,U')$ and $A\in L(U',U)$. Assume that $F:U\mapsto U'$ is $C^1$, $A$ is injective and $AF:U\mapsto U$. Consider an approximate solution $\bar{u}$ of $F(u)=0$ (usually obtained using Newton's method on a finite-dimensional discretization). Let $Y_0$, $Z_0$, $Z_1$ and $Z_2$ be positive constants satisfying $$ \begin{align*} \Vert AF(\bar{u})\Vert_U & \leq Y_0, \\[1.5pt] \Vert I - AA^\dagger\Vert_{B(U)} & \leq Z_0, \\[1.5pt] \Vert A[DF(\bar{u})-A^\dagger]\Vert_{B(U)} & \leq Z_1, \\[1.5pt] \Vert A[DF(u)-DF(\bar{u})]\Vert_{B(U)}& \leq Z_2r, \end{align*} $$ for all $\Vert u - \bar{u}\Vert_U\leq r$, where $\Vert\cdot\Vert_{B(U)}$ is the induced operator norm for bounded linear operators from $U$ to itself. Define the radii polynomial $$ p(r) = Z_2 r^2 - (1-Z_1-Z_0)r + Y_0. $$ If there exists $r_0>0$ such that $p(r_0) One of the main challenges in this framework is to construct such an operator \(A$. (Note that $A^\dagger$ is an approximation of the Fréchet derivative $DF(\bar{u})$. In practice we often start by selecting a good $A^\dagger$.) An example in 1D Consider the Allen–Cahn equation with periodic boundary conditions, $$ u_t = F(u) = \epsilon^2 u_{xx} + u^3 - u. $$ In Fourier space, the linear part $Lu=\epsilon^2 u_{xx}-u$ is diagonal with entries $-\epsilon^2k^2 - 1$, $k\in\mathbb{Z}$. Let us denote by $\bar{u}_{N}$ the numerical solution obtained using wavenumbers $k$ between $-N$ and $N$ ($2N+1$ grid points) and by $DF_N(\bar{u}_N)$ the $(2N+1) \times (2N+1)$ Jacobian matrix at $\bar{u}_N$, which we can compute. In this case, a good choice for $A^\dagger$ is $$ A^\dagger = \begin{pmatrix} \scriptsize{DF_N(\bar{u}_N)} & 0 & 0 & \ldots \\ 0 & \scriptsize{-\epsilon^2(N+1)^2 -1} & 0 & \ldots \\ 0 & 0 & \scriptsize{-\epsilon^2(N+2)^2 -1} & \ldots \\ \vdots & \vdots & \vdots & \ddots \end{pmatrix}. $$ The idea is that for large enough $N$ the Fréchet derivative of $L$ will dominate the Fréchet derivative of $F$. (Note that $DL(u)=L$.) Therefore we can choose $$ A = \begin{pmatrix} \scriptsize{DF_N^{-1}(\bar{u}_N)} & 0 & 0 & \ldots \\ 0 & \frac{1}{-\epsilon^2(N+1)^2-1} & 0 & \ldots \\ 0 & 0 & \frac{1}{-\epsilon^2(N+2)^2-1} & \ldots \\ \vdots & \vdots & \vdots & \ddots \end{pmatrix}. $$

Deep networks and bandlimited functions https://hadrien-montanelli.github.io/2019-03-02.html 2019-03-02T00:00:00+00:00

In a recent paper, my colleagues and I considered the deep ReLU network approximation of generalized bandlimited functions $f:B=[0,1]^d\rightarrow\mathbb{R}$ of the form $$ f(\boldsymbol{x}) = \int_{\mathbb{R}^d}F(\boldsymbol{w})K(\boldsymbol{w}\cdot\boldsymbol{x})d\boldsymbol{w}, $$ with $\mathrm{supp}\,F\subset[-M,M]^d$, $M\geq1$, and for some square-integrable function $F:[-M,M]^d\rightarrow\mathbb{C}$ and analytic $K:\mathbb{R}\rightarrow\mathbb{C}$. We showed that, for any measure $\mu$, such functions can be approximated with error $\epsilon$ in the $L^2(B,\mu)$-norm by deep ReLU networks of depth $L=\mathcal{O}\left(\log_2^2\frac{1}{\epsilon}\right)$ and size $W=\mathcal{O}\left(\frac{1}{\epsilon^2}\log_2^2\frac{1}{\epsilon}\right)$, up to some constants that depend on $F$, $K$, $\mu$ and $B$. Our theorem is based on a result by Maurey, and on the ability of deep ReLU networks to approximate Chebyshev polynomials and analytic functions efficiently. Maurey's theorem A famous theorem of Carathéodory states that if a point $x\in\mathbb{R}^d$ lies in the convex hull of a set $P$ then $x$ can be written as the convex combination of at most $d+1$ points in $P$. Maurey's theorem is an extension of Carathéodory's result to the infinite-dimensional case of Hilbert spaces. Theorem (Maurey). Let $H$ be a Hilbert space with norm $\Vert\cdot\Vert$. Suppose there exists $G\subset H$ such that for every $g\in G$, $\Vert g\Vert\leq b$ for some $b>0$. Then, for every $f$ in the convex hull of $G$ and every integer $n\geq 1$, there is a $f_n$ in the convex hull of $n$ points in $G$ and a constant $c>b^2-\Vert f\Vert^2$ such that $\Vert f - f_n\Vert^2\leq \frac{c}{n}$. In practice, we used Maurey's theorem to show that there exists $$ \begin{align} f_{\epsilon}(\boldsymbol{x}) = \sum_{j=1}^{\lceil 1/\epsilon^2\rceil}b_j\big[\cos(\beta_j)\mathrm{Re}(K(\boldsymbol{w}_j\cdot\boldsymbol{x})) - \sin(\beta_j)\mathrm{Im}(K(\boldsymbol{w}_j\cdot\boldsymbol{x}))\big], \end{align} $$ with $\sum_{j=1}^{\lceil 1/\epsilon^2\rceil}\vert b_j\vert \leq C_F$ and $\beta_j\in\mathbb{R}$, such that $$ \Vert f_{\epsilon}(\boldsymbol{x}) - f(\boldsymbol{x})\Vert_{L^2(\mu, B)} \leq 2C_F\sqrt{\mu(B)}\epsilon. $$ In other words, the function $f$ is approximated by a linear combination of analytic functions with error $2C_F\sqrt{\mu(B)}\epsilon$ in the $L^2(B,\mu)$-norm. The task of approximating $f$ by deep ReLU networks has been reduced to approximating analytic functions by deep networks. Chebyshev polynomials and deep ReLU networks The Chebyshev polynomials of the first kind play a central role in approximation theory. They are defined on $[-1,1]$ via the three-term recurrence relation $$ T_n(x) = 2xT_{n-1}(x) - T_{n-2}(x), \;\; n\geq 2, $$ with $T_0=1$ and $T_1(x) = x$. We showed that deep ReLU networks can implement the recurrence relation efficiently. Since truncated Chebyshev series can approximate analytic functions exponentially well, so can deep networks.

Deep networks and the Kolmogorov–Arnold theorem https://hadrien-montanelli.github.io/2019-06-25.html 2019-06-25T00:00:00+00:00

In my latest paper, my colleague and I prove a theorem about the approximation of multivariate functions by deep ReLU networks, for which the curse of dimensionality is lessened. Our theorem is based on a constructive proof of the Kolmogorov–Arnold superposition theorem, and on a subset of multivariate continuous functions whose outer superposition functions can be efficiently approximated by deep ReLU networks. Kolmogorov–Arnold superposition theorem At the second International Congress of Mathematicians in Paris 1900, Hilbert presented ten of his 23 problems, including the 13th problem about equations of degree seven. He considered the following equation, $$ x^7 + ax^3 + bx^2 + cx + 1 = 0, $$ and asked whether its solution $x(a,b,c)$, seen as a function of the three parameters $a$, $b$ and $c$, can be written as the composition of functions of only two variables. Hilbert's 13th problem was solved by Kolmogorov and his 19 years old student Arnold in a series of papers in the 1950s. Kolmogorov first proved in 1956 that any continuous function of several variables can be expressed as the composition of functions of three variables. His student Arnold extended his theorem in 1957; three variables were reduced to two. Kolmogorov finally showed later that year that only functions of one variable were needed. The latter result is known as the Kolmogorov–Arnold superposition theorem, and states that any continuous functions $f:[0,1]^n\rightarrow\mathbb{R}$ can be decomposed as $$ f(x_1,\ldots,x_n) = \sum_{j=0}^{2n}\phi_j\left(\sum_{i=1}^n\psi_{i,j}(x_i)\right), $$ with $2n+1$ continuous outer functions $\phi_j:\mathbb{R}\rightarrow\mathbb{R}$ (dependent of $f$) and $2n^2+n$ continuous inner functions $\psi_{i,j}:[0,1]\rightarrow\mathbb{R}$ (independent of $f$). The Kolmogorov–Arnold superposition theorem was further improved in the 1960s and the 1970s. Lorentz showed in 1962 that the outer functions $\phi_j$ might be chosen to be the same function $\phi$, and replaced the inner functions $\psi_{i,j}$ by $\lambda_i\psi_j$, for some positive rationally independent constants $\lambda_i\leq 1$, while Sprecher replaced the inner functions $\psi_{i,j}$ by Hölder continuous functions $x_i\mapsto\lambda^{ij}\psi(x_i+j\epsilon)$ in 1965. Two years later, Fridman demonstrated that the inner functions could be chosen to be Lipschitz continuous, but his decomposition used $2n+1$ outer functions and $2n^2+n$ inner functions. Finally, Sprecher provided in 1972 a decomposition with Lipschitz continuous functions $x_i\mapsto\lambda^{i-1}\psi(x_i+j\epsilon)$. (See our paper for the references.) Approximation theory for deep ReLU networks One of the most important theoretical problems in deep network approximation theory is to determine why and when deep networks lessen or break the curse of dimensionality for multivariate continuous functions, characterized by the $\mathcal{O}(\epsilon^{-n})$ growth of the network size, as the error $\epsilon\rightarrow0$. In our paper, we use a variant of Sprecher's 1965 version of the theorem, which reads $$ f(x_1,\ldots,x_n) = \sum_{j=0}^{2n}\phi_j\left(\sum_{i=1}^n\lambda_i\psi(x_i+ja)\right), $$ for some constants $\lambda_1=1>\lambda_2>\ldots>\lambda_n$ and $a=[(2n+1)(2n+2)]^{-1}$, and with Hölder continuous inner function $\psi$. We first show that the outer functions may be approximated by Lipschitz continuous functions. Then, we prove that—for a particular subset of the continuous functions—the inner and outer functions can be approximated with error $\epsilon$ by deep networks of size $\mathcal{O}(\epsilon^{-\log n})$ and $\mathcal{O}(\epsilon^{-1/2})$. The resulting network that approximates $f$ has depth and size $\mathcal{O}(\epsilon^{-\log n})$; the curse of dimensionality is lessened. Check it out if you are interested!

The U.K. government's controversial plan of action https://hadrien-montanelli.github.io/2020-03-19.html 2020-03-19T00:00:00+00:00

I've been paying careful attention to the various government policies implemented in response to the coronavirus pandemic. Having many friends and colleagues in the U.K., I am particularly concerned with the British government's plan of action. The U.K. government's controversial plan of action At first, the U.K.'s plan differed from other countries' plans—the government imposing merely a few restrictions on the population, relying on so-called herd immunity. This was a terrible plan; for more information, see this article. Luckily, the government has now recognized the situation's alarming nature and has changed its approach, based on this report from Imperial College London. The current plan implements a strategy of social distancing, which is essential for avoiding the overload of the healthcare system. Following the presentation of the original plan based on herd immunity, I spoke to my friend who works for the U.K. government in London. She strongly believed that her government "knew better" and that its decisions were "based on science." Well, as a mathematician and scientist, this really puzzled me for three main reasons. First, the relevant science in this context (e.g., applied mathematics and applied epidemiology) cannot model the spread of the disease in an exact, foolproof way. Pure mathematics is an exact science, in the sense that everything can be either true or false. Here, what we have are scientific models, which utilize simple assumptions to try to represent reality. Those models often use data combined with numerical simulations to give us a snapshot of the future—i.e., a prediction. This process of modelling introduces many sources of error, such as: modelling errors (i.e., the model is a simplified version of the phenomenon we're describing, thereby potentially omitting significant factors), sample selection bias (i.e., the data is not representative of the population), numerical errors (i.e., the problem solved on the computer is de facto itself an approximation of the model), and software bugs. My point is that modelling gives us some indication of what the future may be, but yields far from perfect results. Second, research in Academia heavily relies on peer review. Once researchers believe they've come up with a new model and have obtained interesting results, they send their work to a scientific journal. This is the crucial step—where experts chosen by the journal validate or invalidate the model, check the data and verify the algorithms, comment on the results, make suggestions, ask questions, and ultimately decide whether the paper should or should not be published. The model and the numerical simulations that the U.K. government's original plan relied on were never published, even though Prof. Chris Witty said that they intended to publish. More than 500 academics criticized the plan in an open letter. To bypass peer review for one's model and simulations in this pressing time suggests complacency. And to withhold the paper itself from the public is arguably inconsiderate and irresponsible. A politician or an individual blindly believing in science is dangerous, particularly if those beliefs have policy-making implications—as in the current context. In essence, science should not be a dogma. Third, when the lives of a considerable part of the population are at risk (the worst-case scenario predicted by Imperial College approximates 500,000 deaths in the U.K.), shouldn't a government be especially cautious to protect its citizens? I would rather have a government that potentially overreacts in order to safeguard the health of its most vulnerable people. A simple model for the exponential spread of diseases I present in this section a very simple model for the spread of a disease and illustrate how social distancing helps to flatten the curve. This was inspired by this excellent article from the Washington Post. Here are the assumptions of my toy model: individuals are represented by points on a rectangular 2D domain; at regular time intervals, they move in a (normally distributed) random direction; healthy individuals become sick if they get close enough to sick individuals; once they're sick, they'll recover after some time; once they've recovered, they cannot get sick again or carry the disease. Following the Washington Post's article, social distancing is modelled by some random individuals being frozen, i.e., not moving around. These assumptions are indeed overly simplistic, but I guarantee you that even papers like Imperial College's report use relatively simple models. It's impossible to realistically represent all the interactions between people in a city/country. Of course, it'd be possible to get a more realistic model by, e.g., dividing the domain into subdomains with different densities of population or adding a probability of spreading/contracting the disease. Numerical experiments For my first experiment I use the following parameters: 100 individuals freely moving on a 10x10 2D domain; healthy individuals will contract the disease if within a 0.5 radius from a sick individual; they move at speed 5 in a random direction every 0.1 unit of time; they recover after 5 units of time; there is initially one sick individual. I plot below the number of cases versus time, illustrating the exponential growth of the spread of disease without social distancing. The following pictures show simulations ran with the same parameters as above but with 25 (top left), 50 (top right), 75 (bottom left), and 90 (bottom right) individuals adopting a strategy of social distancing. The graphs show how social distancing helps to flatten the curve, which is crucial for preventing the overload of the healthcare system. Spread the word, not the disease! Python code Finally, to be as transparent as possible—here is the code.

Exponential integrators for stiff PDEs https://hadrien-montanelli.github.io/2020-05-19.html 2020-05-19T00:00:00+00:00

One of the papers I wrote during my Ph.D. at Oxford with Niall Bootland was just accepted in Mathematics and Computers in Simulation. In this post, I review the main ideas of the paper. Periodic stiff PDEs We are interested in computing smooth solutions in a periodic domain in 1D/2D/3D of stiff PDEs of the form $$ u_t = \mathcal{L}u + \mathcal{N}(u), \quad u(0,\boldsymbol{x})=u_0(\boldsymbol{x}), $$ where $u(t,\boldsymbol{x})$ is a function of time $t$ and space $\boldsymbol{x}$, $\mathcal{L}$ is a linear differential operator with constant coefficients, and $\mathcal{N}$ is a nonlinear operator of lower order with constant coefficients. In applications, PDEs of this kind typically arise when two or more different physical processes are combined, and many PDEs of interest in science and engineering take this form. For example, the Korteweg–de Vries equation $u_t = -u_{xxx} - uu_x$, the starting point of the study of nonlinear waves and solitons, couples third-order linear dispersion with first-order convection, and the Allen–Cahn equation $u_t = \epsilon u_{xx} + u - u^3$ couples second-order linear diffusion with a nondifferentiated cubic reaction term. Often a system of equations rather than a single scalar equation is involved, for example in reaction-diffusion systems such as the Gray–Scott and Schnakenberg equations, which involve two components coupled together. (The importance of coupling of nonequal diffusion constants in science was made famous by Alan Turing in 1952 in the most highly-cited of all his papers.) Fourth-order terms also arise, for example in the Cahn–Hilliard equation, whose solutions describe structures of alloys, and in the Kuramoto–Sivashinsky equation, related to combustion problems among others, whose solutions are chaotic. The figure at the top of this post shows six examples of solutions of such PDEs. Discretization in Fourier space Our paper describes and compares specialized methods that take advantage of two special features of the family of PDEs above. The first one is the periodic boundary conditions. This allows us to discretize the spatial component with a Fourier spectral method on $N$ points; the PDE becomes a system of $N$ ODEs, $$ \widehat{u}' = \mathbf{L}\widehat{u} + \mathbf{N}(\widehat{u}), \quad \widehat{u}(0)=\widehat{u}_0, $$ where $\widehat{u}(t)$ is the vector of $N$ Fourier coefficients of the trigonometric interpolant of $u(t,X)$ at time $t$, and $\mathbf{L}$ (a $N\times N$ matrix) and $\mathbf{N}$ are the discretized versions of $\mathcal{L}$ and $\mathcal{N}$ in Fourier space. For example, in 1D on $[0, 2\pi]$ with $\mathcal{L}u=u_{xx}$ and an even number $N$ of equispaced grid points $\{x_j=2\pi j/N\}_{j=0}^{N-1}$, we look for a solution $u(t,x)$ of the form $$ u(t,x) \approx \sum_{k=-N/2}^{N/2}{\hspace{-0.3cm}}'{\;\,} \widehat{u}_k(t) e^{ikx} $$ with Fourier coefficients $$ \widehat{u}_k(t) = \frac{1}{N}\sum_{j=0}^{N-1}u(t,x_j)e^{-ikx_j},\;\; -\frac{N}{2}\leq k\leq \frac{N}{2}-1, $$ and with $\widehat{u}_{N/2}(t) = \widehat{u}_{-N/2}(t)$. (The prime on the summation sign signifies that the terms $k=\pm N/2$ are halved.) Since FFT codes only store $N$ coefficients, the vector $\widehat{u}(t)$ is defined as $$ \widehat{u}(t) = \Big(\frac{\widehat{u}_{-N/2}}{2}+\frac{\widehat{u}_{N/2}}{2}, \widehat{u}_{-N/2+1}(t),\ldots,\widehat{u}_{N/2-1}(t)\Big)^T. $$ For this PDE, $\mathbf{L}=\mathbf{D}_N^{(2)}$ is the (diagonal) second-order Fourier differentiation matrix with entries $-k^2$, $-N/2\leq k\leq N/2-1$. Note that stiffness is related to $\mathbf{L}$ having large eigenvalues since the stability of spectral methods for time-dependent PDEs requires that the eigenvalues, scaled by the time-step, lie in the stability region of the time-stepping formula. A simple exponential integrator The second special feature of these PDEs is that they are is semilinear, i.e., the higher-order terms of the equation are linear. Exponential integrators are a class of numerical methods for systems of ODEs that are aimed at taking advantage of this. The linear part $\mathbf{L}$, responsible for the stiffness, is integrated exactly using the matrix exponential while a numerical scheme is applied to $\mathbf{N}$. One of the simplest exponential integrators, commonly known as the Exponential Time Differencing (ETD) Euler method, is given by $$ \widehat{u}^{n+1} = e^{h\mathbf{L}}\widehat{u}^n + h\varphi_1(h\mathbf{L})\mathbf{N}(\widehat{u}^n), $$ where $h=t_{n+1}-t_n$ is the time-step and $$ \varphi_1(z) = \frac{e^z-1}{z}. $$ Note that, in practice, the nonlinear evaluations $\mathbf{N}(\widehat{u}^n)$ is carried out in value space, that is, $\mathbf{N}(\widehat{u}^n)$ means $\mathbf{F}(\mathbf{N}(\mathbf{F}^{-1}\widehat{u}^n))$, with discrete Fourier transform $\mathbf{F}$. Since the matrix $\mathbf{L}$ diagonal, computing the matrix exponential and matrix-vector products cost only $\mathcal{O}(N)$ operations. The dominant cost per time-step is therefore the cost of an FFT, i.e., $\mathcal{O}(N\log N)$ operations. Results of our numerical experiments and Chebfun We compare in our paper 30 exponential integrators of fourth and higher order on 11 model problems in 1D, 2D, and 3D. Our conclusion is that it is hard to do much better than one of the simplest of these formulas, the ETDRK4 scheme of Cox and Matthews. Our numerical experiments were performed using MATLAB and have been embedded within Chebfun. More specifically, the spin, spin2 and spin3 codes implement a Fourier spectral method and exponential integrators to solve PDEs in 1D, 2D, and 3D periodic domains. The simplest way to see spin in action is to type simply spin('ks') (for the Kuramoto–Sivashinsky equation) or spin2('gl') (for the 2D Ginzburg–Landau equation) to invoke an example computation. It is also possible to define your own PDE using the spinop class. Check out the documentation for details.

Chloroquine & COVID-19 https://hadrien-montanelli.github.io/2020-10-25.html 2020-10-25T00:00:00+00:00

Depuis mon retour en France, un sujet de conversation récurrent est bien évidemment la pandémie de la Covid-19. Et là, grosse surprise : beaucoup de gens autour de moi pensent toujours que l'hydroxychloroquine est, ou pourrait être, un médicament efficace contre la Covid-19. Ce qui est surprenant, c'est qu'aux États-Unis, la folie derrière l'hydroxychloroquine est terminée ! Mais pas en France. Contexte Pour faire vite, l'hydroxychloroquine en France c'est un peu comme le réchauffement climatique aux États-Unis. D'un côté, il y a la (très grande) majorité des scientifiques, et les gens qui font confiance à la science. En France, ce sont ceux qui ont compris que l'hydroxychloroquine ne marche pas ; aux États-Unis, ce sont ceux pour lesquels le réchauffement climatique est bien réel. De l'autre, une (minuscule) minorité de scientifiques, dont le Professeur Raoult, les gens qui se méfient de la science, et les adeptes de la théorie du complot [1]; ce sont ceux qui, en France, croient en l'efficacité de l'hydroxychloroquine et, aux États-Unis, doutent du réchauffement climatique. Au milieu de ces deux groupes, il y a le reste : ceux qui écoutent les médias grand public et ne croient pas aux théories du complot, mais qui ne lisent pas forcément les médias anglo-saxons et les revues scientifiques spécialisées. Ce qui est surprenant c'est, qu'en France, les gens que j'ai rencontrés dans ce groupe-là sont plutôt optimistes vis-à-vis de l'hydroxychloroquine, alors que cela fait des mois que l'hydroxychloroquine est enterrée. Bon, disons-le une bonne fois pour toute : le réchauffement climatique est bien réel et l'hydroxychloroquine ne marche pas. Pour le réchauffement climatique, il y a le site de la NASA. Pour l'hydroxychloroquine, il y a, à ce jour, au moins trois articles qui présentent des essais cliniques de grande ampleur qui l'ont prouvé. Les articles que je citerai ci-dessous ont été largement relayés dans les médias aux États-Unis, et probablement beaucoup moins en France. J'ai lu ces articles en détail [2]. Je présenterai les résultats dans la dernière partie, mais d'abord, essayons de comprendre ce qu'il s'est passé. Comment en est-on arrivé là ? Tout a commencé avec cet article d'un groupe de recherche en Chine, publié le 4 Février 2020. L'article présente des expériences in vitro (en laboratoire) pour lesquelles l'hydroxychloroquine et le remdésivir bloquent le SARS-CoV-2, le coronavirus responsable de la Covid-19. Ce type d'expériences consiste à utiliser des cellules (non humaines) en laboratoire, type Vero, à y injecter le virus, la substance active du médicament à tester, et à mesurer le taux de reproduction du virus après 48h. Ces tests-là ont été concluants pour l'hydroxychloroquine et le remdésivir, ce qui a stimulé la recherche in vivo (sur des patients) pour les deux. À ce stade, tout ce qui est prouvé, c'est que l'hydroxychloroquine et le remdésivir sont efficaces quand le SARS-CoV-2 est injecté en laboratoire dans des populations de cellules qui proviennent de cellules de rein extraites d'un singe vert africain. Bref, on est très loin d'un médicament efficace dans le corps humain. Ensuite, il y a eu cet article du Professeur Raoult et de son équipe, en ligne depuis le 17 Mars 2020 et publié en Juillet 2020. C'est un essai de très petite taille (20+16 participants), sur des patients hospitalisés, dont le résultat est un test PCR chaque jour lors des six jours de soin. Les patients sont relativement jeunes, présentent des cas peu graves, et ont tous survécu [3]. Les résultats sont répertoriés dans le tableau ci-dessous. Groupe avec hydroxychloroquine (N=20) Groupe contrôle (N=16) Âge 51.2 ± 18.7 37.3 ± 24.0 Sexe 45.0% H 37.5% H Ethnicité Non renseigné Non renseigné Antécédents Non renseigné Non renseigné Négatif à 6 jours 100% 12.5% Ce sont des résultats encourageants, mais qui ont de nombreuses limites ; l'essai du Professeur Raoult ne suit pas la méthode scientifique qui permet de garantir l'efficacité d'un médicament, que j'explique brièvement maintenant. Comment tester l'efficacité d'un médicament ? D'abord, il faut utiliser un grand nombre de patients pour réduire au plus possible le facteur chance, au moins 500, et généralement plusieurs milliers. Ensuite, il faut diviser ces patients en deux groupes aux caractéristiques similaires (âge, sexe, ethnicité et antécédents médicaux) : un groupe auquel sera donné le médicament, en plus des soins habituels, et un autre groupe qui ne recevra que les soins habituels. Enfin, pour éviter l'effet placebo, il est essentiel que les patients ne sachent pas si le médicament leur a été administré. Les essais cliniques vont même plus loin, et sont « doublement aveugles » : les équipes médicales qui apportent les soins habituels aux patients ne savent pas non plus si le patient a reçu le médicament, l'idée étant de se défaire de tous les biais possibles. On voit bien ici les limites des essais de l'équipe du Professeur Raoult : nombre de patients bien trop faible, groupes aux caractéristiques différentes, manques d'informations sur les patients (ethnicité et antécédents médicaux), et pas de « double aveugle ». En bref, l'essai clinique du Professeur Raoult n'a aucune valeur scientifique. Trois essais cliniques qui enterrent l'hydroxychloroquine Les deux premiers essais sont des essais de grande taille (1561+3155 et 947+906 participants), sur des patients hospitalisés, dont le résultat est le taux de mortalité à 28 jours. Le premier est l'essai Recovery mené par l'Université d'Oxford et se concentre sur des cas graves. Le second est l'essai Solidarity de l'Organisation mondiale de la Santé et s'intéresse à des cas moins graves. Le tableau ci-dessous regroupe les résultats de l'essai Recovery d'Oxford, présentés dans cet article, en ligne depuis le mois de Juillet 2020, et publié le 8 Octobre 2020. Groupe avec hydroxychloroquine (N=1561) Groupe contrôle (N=3155) Âge moyen 65.2 ± 15.2 65.4 ± 15.4 Sexe 61.5% H 62.6% H Ethnicité 75.7% Blanc 72.8% Blanc Antécédents 56.5% 57.3% Mortalité sous 28 jours 27.0% 25.0% Le traitement à base d'hydroxychloroquine ne permet pas de baisser le taux de mortalité. Regardons maintenant un autre tableau, qui détaille les résultats de l'essai Solidarity de l'OMS, exposés dans cet article en ligne depuis le 15 Octobre 2020, dont on attendait les résultats depuis le mois de Mai, lorsque l'OMS avait annoncé qu'elle stoppait les essais sur l'hydroxychloroquine, décision confirmée au mois de Juillet. Groupe avec hydroxychloroquine (N=947) Groupe contrôle (N=906) Âge > 50 64.6% 65.0% Sexe 60.6% H 59.1% H Ethnicité 59.6% Afrique/Asie 59.9% Afrique/Asie Antécédents 53.9% 57.9% Mortalité sous 28 jours 11.0% 9.27% Encore une fois, le traitement à base d’hydroxychloroquine ne permet pas de baisser le taux de mortalité. Le troisième et dernier essai est un essai de moyenne taille (414+407), sur des patients « contact » négatifs au début de l'essai et non hospitalisés, dont le but est de tester si l'hydroxychloroquine influence la contraction de la Covid-19. Le résultat est un test PCR après 14 jours. L'essai a été mené aux États-Unis et au Canada. Les résultats ci-dessous proviennent de cet article, publié le 3 Juin 2020. Groupe avec hydroxychloroquine (N=414) Groupe contrôle (N=407) Âge médian 41 40 Sexe 52.7% F 50.6% F Ethnicité Non renseigné Non renseigné Antécédents 26.1% 28.7% Positif à 14 jours 11.8% 14.3% Le traitement à base d’hydroxychloroquine n'a pas d'influence statistiquement significative sur la contraction de la maladie. Conclusion Après plusieurs mois de recherche intensive, les scientifiques se sont mis d'accord : l'hydroxychloroquine ne permet pas de soigner ou de prévenir la Covid-19. D'ailleurs, son autorisation de mise en urgence sur le marché a été retirée dès la fin du mois de Mai 2020 dans l'Union Européenne, au Royaume-Uni et aux États-Unis. J'étais aux États-Unis jusqu'au début de l'automne, et j'avoue que la passion française derrière l'hydroxychloroquine est toujours un mystère pour moi. Les seules explications que je vois sont (1) le manque de visibilité des médias anglo-saxons et des revues scientifiques spécialisées (qui sont exclusivement en anglais) en France, (2) une sorte de patriotisme qui a poussé l'opinion publique à défendre de façon irrationnelle un chercheur Français, et (3) une défiance vis-à-vis du gouvernement et de la science. Notes [1] Le Professeur Raoult est devenu une figure centrale des théories complotistes. Il a notamment cité plusieurs fois des conflits d'intérêt au sein du Conseil scientifique Covid-19 pour justifier les actions du gouvernement contre l'hydroxychloroquine. Selon une des théories, vu que l'hydroxychloroquine est bon marché et existe déjà, les laboratoires privés pourraient se faire beaucoup plus d'argent en vendant un nouveau médicament qu'ils développeraient spécifiquement pour la Covid-19. [2] Je suis chercheur en mathématiques appliquées. J'utilise des méthodes mathématiques pour résoudre des problèmes dans des domaines variés dont, par example, la physique, la biologie et l'intelligence artificielle. Lire des articles scientifiques en anglais qui ne sont pas forcément dans mon domaine de prédilection est mon métier. [3] Il est indiqué dans l'article que six patients qui avaient commencé le traitement avec l'hydroxychloroquine ont finalement été retirés de l'essai ; l'un d'entre eux est décédé.

Quantum computing 1 https://hadrien-montanelli.github.io/2020-11-11.html 2020-11-11T00:00:00+00:00

In my first research project as an undergraduate student in the summer of 2010, I worked on groups and symmetries in classical and quantum physics. One day in the library I came across Noether's theorem, which links symmetries (e.g., time invariance) to conserved quantities (e.g., energy). Noether was one of the leading mathematicians of her time and made major contributions to both algebra and mathematical physics. Learning about her work was a decisive moment in my life because I realized for the first time how deeply connected mathematics and physics were. Thanks to Noether, I got really interested in mathematical physics and took classes in statistical mechanics and special relativity. My second research project as an undergraduate student was on Riemannian geometry and general relativity, with my friend and former classmate Olivier Martre—oh dear, how hard it was! A couple of years down the road, I had to choose between an M.Sc. in applied mathematics and an MS.c. in mathematical physics. I ended pursuing a career in applied mathematics, but physics never really left my life—most of my work in applied mathematics has been on the numerical solution of the differential equations of physics. In particular, during my Ph.D. at Oxford, I built a partial differential equation solver called spin; the guide can be found here. While it stands for "stiff PDEs integrator", I named it after the quantum mechanical spin. The spin code is part of the MATLAB-based package Chebfun, a project led by my former Ph.D. advisor Nick Trefethen. Quantum states and observables In quantum physics, the state of a physical system is described by its wave function $\psi$, an element of a separable complex Hilbert space $X$ [1]. Physical quantities are represented by self-adjoint operators $L:X\rightarrow X$ acting on $\psi$, called observables. Note that $X$ may or may not be finite-dimensional; in finite dimension, $L$ is a Hermitian matrix. Examples of operators include the position operator (position), the momentum operator (momentum), the spin angular momentum operator (spin) and the Hamiltonian operator (total energy). Let us first look at a finite-dimensional example by considering the spin operator along the $y$-axis for spin-$\frac{1}{2}$ particles, such as the electron. In this case, the quantum state $\psi=(\psi_1,\psi_2)^T$, with respect to $y$-spin, lives in is the Hilbert space $X=\mathbb{C}^2$ of pairs of complex numbers, with inner product $$ \langle\psi,\phi\rangle = \overline{\psi}_1\phi_1 + \overline{\psi}_2\phi_2, $$ and the spin operator along the $y$-axis is the matrix $L=S_y$ given by $$ S_y = \frac{\hbar}{2}\begin{pmatrix} 0 & -i \\ i & 0 \end{pmatrix}. $$ Since it is a Hermitian matrix, the spectral theorem guarantees that it is diagonalizable with orthonormal eigenvectors and real eigenvalues. The eigenvalues are $\pm\frac{\hbar}{2}$, while the eigenvectors are given by $$ \psi_{y+}=\frac{1}{\sqrt{2}}\begin{pmatrix} 1 \\ i \end{pmatrix}, \quad \psi_{y-}=\frac{1}{\sqrt{2}}\begin{pmatrix} 1 \\ -i \end{pmatrix}. $$ The quantum state of a spin-$\frac{1}{2}$ particle, with respect to $y$-spin, may then be written as $$ \psi = \langle\psi_{y+},\psi\rangle\psi_{y+} + \langle\psi_{y-},\psi\rangle\psi_{y-}, $$ with the normalization condition $$ \langle\psi,\psi\rangle = \vert\langle\psi_{y+},\psi\rangle\vert^2 + \vert\langle\psi_{y-},\psi\rangle\vert^2 = 1. $$ When the $y$-spin is measured, the probability that it will be $\pm\frac{\hbar}{2}$ is $\vert\langle\psi_{y\pm},\psi\rangle\vert^2$. Following the measurement, $\psi$ will collapse into the corresponding eigenstate. Note that the expected value of $\psi$ is given by $$ \langle\psi\rangle = \frac{\hbar}{2}\vert\langle\psi_{y+},\psi\rangle\vert^2 - \frac{\hbar}{2}\vert\langle\psi_{y-},\psi\rangle\vert^2. $$ Let us now look at an example in infinite dimension by considering the Hamiltonian operator of a particle of mass $m$ in a quadratic potential $V(x)=\frac{1}{2}m\omega^2x^2$, the so-called quantum harmonic oscillator with frequency $\omega$. In this case, the quantum state $\psi$, with respect to total energy, belongs to the Hilbert space $X=L^2(\mathbb{R})$ of square-integrable functions $\psi$, with inner product $$ \langle\psi,\phi\rangle = \int_{\mathbb{R}}\overline{\psi}(x)\phi(x)dx, $$ and the Hamiltonian operator is the operator $L=H$ defined by $$ H\psi(x) = -\frac{\hbar}{2m}\psi''(x) + \frac{1}{2}m\omega^2x^2\psi(x), \;\;x\in\mathbb{R}, $$ with domain [2] $$ D(H) = \big\{\psi\in H^2(\mathbb{R})\,|\,x^2\psi\in L^2(\mathbb{R})\big\}\subset L^2(\mathbb{R}). $$ It is an example of an unbounded operator, since the elements of its spectrum have unbounded amplitude. It is diagonalizable with orthonormal eigenfunctions and a discrete real spectrum [3]. The countably many eigenvalues are given by $$ \lambda_n = (2n+1)\frac{\hbar}{2}\omega, \quad n=0,1,2,\ldots, $$ while the eigenfunctions are scaled Hermite functions $\psi_n$. Therefore, one may write the quantum state of the particle, with respect to total energy, as $$ \psi(x) = \sum_{n=0}^{\infty}\langle\psi_n,\psi\rangle\psi_n(x), $$ with the normalizaton condition $$ \langle\psi,\psi\rangle = \sum_{n=0}^{\infty}\vert\langle\psi_n,\psi\rangle\vert^2 = 1. $$ As in the finite-dimensional case, the meaning of the decomposition in the basis of eigenfunctions is that if we perform a total energy measurement, the state will have total energy $\lambda_n$ with probability $\vert\langle\psi_n,\psi\rangle\vert^2$, and following the measurement, the state will collapse into the corresponding eigenstate. Here, the expected value of $\psi$ is given by $$ \langle\psi\rangle = \sum_{n=0}^{\infty}\lambda_n\vert\langle\psi_n,\psi\rangle\vert^2. $$ In Chebfun, quantum states can be computed with the quantumstates command; see, e.g., this example. Bit versus qubit Classical computers use bits to store information, which are commonly represented as "0" and "1". The way I like to think about bits is that each bit is like a small magnet that can either point North ("0") or South ("1"). By applying an appropriate magnetic field, one can change the magnetization of a bit, and change "0" to "1", or vice versa. Bits are then combined together into arrays of bits, called words. Numbers, for example, are stored as 32- or 64-bit words, following the IEEE 754 technical standard. Quantum computers use qubits. A qubit is a two-state quantum-mechanical system, for example, the $y$-spin of the electron described in the previous section. In that scenario, a qubit would be $$ \psi = \alpha\psi_{y+} + \beta\psi_{y-}, $$ where $\psi_{y+}$ corresponds to "0" and $\psi_{y-}$ to "1". As in the classical case, one can change the value of a qubit by applying an appropriate magnetic field. Note that a qubit does not have a value in-between "0" and "1"; rather, as explained previously, when measured, it has a probability $\vert\alpha\vert^2$ of the value "0" and a probability $\vert\beta\vert^2$ of the value "1". In the next post, I will talk about the Schrödinger equation and quantum logic gates. [1] To be more precise, one has to consider equivalent classes of elements in $H$ called rays, with $\psi\sim\phi$ when $\phi=\lambda\psi$ for some non-zero $\lambda\in\mathbb{C}$. [2] $H^2(\mathbb{R})$ denotes the usual Sobolev space of functions in $L^2(\mathbb{R})$ whose weak derivatives up to order two have a finite $L^2(\mathbb{R})$-norm. [3] Self-adjoint unbounded operators can always be diagonalized in some sense but, in general, the spectrum has both a discrete and a continuous part; see, e.g., this.

Quantum computing 2 https://hadrien-montanelli.github.io/2020-11-17.html 2020-11-17T00:00:00+00:00

Episode 2 of my series on quantum computing—in this post, I talk about the Schrödinger equation and quantum logic gates, which are the building blocks of quantum circuits and quantum computers. Schrödinger equation Let $\psi$ describe the quantum state of a physical system. As explained in my previous post, $\psi$ may be a vector or a function. The Schrödinger equation tells us that the time evolution of $\psi$ is given by $$ i\hbar\frac{\partial}{\partial t}\psi(t) = H\psi(t), $$ where $H$ is the Hamiltonian operator of the system. If $H$ is time-independent, then $$ \psi(t) = \exp\left(-\frac{i}{\hbar}tH\right)\psi(0). $$ Note that since $H$ is a self-adjoint operator, the operator $\exp\left(-\frac{i}{\hbar}tH\right)$ is a unitary operator. A consequence of the Schrödinger equation is that starting from a quantum state $\psi_1$, it is possible to obtain a specific quantum state $\psi_2$ by evolving the system with the appropriate Hamiltonian for a suitable period of time. Qubits In my previous post, I introduced qubits as two-state quantum-mechanical systems and gave the $y$-spin of the electron as an example. In practice, qubits actually correspond to the $z$-spin of the electron, which I shall quickly review now. The quantum state $\psi=(\psi_1,\psi_2)^T$ of an electron, with respect to $z$-spin, lives in is the Hilbert space $\mathbb{C}^2$ of pairs of complex numbers, with inner product $$ \langle\psi,\phi\rangle = \overline{\psi}_1\phi_1 + \overline{\psi}_2\phi_2, $$ and the spin operator along the $z$-axis is the Hermitian matrix $$ S_z = \frac{\hbar}{2}\begin{pmatrix} 1 & 0 \\ 0 & -1 \end{pmatrix}. $$ The eigenvalues are $\pm\frac{\hbar}{2}$, while the eigenvectors are given by $$ \vert0\rangle=\begin{pmatrix} 1 \\ 0 \end{pmatrix}, \quad \vert1\rangle=\begin{pmatrix} 0 \\ 1 \end{pmatrix}. $$ The quantum state of the electron, with respect to $z$-spin, may then be written as $$ \psi = \psi_1\vert0\rangle + \psi_2\vert1\rangle. $$ This is the most general form of a qubit. It includes not only the $\vert0\rangle$ and $\vert1\rangle$ qubits, the quantum equivalent of the classical "0" and "1" bits, but also superpositions. Quantum logic gates Suppose the electron is in the state $\vert0\rangle$ and we want to change it to $\vert1\rangle$. Since the spin Hamiltonian is related to the magnetic field, using the Schrödinger equation, we can obtain $\vert1\rangle$ from $\vert0\rangle$ by applying an appropriate magnetic field for a suitable period of time. Mathematically, this means multiplying $\vert0\rangle$ by an appropriate unitary matrix. This is what quantum logic gates do—these are unitary matrices that act on qubits and correspond to the application of a specific magnetic field for a certain period of time. Quantum logic gates abstract the underlying physics away. For example, the quantum NOT gate corresponds to the following $X$ Pauli matrix, $$ X = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}, $$ and has the following NOT-like behavior, $$ X\vert0\rangle = \vert1\rangle, \quad X\vert1\rangle = \vert0\rangle. $$ It is the quantum analog of the classical NOT gate. The Hadamard gate, which is one of the most important quantum gates, is defined by, $$ H = \frac{1}{\sqrt{2}}\begin{pmatrix} 1 & 1 \\ 1 & - 1 \end{pmatrix}. $$ It has a more sophisticated behavior since it puts the basis states into a superposition, $$ \begin{align} & H\vert0\rangle = \frac{1}{\sqrt{2}}(\vert0\rangle + \vert1\rangle), \\ & H\vert1\rangle = \frac{1}{\sqrt{2}}(\vert0\rangle - \vert1\rangle). \end{align} $$ Tensor products The quantum states of two electrons, with respect to $z$-spin, may be described in the tensor product basis as $$ \begin{align} \psi = \psi_1\vert0\rangle\otimes\vert0\rangle + \psi_2\vert0\rangle\otimes\vert1\rangle + \psi_3\vert1\rangle\otimes\vert0\rangle + \psi_4\vert1\rangle\otimes\vert1\rangle, \end{align} $$ where the tensor product basis vectors are given by $$ \begin{align} & \vert0\rangle\otimes\vert0\rangle=\begin{pmatrix} 1 \\ 0 \\ 0 \\ 0 \end{pmatrix}, \quad \vert0\rangle\otimes\vert1\rangle=\begin{pmatrix} 0 \\ 1 \\ 0 \\ 0 \end{pmatrix}, \\\\ & \vert1\rangle\otimes\vert0\rangle=\begin{pmatrix} 0 \\ 0 \\ 1 \\ 0 \end{pmatrix}, \quad \vert1\rangle\otimes\vert1\rangle=\begin{pmatrix} 0 \\ 0 \\ 0 \\ 1 \end{pmatrix}. \end{align} $$ It is possible to define quantum logic gates for systems of electrons as well. For instance, the qantum controlled NOT gate is given by the following $C_X$ matrix, $$ C_X = \begin{pmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 1 & 0 \end{pmatrix}. $$ Its action on the basis vectors is as follows, $$ \begin{align} & C_X\big(\vert0\rangle\otimes\vert0\rangle\big) = \vert0\rangle\otimes\vert0\rangle, \\[1.5pt] & C_X\big(\vert0\rangle\otimes\vert1\rangle\big) = \vert0\rangle\otimes\vert1\rangle, \\[1.5pt] & C_X\big(\vert1\rangle\otimes\vert0\rangle\big) = \vert1\rangle\otimes\vert1\rangle, \\[1.5pt] & C_X\big(\vert1\rangle\otimes\vert1\rangle\big) = \vert1\rangle\otimes\vert0\rangle. \end{align} $$ There are many more quantum gates, including the Pauli-$Y$ and Pauli-$Z$ gates, and the SWAP gate. Quantum gates are then combined into sequences to form quantum circuits. In the next post, I will talk about using quantum computing for solving nonlinear ordinary differential equations.

Quantum computing 3 https://hadrien-montanelli.github.io/2020-11-28.html 2020-11-28T00:00:00+00:00

In this post, I talk about quantum computing for solving nonlinear ordinary differential equations (ODEs). In a recent paper, Michael Lubasch and his colleagues proposed an algorithm to solve such equations using a combination of classical and quantum computers. The main ideas are: reformulate the problem as an optimization one; use hybrid systems of quantum and classical computers to solve it. The cost function and its gradient are typically evaluated on a quantum computer, while the higher-level iterative algorithm is performed on a classical computer. Setup The authors consider, as an example, the following nonlinear Schrödinger ODE eigenvalue problem, $$ \left[-\frac{1}{2}\frac{d^2}{dx^2} + V(x) + g\vert f(x)\vert^2\right]f(x) = \lambda f(x), $$ for $x\in[a,b]$, and with $f(a)=f(b)$ and $\int_a^b\vert f(x)\vert^2dx=1$. They are only interested in the ground state of the problem, i.e., the function $f$ that corresponds to the minimum eigenvalue $\lambda_{\min}$. The nonlinear ODE eigenvalue problem reduces to a nonlinear ODE, which can be solved on a classical computer with an iterative method such as Newton's method, combined with, e.g., finite differences or a spectral method. Method The idea in the paper is to, instead, write the problem as an optimization one. They use the fact that the ground state minimizes the following cost function, $$ \begin{align} J(f) = \int_a^b\bigg[&\frac{1}{2}\vert f'(x)\vert^2+V(x)\vert f(x)\vert^2 +\frac{g}{2}\vert f(x)\vert^4\bigg]dx. \end{align} $$ They discretize $J$ with the trapezoidal rule and finite differences on a uniform grid $x_k=a+k(b-a)/N$, $0\leq k\leq N-1$. The cost function $J$ becomes a function of the $N$ scaled values $\psi_k=\sqrt{(b-a)/N}f(x_k)$. Then, they encode the $N=2^n$ amplitudes $\psi_k$ in the normalized state $$ \vert\psi\rangle = \sum_{k=0}^{N-1}\psi_k\vert\mathrm{binary}(k)\rangle, $$ where $\mathrm{binary}(k)$ denotes the binary representation of the integer $k=\sum_{j=1}^nq_j2^{n-j}$. Note that the above representation uses $n$ qubits and basis states $\vert\boldsymbol{q}\rangle=\vert q_1\ldots q_n\rangle$. Their hybrid algorithm goes like this. First, the quantum register is put in a given state $\vert\psi(\boldsymbol{\theta})\rangle=U(\boldsymbol{\theta})\vert\boldsymbol{0}\rangle$ via a quantum circuit $U(\boldsymbol{\theta})$ of depth $d=\mathcal{O}(\mathrm{poly}(n))$, such that their quantum ansatz requires $\mathcal{O}(nd)=\mathcal{O}(\mathrm{poly}(n))$ parameters, exponentially fewer parameters than classical schemes—this is an example of quantum supremacy. Second, a quantum nonlinear processing unit (QNPU) transforms the state $\vert\psi(\boldsymbol{\theta})\rangle$ into a state $\vert\psi_J(\boldsymbol{\theta})\rangle$ for which it is possible to measure a quantity $\langle M_J(\boldsymbol{\theta})\rangle$ that is directly proportional to the cost function $J(\boldsymbol{\theta})$. Third, the evaluation of $J(\boldsymbol{\theta})$ and the optimization over the parameters $\boldsymbol{\theta}$ are carried out on a classical computer. Theoretical justification The quantum ansatz is based on matrix product states (MPS); for details, see, e.g., this. The key feature is that any state $\psi$ can be written as $$ \vert\psi\rangle = \sum_{q_1,\ldots,q_n\in\{0,1\}}\mathrm{tr}\left[A^{(1)}_{q_1}\ldots A^{(n)}_{q_n}\right]\vert q_1\ldots q_n\rangle, $$ where $A^{(k)}_0$ and $A^{(k)}_1$ are $D_k\times D_{k+1}$ matrices, for sufficiently large $D_1,\ldots,D_{n+1}$. The bond dimension of such a representation is $\chi=\max_{1\leq k\leq n+1} D_k$ and, in general, $\chi=\mathcal{O}(\mathrm{exp}(n))$. States are referred to as MPS when $\chi$ is not too big, typically when $\chi=\mathcal{O}(\mathrm{poly}(n))$. Note that MPS require $\mathcal{O}(n\chi^2)=\mathcal{O}(\mathrm{poly}(n))$ parameters ($n$ matrices of size at most $\chi$). The power of their quantum ansatz is that it contains all MPS with bond dimension $\chi=\mathcal{O}(\mathrm{poly}(n))$. Since Chebyshev polynomials can be approximated exponentially well by MPS, MPS with such a $\chi$ contain all polynomials of degree up to $\mathcal{O}(\mathrm{exp}(\chi))=\mathcal{O}(N)$, as shown by Khoromskij here. Classical MPS versus quantum ansatz One could use a classical MPS-based method for a fairer comparison. To show the advantage of their quantum ansatz over classical MPS, they consider potentials of the form $$ V(x) = s_1\sin(\kappa_1 x) + s_2\sin(\kappa_2 x), $$ with $\kappa_2=2\kappa_1/(1+\sqrt{5})$, and for various values of $s_1/s_2$ and $\kappa_1$. They show, both theoretically and numerically, that, in the $s_1/s_2\approx1$ regime, the number of parameters for classical MPS scales like $\mathcal{O}(\mathrm{poly}(\kappa_1))$, while it scales like $\mathcal{O}(\log\kappa_1)$ for their quantum ansatz—quantum supremacy is achieved once again. In the next and final episode of my series on quantum computing, I will talk about quantum computing for supervised learning.

Quantum computing 4 https://hadrien-montanelli.github.io/2020-12-06.html 2020-12-06T00:00:00+00:00

In this post, I talk about quantum computing for supervised learning. Marcello Benedetti and his colleagues wrote a very nice review last year, which I highly recommend. The main ideas are similar to those described in my previous post about quantum computing for ODEs: reformulate the problem as an optimization one; use hybrid systems of quantum and classical computers to solve it. Setup The core idea in supervised learning goes like this. We consider data $\mathcal{X}\subseteq\mathbb{R}^d$, labels $\mathcal{Y}\subseteq\mathbb{R}$ and a classifier $f:\mathcal{X}\mapsto\mathcal{Y}$. We want to approximate $f$ by $\widehat{f}$ using $n$ labeled data $(\boldsymbol{x}_1,y_1),\ldots,(\boldsymbol{x}_n,y_n)\in\mathcal{X}\times\mathcal{Y}$. The approximate classifier $\widehat{f}$ is often parametrized by a vector of parameters $\boldsymbol{\theta}$, and the training process consists in minimizing some cost function $J(\boldsymbol{\theta})$. They review two methods for quantum supervised machine learning, quantum kernel estimator (QKE) and variational quantum model (VQM). In both methods, the training set is mapped to feature space with a used-defined map $\phi$ during the pre-processing stage. Method 1 (QKE) First, the data $\phi(\boldsymbol{x}_i)$ is encoded in a quantum circuit by applying some operator $U_{\phi(\boldsymbol{x}_i)}$. Second, a SWAP test is used to evaluate the kernel. Third, the evaluation of $J(\boldsymbol{\theta})$ and the optimization over the parameters $\boldsymbol{\theta}$ are carried out on a classical computer. A typical example of such a $J(\boldsymbol{\theta})$ would be the following quantum support-vector machines cost function, $$ J(\boldsymbol{\theta}) = \frac{1}{2}\sum_{i=1}^n\sum_{j=1}^n\theta_i\theta_jy_iy_j\langle K(\boldsymbol{x}_j,\boldsymbol{x}_i)\rangle-\sum_{i=1}^n\theta_i. $$ Method 2 (VQM) The first step is the same as in QKE. Second, a variational circuit $U(\boldsymbol{\theta})$ implements the forecasting model. This yields state $\vert\psi_J(\boldsymbol{\theta})\rangle$ for which it is possible to measure a quantity $\langle M_J(\boldsymbol{\theta})\rangle$ that is directly related to the forecast. Third, the evaluation of both the forecast $\widehat{f}(\boldsymbol{x}_i;\langle M_J(\boldsymbol{\theta})\rangle)$ and $J(\boldsymbol{\theta})$, which measures the forecasting error, as well as the optimization over $\boldsymbol{\theta}$ are carried out on a classical computer. An example of a cost function $J(\boldsymbol{\theta})$ would be $$ J(\boldsymbol{\theta}) = \frac{1}{n}\sum_{i=1}^n\left(\widehat{f}(\boldsymbol{x}_i;\langle M_J(\boldsymbol{\theta})\rangle)-y_i\right)^2. $$ Encoder circuit The encoder circuit $U_{\phi(\boldsymbol{x})}$ can be seen as an extra layer of feature-mapping, where the inner product defines a kernel $$ K(\boldsymbol{x}_i,\boldsymbol{x}_j)=\vert\langle\boldsymbol{0}\vert U^{*}_{\phi(\boldsymbol{x}_j)}U_{\phi(\boldsymbol{x}_i)}\vert\boldsymbol{0}\rangle\vert^2. $$ As I mentioned above, the SWAP test can then be used to estimate $\langle K(\boldsymbol{x}_i,\boldsymbol{x}_j)\rangle$. Variational circuit In practice, the gate structure is fixed; only the gate parameters vary. A simple example of gate parametrization is given by $$ U(\boldsymbol{\theta}) = \prod_{\ell=1}^L\exp\left(-\frac{i}{2}\theta_\ell P_\ell\right), $$ where $P_\ell\in\{I,Z,X,Y\}^{\otimes n}$ is a tensor of $n$ Pauli matrices. It is crucial to efficiently encode the variational circuit. A fixed gate structure allows for the number of gate parameters to grow polynomially in $n$. They emphasize on two key differences between variational circuits and neural networks. The first one is that quantum circuit operations are linear, as opposed to the nonlinear activation functions used in neural networks. The second one is that it is not possible to access the quantum state during the computation, which makes it hard to design a quantum analog of backpropagation. Circuit learning Circuit learning relies on standard optimization procedures, which are carried out on the classical computer. For simple gates such as the Pauli gates presented above, it is possible to derive a closed-form expression for the gradient with respect to $\boldsymbol{\theta}$.

Weakly singular integrals over curved elements https://hadrien-montanelli.github.io/2021-11-25.html 2021-11-25T00:00:00+00:00

In this post, I talk about a new method for computing weakly singular and near-singular integrals that arise when solving the 3D Helmholtz equation with curved boundary elements. Background on integral equations The Helmholtz equation $\Delta u + k^2u = 0$ in the presence of an obstacle may be rewritten as an integral equation on the obstacle's boundary via layer potentials. For example, the radiating solution to the Dirichlet problem $\Delta u + k^2u = 0$ in $\mathbb{R}^3\setminus\overline{\Omega}$ with $u = u_D$ on the boundary $\Gamma=\partial\Omega$, for some bounded $\Omega$ whose complement is connected, can be obtained via the equation $$ \int_{\Gamma}G(\boldsymbol{x},\boldsymbol{y})\varphi(\boldsymbol{y})d\Gamma(\boldsymbol{y}) = u_D(\boldsymbol{x}), \quad \boldsymbol{x}\in\Gamma, $$ based on the single-layer potential; $G$ is the Green's function of the Helmholtz equation in 3D, $$ G(\boldsymbol{x},\boldsymbol{y}) = \frac{1}{4\pi}\frac{e^{ik\vert\boldsymbol{x}-\boldsymbol{y}\vert}}{\vert\boldsymbol{x}-\boldsymbol{y}\vert}. $$ Once the equation is solved for $\varphi$, which is unique if $k^2$ is not an eigenvalue of $-\Delta$ in $\Omega$, the solution $u$ may be represented by the left-hand side for all $\boldsymbol{x}\in\mathbb{R}^3\setminus\Omega$. This technique is not limited to the Helmholtz equation, Maxwell's equations and elasticity problems can be reformulated as integral equations as well. Challenges From a numerical point of view, integral equations are particularly challenging for several reasons. First, when $\boldsymbol{x}$ approaches $\boldsymbol{y}$, the integral becomes singular and standard quadrature schemes fail to be accurate—analytic integration or carefully-derived quadrature formulas are, hence, required. Second, the resulting linear systems after discretization are often dense. For large wavenumbers $k$, only iterative methods can be used to solve them (with the help of specialized techniques to accelerate the matrix-vector products, such as the Fast Multipole Method or hierarchical matrices). In this respect, the use of higher-order numerical discretization schemes may be helpful in enlarging the interval of feasible wavenumbers. Computing weakly singular integrals When solving boundary integrals equations with curved boundary elements, one has to compute integrals of the form $$ I(\boldsymbol{x}_0) = \int_\mathcal{T}\frac{\varphi(F^{-1}(\boldsymbol{x}))}{\vert\boldsymbol{x} - \boldsymbol{x}_0\vert}dS(\boldsymbol{x}), $$ where $\mathcal{T}$ is a curved triangle defined by a polynomial transformation $F:\widehat{T}\to\mathcal{T}$ of degree $q\geq1$ from some flat reference triangle $\widehat{T}$, $\boldsymbol{x}_0$ is a point on or close to $\mathcal{T}$, and $\varphi:\widehat{T}\to\mathbb{R}$ is a polynomial function of degree $p\geq0$ (not necessarily equal to $q$). Our method is based on the computation of the preimage of the singularity on the reference element using Newton's method, singularity subtraction with high-order Taylor expansions, the continuation approach, and transplanted Gauss quadrature. Step 1. We map $\mathcal{T}$ back to $\widehat{T}$, $$ I(\boldsymbol{x}_0) = \int_{\widehat{T}}\frac{\psi(\boldsymbol{\widehat{x}})}{\vert F(\boldsymbol{\widehat{x}}) - \boldsymbol{x}_0\vert}dS(\boldsymbol{\widehat{x}}), $$ where $\psi(\boldsymbol{\widehat{x}})$ includes the Jacobian $J$. Step 2. We write $$ \boldsymbol{x}_0 = F(\boldsymbol{\widehat{x}}_0) + \boldsymbol{x}_0 - F(\boldsymbol{\widehat{x}}_0) $$ for some $\boldsymbol{\widehat{x}}_0\in\widehat{T}$ such that its image $F(\boldsymbol{\widehat{x}}_0)\in\mathcal{T}$ is the closest point to $\boldsymbol{x}_0$ on $\mathcal{T}$. In other words, we compute the preimage $\boldsymbol{\widehat{x}}_0$ of the singularity or near-singularity. This can be done efficiently with Newton's method combined with a backtracking line search. Step 3. We compute the singular or near-singular term using the first-order Taylor series of $F(\boldsymbol{\widehat{x}}) - \boldsymbol{x}_0$ at $\boldsymbol{\widehat{x}}_0$, $$ T_{-1}(\boldsymbol{\widehat{x}},h) = \frac{\psi(\boldsymbol{\widehat{x}}_0)}{\sqrt{\vert J(\boldsymbol{\widehat{x}}_0)(\boldsymbol{\widehat{x}} - \boldsymbol{\widehat{x}}_0)\vert^2 + h^2}}, $$ and add it to/subtract it from the original integral, $$ \begin{align} I(\boldsymbol{x}_0) = \, & \int_{\widehat{T}}T_{-1}(\boldsymbol{\widehat{x}},h)dS(\boldsymbol{\widehat{x}}) + \int_{\widehat{T}}\left[\frac{\psi(\boldsymbol{\widehat{x}})}{\vert F(\boldsymbol{\widehat{x}}) - \boldsymbol{x}_0\vert} - T_{-1}(\boldsymbol{\widehat{x}},h)\right]dS(\boldsymbol{\widehat{x}}). \end{align} $$ The first integral is singular or near-singular and will be computed in Steps 4–5. The second integral has a bounded integrand—it can be computed with Gauss quadrature on triangles. To render the integrand in the second integral smoother, which would accelerate convergence by quadrature, higher-order Taylor expansions may be added and subtracted. Step 4. Let $$ I_{-1}(h) = \int_{\widehat{T}}T_{-1}(\boldsymbol{\widehat{x}},h)dS(\boldsymbol{\widehat{x}}). $$ The integrand is homogeneous in both $\boldsymbol{\widehat{x}}$ and $h$, and using the continuation approach, we reduce the 2D integral to a sum of three 1D integrals along the edges of $\widehat{T}-\boldsymbol{\widehat{x}}_0$, $$ I_{-1}(h) = \psi(\boldsymbol{\widehat{x}}_0)\sum_{j=1}^3\widehat{s}_j\int_{\partial\widehat{T}_j - \boldsymbol{\widehat{x}}_0}f_{-1}(\boldsymbol{x},h)ds(\boldsymbol{\widehat{x}}), $$ where the $\widehat{s}_j$'s are the distances from the origin to the edges $\partial\widehat{T}_j-\boldsymbol{\widehat{x}}_0$ of $\widehat{T}-\boldsymbol{\widehat{x}}_0$, and $$ f_{-1}(\boldsymbol{\hat{x}},h) = \frac{\sqrt{\vert J(\boldsymbol{\widehat{x}}_0)\boldsymbol{\widehat{x}}\vert^2+h^2}-h}{\vert J(\boldsymbol{\widehat{x}}_0)\boldsymbol{\widehat{x}}\vert^2}. $$ Step 5. On the one hand, when the origin is far from all three edges (e.g., near the center of the triangle), each integrand is analytic and convergence with Gauss quadrature is exponential. On the other, when the origin lies on an edge, the corresponding integrand is singular—however, the distance $\widehat{s}_j$ to that edge equals $0$, the product “$\widehat{s}_j$ times integral” is also $0$, and the integral need not be computed. Issues arise when the origin is close to one of the edges—the integrand is analytic but near-singular, convergence with Gauss quadrature is exponential but slow. We circumvent the near-singularity issue by using transplanted Gauss quadrature. Numerical experiments Let $\Omega\subset\mathbb{R}^3$ be a bounded domain whose complement is connected, let $\Gamma$ be its boundary, and let $k>0$. Given an incident wave $u^i$, a solution to $\Delta u + k^2u = 0$ in $\mathbb{R}^3$, we look for the scattered field $u^s$, a solution to $\Delta u + k^2u = 0$ in $\mathbb{R}^3\setminus\overline{\Omega}$, satisfying the Sommerfeld radiation condition and such that $u^i(\boldsymbol{x}) + u^s(\boldsymbol{x}) = 0$ on $\Gamma=\partial\Omega$. As said above, this leads to $$ \frac{1}{4\pi}\int_{\Gamma}\frac{e^{ik\vert\boldsymbol{x}-\boldsymbol{y}\vert}}{\vert\boldsymbol{x}-\boldsymbol{y}\vert}\varphi(\boldsymbol{y})d\Gamma(\boldsymbol{y}) = -u^i(\boldsymbol{x}), $$ for $\boldsymbol{x}\in\Gamma$. Once the equation is solved for $\varphi$, the solution $u^s$ for all $x\in\mathbb{R}^3\setminus\Omega$ is given by $$ u^s(\boldsymbol{x}) = \frac{1}{4\pi}\int_{\Gamma}\frac{e^{ik\vert\boldsymbol{x}-\boldsymbol{y}\vert}}{\vert\boldsymbol{x}-\boldsymbol{y}\vert}\varphi(\boldsymbol{y})d\Gamma(\boldsymbol{y}). $$ We discretize the integral equation using quadratic boundary elements together with our method for computing singular integrals. We take $k=2\pi$ and consider the scattering of a plane wave $u^i(r,\theta)=e^{ikr\cos\theta}$ by two half-spheres of radius $1$ centered at $(0,0,\pm\delta)$ for small values of $\delta$. For each of these values, we solve the equation and evaluate the total field $u=u^i+u^s$ for a mesh size $h\approx10^{-1}$, which guarantees about five digits of accuracy. We plot $u$ below for $\delta=0.5,0.2,0.1,0$. Let us emphasize that this numerical experiment is particularly challenging when $\delta\ll h$. Standard methods like that of Sauter and Schwab fail to be accurate in this regime; our method produced accurate results for $h\approx10^{-1}$ and $\delta$ as small as $5\times10^{-4}$. Check out our paper!

The linear sampling method for random sources https://hadrien-montanelli.github.io/2022-11-04.html 2022-11-04T00:00:00+00:00

In this post, I talk about an extension of the linear sampling method (LSM) for solving the sound-soft inverse acoustic scattering problem with randomly distributed point sources. For details, check out my paper with Houssem Haddar and Josselin Garnier! Introduction A typical inverse scattering problem is the identification of the shape of a defect inside a medium by sending waves that propagate within. In the data acquisition step, several receivers record the medium's response that forms the data of the inverse problem; in the data processing step, numerical algorithms are used to recover an approximation of the shape from the measurements. What we have just described is an example of active imaging, where both the sources and the receivers are controlled. In passive imaging, only receivers are employed and the illumination comes from uncontrolled, random sources. In this setup, it is the cross-correlations between the recorded signals that convey information about the medium through which the waves propagate. The Helmholtz–Kirchoff identity Acoustic scattering is governed by the Helmholtz equation $\Delta u+k^2u=0$, whose Green's function is $$ \phi(\boldsymbol{x},\boldsymbol{y}) = \frac{i}{4}H_0^{(1)}(k\vert\boldsymbol{x}-\boldsymbol{y}\vert) $$ in dimension two and $$ \quad \phi(\boldsymbol{x},\boldsymbol{y}) = \frac{e^{ik\vert\boldsymbol{x}-\boldsymbol{y}\vert}}{4\pi \vert\boldsymbol{x}-\boldsymbol{y}\vert} $$ in dimension three. The number $k>0$ is the wavenumber (the wavelength is $\lambda=2\pi/k$), and $i$ denotes the imaginary unit. The Helmholtz–Kirchhoff identity for the Green's function reads $$ \phi(\boldsymbol{x},\boldsymbol{y}) - \overline{\phi(\boldsymbol{x},\boldsymbol{y})} = 2ik\int_{\Sigma}\overline{\phi(\boldsymbol{x},\boldsymbol{z})}\phi(\boldsymbol{y},\boldsymbol{z})dS(\boldsymbol{z}), $$ where the surface $\Sigma$ encloses $\boldsymbol{x}$ and $\boldsymbol{y}$. The identity above connects the imaginary part of the measurements at $\boldsymbol{x}$ of point sources located at $\boldsymbol{y}$ (left) to the cross-correlation between the measurements at $\boldsymbol{x}$ and $\boldsymbol{y}$ of point sources on $\Sigma$ (right). The Helmholtz–Kirchhoff identity applies to total fields, too. Let $D\subset\mathbb{R}^d$ (for $d=2$ or 3) be a bounded domain whose complement is connected and $\boldsymbol{y}\in\mathbb{R}^d$. A point source located at $\boldsymbol{y}\in\mathbb{R}^d\setminus\overline{D}$ transmits a time-harmonic signal that generates the incident field $\phi(\cdot,\boldsymbol{y})$. The scattered field $u^s(\cdot,\boldsymbol{y})$ for a sound-soft defect $D$ is the solution to $$ \left\{ \begin{array}{ll} \Delta u^s(\cdot,\boldsymbol{y}) + k^2 u^s(\cdot,\boldsymbol{y}) = 0 \quad \text{in $\mathbb{R}^d\setminus\overline{D}$}, \\[0.4em] \phi(\cdot,\boldsymbol{y}) + u^s(\cdot,\boldsymbol{y}) = 0 \quad \text{on $\partial D$}, \\[0.4em] \text{$u^s(\cdot,\boldsymbol{y})$ is radiating}. \end{array} \right. $$ The total field is then defined as $u(\boldsymbol{x},\boldsymbol{y})=\phi(\boldsymbol{x},\boldsymbol{y})+u^s(\boldsymbol{x},\boldsymbol{y})$. The Helmholtz–Kirchhoff identity for total fields is: $$ u(\boldsymbol{x},\boldsymbol{y}) - \overline{u(\boldsymbol{x},\boldsymbol{y})} = 2ik\int_{\Sigma}\overline{u(\boldsymbol{x},\boldsymbol{z})}u(\boldsymbol{y},\boldsymbol{z})dS(\boldsymbol{z}). $$ This immediately implies the following relationship for the scattered fields, $$ \begin{align} u^s(\boldsymbol{x},\boldsymbol{y}) - \overline{u^s(\boldsymbol{x},\boldsymbol{y})} = 2ik\int_{\Sigma}\overline{u(\boldsymbol{x},\boldsymbol{z})}u(\boldsymbol{y},\boldsymbol{z})dS(\boldsymbol{z}) - \left[\phi(\boldsymbol{x},\boldsymbol{y}) - \overline{\phi(\boldsymbol{x},\boldsymbol{y})}\right]. \end{align} $$ The standard LSM in active imaging Let $\boldsymbol{x}_j$, $1\leq j\leq J$, and $\boldsymbol{y}_m$, $1\leq m\leq M$, be measurement points and point sources located on a surface $\partial B$ that encloses $D$. The standard LSM for near-field measurements, in active imaging, relies on the construction of the near-field matrix $N^\mathrm{ac}$ with entries $$ N^\mathrm{ac}_{jm} = u^s(\boldsymbol{x}_j,\boldsymbol{y}_m). $$ The matrix $N^\mathrm{ac}$ corresponds to the medium's response, measured at $\boldsymbol{x}_j$, to the illumination by point sources located at $\boldsymbol{y}_m$. Here, the positions of both measurement points and point sources are known and controlled. This matrix is filled out in the data acquisition step, which entails the solution of the PDE for each $\boldsymbol{y}_m$. In the processing step, we numerically probe the medium by solving the linear system $N^\mathrm{ac}g_{\boldsymbol{z}}=\phi_{\boldsymbol{z}}$ for various $\boldsymbol{z}\in\mathbb{R}^d$. (The right-hand side is defined by $(\phi_{\boldsymbol{z}})_j = \phi(\boldsymbol{x}_j, \boldsymbol{z})$, $1\leq j\leq J$.) The boundary $\partial D$ of the defect $D$ coincides with those points $\boldsymbol{z}$ for which the value of $\Vert g_{\boldsymbol{z}}\Vert_2$ is large. The setup we have described is shown below. The new LSM for passive imaging In passive imaging, we assume that the measurement points $\boldsymbol{x}_j$ are located in some bounded volume $B\subset\mathbb{R}^d\setminus\overline{D}$. Let $\Sigma$ be a surface that encloses $B$ and $D$, and let us assume that there are $L>0$ point sources $\boldsymbol{z}_\ell$ randomly distributed on $\Sigma$. These sources can transmit a unit-amplitude time-harmonic signal, one by one, so that it is possible to measure the total fields $u(\boldsymbol{x}_j,\boldsymbol{z}_\ell)$. Moreover, it is possible to compute $\phi(\boldsymbol{x}_j,\boldsymbol{x}_m)$, so we can evaluate the cross-correlation matrix $C$ with entries $$ \begin{align} C_{jm} = \frac{2ik\vert\Sigma\vert}{L}\sum_{\ell=1}^L\overline{u(\boldsymbol{x}_j,\boldsymbol{z}_\ell)}u(\boldsymbol{x}_m,\boldsymbol{z}_\ell) - \left[\phi(\boldsymbol{x}_j,\boldsymbol{x}_m) - \overline{\phi(\boldsymbol{x}_j,\boldsymbol{x}_m)}\right], \end{align} $$ where $\vert\Sigma\vert$ is the area of $\Sigma$. The matrix $C$ corresponds to the discretization of the right-hand side of the Helmholtz–Kirchhoff identity (for scattered fields) at points $\boldsymbol{z}_\ell$ with uniform weights and evaluted at $\boldsymbol{x}_j$ and $\boldsymbol{x}_m$, i.e., $$ \begin{align} C_{jm} \approx 2ik\int_{\Sigma}\overline{u(\boldsymbol{x}_j,\boldsymbol{z})}u(\boldsymbol{x}_m,\boldsymbol{z})dS(\boldsymbol{z}) - \left[\phi(\boldsymbol{x}_j,\boldsymbol{x}_m) - \overline{\phi(\boldsymbol{x}_j,\boldsymbol{x}_m)}\right]. \end{align} $$ Therefore, it satisfies $$ C_{jm} \approx N_{jm} - \overline{N_{jm}}, $$ where the matrix $N$ is the standard near-field matrix for co-located receivers and sources: $$ N_{jm} = u^s(\boldsymbol{x}_j,\boldsymbol{x}_m). $$ This motivates the introduction of the imaginary near-field matrix $I$, $$ I_{jm} = N_{jm} - \overline{N_{jm}} = u^s(\boldsymbol{x}_j,\boldsymbol{x}_m) - \overline{u^s(\boldsymbol{x}_j,\boldsymbol{x}_m)}. $$ In our paper, we justified the utilization of the matrix $I$ for the LSM in active imaging. If the LSM works for $I$ in active imaging, then, via the Helmholtz–Kirchhoff identity $C_{jm} \approx I_{jm}$, it works for $C$ in passive imaging. The setup we have described is shown below. Numerical experiments We consider the scattering of points sources by a kite of size $\lambda/2$ centered at $2\lambda + 2\lambda i$ for $k=2\pi$ (wavelength $\lambda =1$). We compare the results obtained for the near-field matrix $N$, the imaginary near-field matrix $I$, and the cross-correlation matrix $C$. For $N$ and $I$, we take $J=80$ equispaced co-located sources and receivers on the circle of radius $5\lambda$, $$ \boldsymbol{x}_j = 5\lambda e^{i\theta_j}, \quad \theta_j = \frac{2\pi}{J}(j - 1). $$ For the matrix $C$, the $L=80$ random sources are located on the circle of radius $50\lambda$, $$ \boldsymbol{z}_\ell = 50\lambda e^{i\theta_\ell}, \quad \theta_\ell = \frac{2\pi}{L}(\ell -1+ \beta_\ell), $$ where $\beta_\ell$ is drawn from the uniform distribution on $[0,\beta]$ with $\beta=0.1$, and we measure at the points $\boldsymbol{x}_j$. To simulate noisy measurements, we add some white noise of amplitude $5\times 10^{-2}$ to each matrix; we call the noisy matrices $N_\delta$, $I_\delta$, and $C_\delta$. Finally, we probe the medium on a $100\times100$ uniform grid on $[-6\lambda,6\lambda]\times[-6\lambda,6\lambda]$. Here are the results. The first column displays the matrices and the second column the indicator function (values outside of the circle of radius $5\lambda$ were zeroed out). The sources are represented by dots, and the measurement points by crosses. For $N$ and $I$, we plotted every other source/measurement point; for $C$, the random sources are outside of the plot. Our novel LSM (third row), based on cross-correlations and random sources, shows similar results to the LSM with deterministic sources (first row). The LSM can be utilized in passive imaging!

Image matting with examples in Python https://hadrien-montanelli.github.io/2022-12-30.html 2022-12-30T00:00:00+00:00

In this post, I talk about image and video matting—the process of extracting specific objects from an image to place them in a scene of a movie or compose them onto another background. Introduction Formally, matting techniques take as input in image $I$, which is assumed to be a convex combination of a foreground image $F$ and a background image $B$, $$ I = \alpha F + (1-\alpha)B, $$ where $\alpha$ is the pixel's foreground opacity or matte. The above equation applies to each pixel of the image $I$ and for each RGB component of this pixel. In other words, the above equation is an identity between vectors, which we can write as $$ \begin{bmatrix}\mathrm{R}_I\\\mathrm{G}_I\\\mathrm{B}_I\end{bmatrix} = \alpha\begin{bmatrix}\mathrm{R}_F\\\mathrm{G}_F\\\mathrm{B}_F\end{bmatrix} + (1-\alpha)\begin{bmatrix}\mathrm{R}_B\\\mathrm{G}_B\\\mathrm{B}_B\end{bmatrix} \quad\text{at each pixel}. $$ This is what we call the matting equation. Here, the input image $I$ is given and the matte $\alpha$ and the foreground and background images $F$ and $B$ are unknowns, that is, we have seven variables $$ (\mathrm{R}_F,\mathrm{G}_F,\mathrm{B}_F,\mathrm{R}_B,\mathrm{G}_B,\mathrm{B}_B,\alpha) $$ but only three equations—one per RGB component. (More precisely, if the image is $N\times N$, we have $7N^2$ variables and $3N^2$ equations.) Mathematically, we say that this set of equations is underdetermined. There is no general and obvious way to solve it unless we make some simplifying assumptions or else are given some additional inputs. Green screen matting In green screen matting, also called chroma keying, we assume that the background image $B$ is a given uniform green color $\mathrm{G}_B$. In terms of RGB components, this means that $$ B = \begin{bmatrix}0\\\mathrm{G}_B\\0\end{bmatrix}. $$ In this case, the matting equation simplifies to $$ \begin{bmatrix}\mathrm{R}_I\\\mathrm{G}_I\\\mathrm{B}_I\end{bmatrix} = \alpha\begin{bmatrix}\mathrm{R}_F\\\mathrm{G}_F\\\mathrm{B}_F\end{bmatrix} + (1-\alpha)\begin{bmatrix}0\\\mathrm{G}_B\\0\end{bmatrix}. $$ We now have four variables $(\mathrm{R}_F,\mathrm{G}_F,\mathrm{B}_F,\alpha)$ and three equations—this is a much simpler problem. We can even obtain exact solutions in several specific cases. Gray images For gray foreground images $F$, we may assume that $$ \mathrm{R}_F = \mathrm{G}_F = \mathrm{B}_F, $$ which leads to $$ \begin{bmatrix}\mathrm{R}_I\\\mathrm{G}_I\\\mathrm{B}_I\end{bmatrix} = \alpha\begin{bmatrix}\mathrm{R}_F\\\mathrm{R}_F\\\mathrm{R}_F\end{bmatrix} + (1-\alpha)\begin{bmatrix}0\\\mathrm{G}_B\\0\end{bmatrix}. $$ In this case, we only have two variables $(\mathrm{R}_F,\alpha)$. The three equations read $$ \left\{ \begin{array}{ll} \mathrm{R}_I = \alpha\mathrm{R}_F, \\ \mathrm{G}_I = \alpha\mathrm{R}_F + (1-\alpha)\mathrm{G}_B, \\ \mathrm{B}_I = \alpha\mathrm{R}_F. \end{array} \right. $$ The first and third equations yield $$ \alpha\mathrm{R}_F = \mathrm{R}_I = \mathrm{B}_I,$$ while the second equation gives $\mathrm{G}_I = \mathrm{R}_I + (1-\alpha)\mathrm{G}_B$, that is, $$ \alpha = \frac{\mathrm{R}_I - (\mathrm{G}_I - \mathrm{G}_B)}{\mathrm{G}_B}. $$ To obtain a new image $J$ with a different background $K$, one can simply compute $$ J = \alpha F +(1-\alpha)K, $$ that is, $$ \begin{align} \begin{bmatrix}\mathrm{R}_J\\\mathrm{G}_J\\\mathrm{B}_J\end{bmatrix} = \alpha\begin{bmatrix}\mathrm{R}_F\\\mathrm{R}_F\\\mathrm{R}_F\end{bmatrix} + (1-\alpha)\begin{bmatrix}\mathrm{R}_K\\\mathrm{G}_K\\\mathrm{B}_K\end{bmatrix} = \begin{bmatrix}\mathrm{R}_I\\\mathrm{R}_I\\\mathrm{R}_I\end{bmatrix} +(1-\alpha)\begin{bmatrix}\mathrm{R}_K\\\mathrm{G}_K\\\mathrm{B}_K\end{bmatrix}. \end{align} $$ Let's look at an example. Consider the following image from the alpha matting evaluation website together with its "exact" matte: import cv2 import matplotlib.pyplot as plt import numpy as np I = cv2.imread('GT04.png') # load image I (BGR format) I = cv2.cvtColor(I, cv2.COLOR_BGR2RGB)/255 # convert to RGB and normalize plt.imshow(I), plt.axis('off') # plot I plt.show(); alpha_ex = cv2.imread('GT04_alpha.png', cv2.IMREAD_GRAYSCALE)/255 # load exact alpha and normalize plt.imshow(alpha_ex, cmap='gray'), plt.axis('off') # plot alpha plt.show(); We create a gray version of the image with a green background as follows: n_rows = I.shape[0] # number of rows in I n_cols = I.shape[1] # number of columns in I n_pixels = n_rows * n_cols # number of pixels in I I = np.reshape(I, (n_pixels, 3)) I[:, 0] = I[:, 1] # red = green I[:, 2] = I[:, 1] # blue = green alpha_ex = np.reshape(alpha_ex, (n_pixels, 1)) G_B = 1 # green screen value I = alpha_ex * I + (1 - alpha_ex) * [0, G_B, 0] # replace background with green screen I = np.reshape(I, (n_rows, n_cols, 3)) plt.imshow(I), plt.axis('off') # plot I plt.show(); Let's extract the RGB components of $I$: R_I = I[:, :, 0] # red component of I G_I = I[:, :, 1] # green component of I B_I = I[:, :, 2] # blue component of I Let's compute the matte $\alpha$: alpha = (R_I - (G_I - G_B))/G_B # compute alpha with formula plt.imshow(alpha, cmap='gray'), plt.axis('off') # plot alpha plt.show(); The relative error between our matte and the "exact" one is around machine precision: alpha = np.reshape(alpha, (n_pixels, 1)) error = np.linalg.norm(alpha - alpha_ex)/np.linalg.norm(alpha_ex) print(f'Error (alpha): {error:.2e}') Error (alpha): 3.63e-17 We're going to replace the green screen with the following image of Toronto's skyline: K = cv2.imread('toronto.jpg') # load image (BGR format) K = cv2.cvtColor(K, cv2.COLOR_BGR2RGB)/255 # convert to RGB and normalize K = K[:n_rows, :n_cols, :] # adjust size of K to size of I plt.imshow(K), plt.axis('off') # plot K plt.show(); K = np.reshape(K, (n_pixels, 3)) Let's now replace the green screen with the image $K$: R_I = np.reshape(R_I, (n_pixels, 1)) J = np.tile(R_I, 3) + (1 - alpha) * K # new image J with different background K J = np.reshape(J, (n_rows, n_cols, 3)) plt.imshow(J), plt.axis('off') # plot J plt.show(); The result is flawless—it is not surprising that science fiction movies use green screens particularly well since many foreground objects are neutrally colored spacecraft. Other exact solutions Smith and Blinn [1] found an exact solution of the matting equation for foreground images $F$ that verify $$ \mathrm{B}_F = a_0 + a_1\mathrm{G}_F, $$ for some coefficients $a_0$ and $a_1$. We present here a slightly more general solution for foreground images $F$ that satisfy $$ \mathrm{B}_F = a_0 + a_1\mathrm{G}_F + a_2\mathrm{R}_F. $$ Using this relationship, we obtain the following system of equations, $$ \begin{bmatrix}\mathrm{R}_I\\\mathrm{G}_I\\\mathrm{B}_I\end{bmatrix} = \alpha\begin{bmatrix}\mathrm{R}_F\\\displaystyle\frac{\mathrm{B}_F - a_0 - a_2\mathrm{R}_F}{a_1}\\\mathrm{B}_F\end{bmatrix} + (1-\alpha)\begin{bmatrix}0\\\mathrm{G}_B\\0\end{bmatrix}. $$ Here, we have three variables $(\mathrm{R}_F,\mathrm{B}_F,\alpha)$. The three equations read $$ \left\{ \begin{array}{ll} \mathrm{R}_I = \alpha\mathrm{R}_F, \\ \mathrm{G}_I = \alpha\displaystyle\frac{\mathrm{B}_F - a_0 - a_2\mathrm{R}_F}{a_1} + (1-\alpha)\mathrm{G}_B, \\ \mathrm{B}_I = \alpha\mathrm{B}_F. \end{array} \right. $$ The first and third equations lead to $$ \alpha\mathrm{R}_F = \mathrm{R}_I, \quad \alpha\mathrm{B}_F = \mathrm{B}_I,$$ while the second equation gives $\mathrm{G}_I = (\mathrm{B}_I - a_0\alpha - a_2\mathrm{R}_I)/a_1 + (1-\alpha)\mathrm{G}_B$, that is, $$ \alpha = \frac{\mathrm{B}_I - a_1(\mathrm{G}_I - \mathrm{G}_B) - a_2\mathrm{R}_I}{a_0 + a_1\mathrm{G}_B}. $$ Again, to obtain a new image $J$ with a different background $K$, one can simply compute $$ J = \alpha F +(1-\alpha)K, $$ that is, $$ \begin{align} \begin{bmatrix}\mathrm{R}_J\\\mathrm{G}_J\\\mathrm{B}_J\end{bmatrix} = \alpha\begin{bmatrix}\mathrm{R}_F\\\displaystyle\frac{\mathrm{B}_F - a_0 - a_2\mathrm{R}_F}{a_1}\\\mathrm{B}_F\end{bmatrix} + (1-\alpha)\begin{bmatrix}\mathrm{R}_K\\\mathrm{G}_K\\\mathrm{B}_K\end{bmatrix} = \begin{bmatrix}\mathrm{R}_I\\\displaystyle\frac{\mathrm{B}_I - a_0\alpha - a_2\mathrm{R}_I}{a_1}\\\mathrm{B}_I\end{bmatrix} +(1-\alpha)\begin{bmatrix}\mathrm{R}_K\\\mathrm{G}_K\\\mathrm{B}_K\end{bmatrix}. \end{align} $$ Let's create a version of the previous image that verifies a relationship of the form $\mathrm{B}_F = a_0 + a_1\mathrm{G}_F + a_2\mathrm{R}_F$ with $a_0=0.4$, $a_1=0.3$ and $a_2=0.3$: I = cv2.imread('GT04.png') # load image I (BGR format) I = cv2.cvtColor(I, cv2.COLOR_BGR2RGB)/255 # convert to RGB and normalize I = np.reshape(I, (n_pixels, 3)) a_0 = 0.4 # coefficient a_0 a_1 = 0.3 # coefficient a_1 a_2 = 0.3 # coefficient a_2 I[:, 2] = a_0 + a_1*I[:, 1] + a_2*I[:, 0] # blue = a_0 + a_1*green + a_2*red I = alpha_ex * I + (1 - alpha_ex) * [0, G_B, 0] # replace background with green screen I = np.reshape(I, (n_rows, n_cols, 3)) plt.imshow(I), plt.axis('off') # plot I plt.show(); Let's extract the RGB components of $I$: R_I = I[:, :, 0] # red component of I G_I = I[:, :, 1] # green component of I B_I = I[:, :, 2] # blue component of I Let's compute $\alpha$: alpha = (B_I - a_1*(G_I - G_B) - a_2*R_I)/(a_0 + a_1*G_B) # compute alpha with formula plt.imshow(alpha, cmap='gray'), plt.axis('off') # plot alpha plt.show(); Once again, the error is around machine precision: alpha = np.reshape(alpha, (n_pixels, 1)) error = np.linalg.norm(alpha - alpha_ex)/np.linalg.norm(alpha_ex) # relative error print(f'Error (alpha): {error:.2e}') Error (alpha): 1.46e-16 We can change the background as follows: alpha = np.reshape(alpha, (n_pixels, 1)) R_I = np.reshape(R_I, (n_pixels, 1)) B_I = np.reshape(B_I, (n_pixels, 1)) J = np.concatenate((R_I, (B_I - a_0*alpha - a_2*R_I)/a_1, B_I), axis=1) # new image J J += (1 - alpha)*K J = np.reshape(J, (n_rows, n_cols, 3)) plt.imshow(J), plt.axis('off') # plot J plt.show(); Extension and connection to Vlahos' formula We've considered so far images whose RGB components verify specific relationships. For arbitrary color images, there is no exact formula for the matte $\alpha$ and the foreground data $(\mathrm{R}_F,\mathrm{G}_F,\mathrm{B}_F)$. However, one may utilize the formula of the previous section as an approximation. We show below an example with the color image we've been working with—this time, however, we use it as is with a green screen (i.e., we don't make it gray or purple-ish). We utilize the formula for the matte $\alpha$ with $a_0=0$, $a_1=0.5$ and $a_2=0$, which yields a relative error below $8\%$. The resulting new image $J$ with a different background $K$ isn't perfect—let's emphasize that it was done with a very simple method that has an extremely low computational cost. Moreover, this image is particularly challenging because of the hair details and color. We provide below the full code in a single cell. # Load the previous color image: I = cv2.imread('GT04.png') I = cv2.cvtColor(I, cv2.COLOR_BGR2RGB)/255 n_rows = I.shape[0] n_cols = I.shape[1] n_pixels = n_rows * n_cols I = np.reshape(I, (n_pixels, 3)) # Load exact matte: alpha_ex = cv2.imread('GT04_alpha.png', cv2.IMREAD_GRAYSCALE)/255 alpha_ex = np.reshape(alpha_ex, (n_pixels, 1)) # Generate a version with a green screen: G_B = 1 I = alpha_ex * I + (1 - alpha_ex) * [0, G_B, 0] I = np.reshape(I, (n_rows, n_cols, 3)) plt.imshow(I), plt.axis('off'), plt.show(); # RGB components: R_I = I[:, :, 0] G_I = I[:, :, 1] B_I = I[:, :, 2] # Coefficients: a_0 = 0 a_1 = 0.5 a_2 = 0 # Compute alpha with formula: alpha = (B_I - a_1*(G_I - G_B) - a_2*R_I)/(a_0 + a_1*G_B) alpha[alpha 1] = 1 plt.imshow(alpha, cmap='gray'), plt.axis('off'), plt.show(); # Compute relative error with exact alpha: alpha = np.reshape(alpha, (n_pixels, 1)) error = np.linalg.norm(alpha - alpha_ex)/np.linalg.norm(alpha_ex) print(f'Error (alpha): {error:.2e}') # Load background K: K = cv2.imread('toronto.jpg') K = cv2.cvtColor(K, cv2.COLOR_BGR2RGB)/255 K = K[:n_rows, :n_cols, :] K = np.reshape(K, (n_pixels, 3)) # Compute new image J with background K: alpha = np.reshape(alpha, (n_pixels, 1)) R_I = np.reshape(R_I, (n_pixels, 1)) B_I = np.reshape(B_I, (n_pixels, 1)) G_I = np.reshape(G_I, (n_pixels, 1)) tmp = np.minimum(G_I, (B_I - a_0*alpha - a_2*R_I)/a_1) J = np.concatenate((R_I, tmp, B_I), axis=1) + (1 - alpha)*K J[J 1] = 1 J = np.reshape(J, (n_rows, n_cols, 3)) plt.imshow(J), plt.axis('off'), plt.show(); Error (alpha): 7.58e-02 Let's make a few comments about the code. First, because the formula for the matte $\alpha$ isn't exact here—it is merely an approximation—it may lead to values below $0$ and above $1$ for both $\alpha$ and $J$—therefore, we clamped the values of $\alpha$ and $J$. Second, the green component, which was given in the previous section by $$ \frac{\mathrm{B}_I - a_0\alpha - a_2\mathrm{R}_I}{a_1}, $$ has been replaced here with $$ \mathrm{min}\left(\mathrm{G}_I, \frac{\mathrm{B}_I - a_0\alpha - a_2\mathrm{R}_I}{a_1}\right). $$ This is inspired by Vlahos' work—Vlahos was an engineer who revolutionized the motion picture industry by introducing novel green screen techniques, essential to movies such as the Stars Wars trilogy and Titanic; he was awarded multiple Oscars and an Emmy Award for his contributions to the field. In his first patent related to green screen technology [2], he claimed that for foreground images $F$ that verify $$ \mathrm{G}_F \leq b_2\mathrm{B}_F, $$ the matte $\alpha$ can be computed via the formula $$ \alpha = 1 - b_1(\mathrm{G}_I - b_2\mathrm{B}_I), $$ where $b_1$ and $b_2$ are tuning adjustment parameters. Once $\alpha$ has been obtained by his formula, he proposed to compute the new image $J$ with a different background $K$ via $$ \begin{bmatrix}\mathrm{R}_J\\\mathrm{G}_J\\\mathrm{B}_J\end{bmatrix} = \begin{bmatrix}\mathrm{R}_I\\\mathrm{min}\left(\mathrm{G}_I, b_2\mathrm{B}_I\right)\\\mathrm{B}_I\end{bmatrix} +(1-\alpha)\begin{bmatrix}\mathrm{R}_K\\\mathrm{G}_K\\\mathrm{B}_K\end{bmatrix}. $$ In the code above, we selected $a_0=a_2=0$, that is, we assumed that the foreground image $F$ verifies $$ \mathrm{B}_F \approx a_1\mathrm{G}_F, $$ to some approximation. In that case, the formula for $\alpha$ from the previous section simplifies to $$ \alpha = \frac{\mathrm{B}_I - a_1(\mathrm{G}_I - \mathrm{G}_B)}{a_1\mathrm{G}_B} = 1 - \frac{1}{\mathrm{G}_B}\left(\mathrm{G}_I - \frac{\mathrm{B}_I}{a_1}\right), $$ and the green component is given by $$ \begin{bmatrix}\mathrm{R}_J\\\mathrm{G}_J\\\mathrm{B}_J\end{bmatrix} = \begin{bmatrix}\mathrm{R}_I\\\displaystyle\frac{\mathrm{B}_I}{a_1}\\\mathrm{B}_I\end{bmatrix} +(1-\alpha)\begin{bmatrix}\mathrm{R}_K\\\mathrm{G}_K\\\mathrm{B}_K\end{bmatrix}. $$ This is exactly Vlahos' process provided that $b_1=1/\mathrm{G}_B$, $b_2=1/a_1$ and that we replace the green component with $$ \mathrm{min}\left(\mathrm{G}_I, \frac{\mathrm{B}_I}{a_1}\right), $$ which is precisely what we did. Conclusion Image matting is the process of recovering the matte $\alpha$ and the foreground and background images $F$ and $B$ from an input image $I$. This is an undetermined problem, which does not have exact solutions in general. When using a green screen, the problem becomes simpler and there are exact solutions in certain cases—for example when the foreground image $F$ is gray. For arbitrary color images, there exist simple green screen algorithms that lead to reasonably good numerical results—we've reviewed one of them, operating in RGB space. In future blog posts, we'll talk about matting techniques without a green screen based on the Laplacian and deep neural networks, as well as video matting. References [1] A. R. Smith and J. F. Blinn, Blue screen matting, in Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’96, New York, NY, USA, 1996, Association for Computing Machinery, pp. 259–268. [2] P. Vlahos, Electronic Composite Photography, U.S. Patent 3595987A, 1971.

Finite-boundary element coupling in MATLAB https://hadrien-montanelli.github.io/2023-02-12.html 2023-02-12T00:00:00+00:00

In this post, I talk about coupling the finite and boundary element methods for solving wave propagation problems in an inhomogeneous medium. Acoustic waves in an inhomogeneous medium We consider the following model with a total field $u$ coupled with a scattered field $w^s$: $$ \begin{align} & \Delta u + k^2nu = 0 \quad \text{in $B\setminus\overline{D}$}, \\[.4em] & u = 0 \quad \text{on $\partial D$}, \\[0.4em] & \Delta w^s + k^2w^s = 0 \quad \text{in $\mathbb{R}^2\setminus\overline{B}$}, \\[.4em] & \text{$w^s$ is radiating}, \\[0.4em] & \displaystyle w^s = u - u^i \;\,\text{and}\;\, \frac{\partial w^s}{\partial\nu} = \frac{\partial (u - u^i)}{\partial\nu} \quad \text{on $\partial B$}. \end{align} $$ The part of the domain where the refractive index $n\neq1$ models an inhomogeneous medium. For the applications I am interested in, $n$ is often a (smooth) random function. To solve this system, we must be able to solve interior problems of the form $$ \begin{align} & \Delta u + k^2nu = f \quad \text{in $B\setminus\overline{D}$}, \\[0.4em] & u = g \quad \text{on $\partial D$}, \\[0.4em] & \frac{\partial u}{\partial\nu} = h \quad \text{on $\partial B$}, \end{align} $$ with a fine element method, as well as exterior problems of the form $$ \begin{align} & \Delta w + k^2w = 0 \quad \text{in $\mathbb{R}^2\setminus\overline{B}$}, \\[0.4em] & w = g \quad \text{on $\partial B$}, \\[0.4em] & \text{$w$ is radiating}, \end{align} $$ with a boundary element method. Interior problems & finite elements Theory (disk) Let $D=\mathcal{D}(0,R)$, the disk of radius $R>0$ centered at the origin, and consider \begin{align*} & \Delta u + k^2nu = f \quad \text{in $D$}, \\[0.4em] & u = g \quad \text{on $\partial D$}, \end{align*} for some $g$ such that $\Delta g = 0$ in $D$. Let us define $\widetilde{u} = u - g$. Then, $$ \begin{align} & \Delta \widetilde{u} + k^2n\widetilde{u} = f - k^2ng \quad \text{in $D$}, \\[0.4em] & \widetilde{u} = 0 \quad \text{on $\partial D$}. \end{align} $$ The variational problem is to find $\widetilde{u}\in H^1_0(D)$ such that $$ \int_D\left(-\nabla\widetilde{u}\nabla v + k^2n\widetilde{u}v\right) = \int_D (f - k^2ng)v, $$ for all $v\in H^1_0(D)$, where $$ H^1_0(D) = \left\{v \in H^1(D) \,:\, v|_{\partial D} = 0\right\}. $$ Once the problem is solved for $\widetilde{u}$, we recover $u=\widetilde{u}+g$. Experiment (disk) Let us take $R=1$, $k=2\pi$, and $$ n(x,y)=1, \quad f(x,y)=x^2, \quad g(x,y)=4. $$ Using the gypsilab toolbox, we may compute the solution as follows. R = 1; k = 2*pi; n = @(X) 0*X(:, 1) + 1; f = @(X) X(:, 1).^2; g = @(X) 0*X(:, 1) + 4; N = 1000; mesh = mshDisk(N, R); D = dom(mesh, 3); V = fem(mesh, 'P1'); V = dirichlet(V, mesh.bnd); L = -integral(D, grad(V), grad(V)) ... + k^2*integral(D, V, n, V); F = integral(D, V, @(X) f(X) - k^2*n(X).*g(X)); u = L\F; In Chebfun, we may use the diskfun.helmholtz code. F = diskfun(@(x,y) f([x, y]) - k^2*g([x, y])); BC = @(theta) 0*theta; u = diskfun.helmholtz(F, k, BC, N); Theory (annulus) Let us now consider a harder problem. Let $D=\mathcal{D}(0, R_1)$ and $B=\mathcal{D}(0,R_2)$ for some $R_2>R_1>0$. Define $C=B\setminus\overline{D}$ and consider $$ \begin{align} & \Delta u + k^2nu = f \quad \text{in $C$}, \\[0.4em] & u = g \quad \text{on $\partial D$}, \\[0.4em] & \frac{\partial u}{\partial\nu} = h \quad \text{on $\partial B$}, \end{align} $$ for some $g$ such that $\Delta g = 0$ in $C$ and $\partial g/\partial\nu = 0$ on $\partial B$. Let us define $\widetilde{u} = u - g$. Then, $$ \begin{align} & \Delta \widetilde{u} + k^2n\widetilde{u} = f - k^2ng \quad \text{in $C$}, \\[0.4em] & \widetilde{u} = 0 \quad \text{on $\partial D$}, \\[0.4em] & \frac{\partial\widetilde{u}}{\partial\nu} = h \quad \text{on $\partial B$}. \end{align} $$ The problem is to find $\widetilde{u}\in H^1_0(C)$ such that $$ \int_{C}\left(-\nabla\widetilde{u}\nabla v + k^2n\widetilde{u}v\right) = \int_{C} (f - k^2ng)v - \int_{\partial B}hv, $$ for all $v\in H^1_0(C)$, where (slight abuse of notation) $$ H^1_0(C) = \left\{v \in H^1(C) \,:\, v|_{\partial D} = 0\right\}. $$ Once the problem is solved for $\widetilde{u}$, we recover $u=\widetilde{u}+g$. Experiment (annulus) Let us take $R_1=0.1$, $R_2=1$, $k=2\pi$, $$ n(x,y)=\cos(y), \quad f(x,y)=x^2, $$ and $g(x,y) = h(x,y) = 1$. R1 = 0.1; R2 = 1; k = 2*pi; n = @(X) cos(X(:, 2)); f = @(X) X(:, 1).^2; g = @(X) 0*X(:, 1) + 1; h = @(X) 0*X(:, 1) + 1; N = 2000; mesh = mshAnnulus(N, R1, R2); C = dom(mesh, 3); [dD, dB] = bndAnnulus(mesh.bnd); dB = dom(dB, 3); V = fem(mesh, 'P1'); V = dirichlet(V, dD); L = -integral(C, grad(V), grad(V)) ... + k^2*integral(C, V, n, V); F = integral(C, V, @(X) f(X) - k^2*n(X).*g(X)) ... - integral(dB, V, h); u = L\F; Exterior problems & boundary elements Theory Let $B=\mathcal{D}(0,R)$ and consider $$ \begin{align} & \Delta w + k^2w = 0 \quad \text{in $\mathbb{R}^2\setminus\overline{B}$}, \\[0.4em] & w = g \quad \text{on $\partial B$}, \quad \text{$w$ is radiating}, \end{align} $$ for some function $g$. Let us look for $w$ as a single-layer potential, $$ w(\boldsymbol{x}) = \int_{\partial B} G(\boldsymbol{x},\boldsymbol{y})\varphi(\boldsymbol{y})dS(\boldsymbol{y}) = (\mathcal{S}\varphi)(\boldsymbol{x}). $$ The problem is to find $\varphi\in H^{-1/2}(\partial B)$ such that $$ \int_{\partial B}(\mathcal{S}\varphi)(\boldsymbol{x})\psi(\boldsymbol{x})dS(\boldsymbol{x}) = \int_{\partial B}g(\boldsymbol{x})\psi(\boldsymbol{x})dS(\boldsymbol{x}), $$ for all $\psi\in H^{-1/2}(\partial B)$. Experimemt Let us take $R=0.5$, $k=2\pi$, and $$ g(x,y) = -e^{ikx}. $$ R = 0.5; k = 2*pi; g = @(X) -exp(1i*k*X(:,1)); G = @(X,Y) femGreenKernel(X, Y, '[H0(kr)]', k); N = 100; mesh = mshCircle(N, R); dB = dom(mesh, 3); W = fem(mesh, 'P1'); S = (1i/4)*integral(dB, dB, W, G, W); S = S - 1/(2*pi)*regularize(dB, dB, W, ... '[log(r)]', W); F = integral(dB, W, g); phi = S\F; Finite-boundary element coupling Theory We now come back to the initial problem: $$ \begin{align} & \Delta u + k^2nu = 0 \quad \text{in $B\setminus\overline{D}$}, \\[.4em] & u = 0 \quad \text{on $\partial D$}, \\[0.4em] & \Delta w^s + k^2w^s = 0 \quad \text{in $\mathbb{R}^2\setminus\overline{B}$}, \\[.4em] & \text{$w^s$ is radiating}, \\[0.4em] & \displaystyle w^s = u - u^i \;\,\text{and}\;\, \frac{\partial w^s}{\partial\nu} = \frac{\partial (u - u^i)}{\partial\nu} \quad \text{on $\partial B$}. \end{align} $$ Let $D=\mathcal{D}(0, R_1)$, $B=\mathcal{D}(0,R_2)$ for some $R_2>R_1>0$, and $C=B\setminus\overline{D}$. Combining the theory of the previous sections for interior and exterior problems, we define a bilinear form $A$ with entries $$ \begin{pmatrix} \displaystyle \int_{\partial B}(\mathcal{S}\varphi)\psi & \displaystyle -\int_{\partial B}u\psi \\[.4em] \displaystyle \int_{\partial B}\left[\left(\mathcal{D}^* - \frac{\mathcal{I}}{2}\right)\varphi\right]v & \displaystyle \int_C\left(-\nabla u\nabla v + k^2nuv\right) \end{pmatrix}, $$ while the linear form $F$ has entries $$ \begin{pmatrix} \displaystyle -\int_{\partial B} u^i\psi \\[.4em] \displaystyle -\int_{\partial B}\frac{\partial u^i}{\partial \nu}v \end{pmatrix}, $$ for $\varphi,\psi\in H^{-1/2}(\partial B)$ and $u,v\in H^1_0(C)$. We may write the system as $$ \left\{ \begin{array}{l} A_{11}\varphi + A_{12} u = F_1, \\[.4em] A_{21}\varphi + A_{22} u = F_2. \end{array} \right. $$ This yields $$ \left\{ \begin{array}{l} -A_{21}A_{11}^{-1}A_{12}u + A_{22}u = F_2 - A_{21}A_{11}^{-1}F_1, \\[.4em] \varphi = A_{11}^{-1}(F_1 - A_{12}u). \end{array} \right. $$ We use GMRES for the first equation (mgcr in gypsilab). Experiment Let us take $R_1=0.5$, $R_2=1$, $k=2\pi$, and $$ u^i(x,y) = e^{ikx}, \quad n(x,y)=1. $$ Since we take $n=1$ everywhere, we should recover the numerical solution of the previous section. The MATLAB code is a little bit more involved. We are going to break it down. We first set up the parameters. R1 = 0.5; R2 = 1; k = 2*pi; lambda = 2*pi/k; ui = @(X) exp(1i*k*X(:, 1)); dui = @(X) 1i*k*X(:,1)/R2.*ui(X); G = @(X,Y) femGreenKernel(X, Y, '[H0(kr)]', k); dG{1} = @(X,Y) femGreenKernel(X, Y, ... 'grady[H0(kr)]1', k); dG{2} = @(X,Y) femGreenKernel(X, Y, ... 'grady[H0(kr)]2', k); dG{3} = @(X,Y) femGreenKernel(X, Y, ... 'grady[H0(kr)]3', k); n = @(X) 0*X(:, 1) + 1; We define the mesh and the finite/boundary elements. N = 1000; mesh = mshAnnulus(N, R1, R2); C = dom(mesh, 3); [dD, dB] = bndAnnulus(mesh.bnd); V = fem(mesh, 'P1'); V = dirichlet(V, dD); W = fem(dB, 'P1'); dB = dom(dB, 3); This is the tricky part where we define the projection between finite and boundary elements, as well as the operators. (The negative sign in the double-layer variable D is to adjust the direction of the normal.) [~, IW, IV] = intersect(W.unk, V.unk, 'rows'); P = sparse(IW, IV, 1, size(W.unk, 1), size(V.unk, 1)); L = -integral(C, grad(V), grad(V)) ... + k^2*integral(C, V, n, V); S = (1i/4)*integral(dB, dB, W, G, W); S = S - 1/(2*pi)*regularize(dB, dB, W, ... '[log(r)]', W); D = (1i/4)*integral(dB, dB, W, dG, ntimes(W)); D = D - 1/(2*pi)*regularize(dB, dB, W, ... 'grady[log(r)]', ntimes(W)); D = -D; I = integral(dB, W, W); We assemble the system and solve it. A11 = S; A21 = P'*(D.' - I/2); A12 = -I*P; A22 = L; F1 = -integral(dB, W, ui); F2 = -integral(dB, V, dui); fun = @(V) -A21*(A11\(A12*V)) + A22*V; u = mgcr(fun, F2 - A21*(A11\F1), [], 1e-3, 1000); phi = A11\(F1 - A12*u);

Strongly singular integrals over curved elements https://hadrien-montanelli.github.io/2023-10-02.html 2023-10-02T00:00:00+00:00

In this post, I talk about a new method for computing strongly singular and near-singular integrals that arise when solving the 3D Helmholtz equation with curved boundary elements. Background on integral equations The Helmholtz equation, which has the form $$ \Delta u + k^2u = 0, $$ appears when one looks for time-harmonic solutions to the wave equation—if $v(\boldsymbol{x},t)=u(\boldsymbol{x})e^{-i\omega t}$ is a solution to $v_{tt}=c^2\Delta v$, then $u$ satisfies the Helmholtz equation with wavenumber $k=\omega /c$. It is of fundamental importance in science and engineering, with applications as diverse as noise scattering, radar and sonar technology, and seismology. For instance, given an incident acoustic wave $u^i$ that is a solution to the Helmholtz equation in $\mathbb{R}^3$, the outgoing scattered field $u^s$ generated by a bounded obstacle $D$ is also a solution to Helmholtz equation, in $\mathbb{R}^3\setminus\overline{D}$, with $u^i+u^s=0$ on the boundary $\partial D$ (for sound-soft scattering). For obstacles $D$ whose complements are connected, a popular technique for calculating scattered fields is based on integral equations. As an example, one can show that the solution $u^s$ to the sound-soft scattering problem of the previous paragraph may be obtained by solving $$ \int_{\partial D}\left\{\frac{\partial G(\boldsymbol{x},\boldsymbol{y})}{\partial\boldsymbol{n}(\boldsymbol{y})} - i\eta G(\boldsymbol{x},\boldsymbol{y})\right\}\varphi^s(\boldsymbol{y})dS(\boldsymbol{y}) + \frac{\varphi^s(\boldsymbol{x})}{2} = -u^i(\boldsymbol{x}), \;\; \boldsymbol{x}\in\partial D, $$ for some arbitrary real number $\eta\neq0$ such that $\eta\mathrm{Re}(k)\geq0$. Once the equation is solved for $\varphi^s$, the scattered field is given by $$ u^s(\boldsymbol{x}) =\int_{\partial D}\left\{\frac{\partial G(\boldsymbol{x},\boldsymbol{y})}{\partial\boldsymbol{n}(\boldsymbol{y})} - i\eta G(\boldsymbol{x},\boldsymbol{y})\right\}\varphi^s(\boldsymbol{y})dS(\boldsymbol{y}), \;\; \boldsymbol{x}\in\mathbb{R}^3\setminus{\overline{D}}. $$ The function $G$ is the Green's function of the Helmholtz equation in 3D, $$ G(\boldsymbol{x},\boldsymbol{y}) = \frac{1}{4\pi}\frac{e^{ik\vert\boldsymbol{x}-\boldsymbol{y}\vert}}{\vert\boldsymbol{x}-\boldsymbol{y}\vert}. $$ Challenges One of the challenges one faces when solving integral equations is the computation of singular integrals—when $\boldsymbol{x}$ is equal to $\boldsymbol{y}$, the Green's function and its derivatives are unbounded. There are many specialized methods to compute such integrals, including singularity subtraction, singularity cancellation, and the continuation approach. We described, in a previous blog post, algorithms to compute weakly singular integrals using singularity subtraction and the continuation approach. The term weakly singular refers to integrals with singularities of the same type as the Green's function. We describe, here, our method to strongly singular integrals, which have singularities of the same nature as the derivatives of the Green's function. Computing strongly singular integrals We consider strongly singular and near-singular integrals of the form $$ I(\boldsymbol{x}_0) = \int_{\mathcal{T}}\frac{(\boldsymbol{x} - \boldsymbol{x}_0)\cdot\boldsymbol{n}(\boldsymbol{x})}{\vert\boldsymbol{x}-\boldsymbol{x}_0\vert^3}\varphi(F^{-1}(\boldsymbol{x}))dS(\boldsymbol{x}), $$ where $\mathcal{T}\subset\mathbb{R}^3$ is a curved triangle defined by a polynomial transformation $F:\widehat{T}\to\mathcal{T}$ of degree $q\geq1$ from some reference planar triangle $\widehat{T}\subset\mathbb{R}^2$, $\boldsymbol{x}_0\in\mathbb{R}^3$ is a point on/or close to $\mathcal{T}$, $\varphi:\widehat{T}\to\mathbb{R}$ is a polynomial function of degree $p\geq0$ (not necessarily equal to $q$, and $\boldsymbol{n}(\boldsymbol{x})$ is the unit normal vector at $\boldsymbol{x}$. Our method is based on the computation of the preimage of the singularity in the reference element's space using Newton's method, singularity subtraction with Taylor-like asymptotic expansions, the continuation approach, and transplanted Gauss quadrature. Step 1. We map $\mathcal{T}$ back to $\widehat{T}$, $$ I(\boldsymbol{x}_0) = \int_{\widehat{T}}\frac{(F(\boldsymbol{\widehat{x}})-\boldsymbol{x}_0)\cdot\boldsymbol{\widehat{n}}(\boldsymbol{\widehat{x}})}{\vert F(\boldsymbol{\widehat{x}})-\boldsymbol{x}_0\vert^3}\varphi(\boldsymbol{\widehat{x}})dS(\boldsymbol{\widehat{x}}), $$ where $\boldsymbol{\widehat{n}}(\boldsymbol{\widehat{x}}) = F_{\widehat{x}_1}(\boldsymbol{\widehat{x}})\times F_{\widehat{x}_2}(\boldsymbol{\widehat{x}})$. Step 2. We compute $\boldsymbol{\widehat{x}}_0\in\mathbb{R}^2$ such that $F(\boldsymbol{\widehat{x}}_0)\in\mathcal{T}$ is the closest point to $\boldsymbol{x}_0$ on $\mathcal{T}$ (this is done with numerical optimization). We then decompose $\boldsymbol{x}_0$ as $$ \boldsymbol{x}_0 = F(\boldsymbol{\widehat{x}}_0) + h\frac{\boldsymbol{\widehat{n}}_0}{\vert\boldsymbol{\widehat{n}}_0\vert}, \quad \boldsymbol{\widehat{n}}_0 = \boldsymbol{\widehat{n}}(\boldsymbol{\widehat{x}}_0), $$ for some scalar $h$, which may be negative, obtained via $$ h = (\boldsymbol{x}_0 - F(\boldsymbol{\widehat{x}}_0)) \cdot \frac{\boldsymbol{\widehat{n}}_0}{\vert\boldsymbol{\widehat{n}}_0\vert}. $$ Step 3. We compute the strongly singular term, $$ T_{-2}(\boldsymbol{\widehat{x}},h) = -\frac{h\varphi_0\vert\boldsymbol{\widehat{n}}_0\vert}{\left[\vert J_0(\boldsymbol{\widehat{x}} - \boldsymbol{\widehat{x}}_0)\vert^2 + h^2\right]^{\frac{3}{2}}}, $$ where $J_0$ is the Jacobian matrix at $\boldsymbol{\widehat{x}}_0$ and $\varphi_0=\varphi(\boldsymbol{\widehat{x}}_0)$, as well as the weakly singular term $T_{-1}$. We subtract them from/add them to the integrand, $$ I(\boldsymbol{x}_0) = \, \int_{\widehat{T}}\left[\frac{(F(\boldsymbol{\widehat{x}})-\boldsymbol{x}_0)\cdot\boldsymbol{\widehat{n}}(\boldsymbol{\widehat{x}})}{\vert F(\boldsymbol{\widehat{x}})-\boldsymbol{x}_0\vert^3}\varphi(\boldsymbol{\widehat{x}}) - T_{-2}(\boldsymbol{\widehat{x}},h) - T_{-1}(\boldsymbol{\widehat{x}},h)\right]dS(\boldsymbol{\widehat{x}}) \\ + \int_{\widehat{T}}T_{-2}(\boldsymbol{\widehat{x}},h)dS(\boldsymbol{\widehat{x}}) + \int_{\widehat{T}}T_{-1}(\boldsymbol{\widehat{x}},h)dS(\boldsymbol{\widehat{x}}). $$ The first integral is regularized—it may be computed with Gauss quadrature on triangles. The last two integrals are singular or near-singular and will be computed in steps 4–5. Step 4. Let $$ \begin{align} I_{-2}(h) = \int_{\widehat{T}}T_{-2}(\boldsymbol{\widehat{x}},h)dS(\boldsymbol{\widehat{x}}), \quad I_{-1}(h) = \int_{\widehat{T}}T_{-1}(\boldsymbol{\widehat{x}},h)dS(\boldsymbol{\widehat{x}}). \end{align} $$ Since $T_{-2}$ and $T_{-1}$ are homogeneous in $\boldsymbol{\widehat{x}}$ and $h$, using the continuation approach, we reduce the 2D integrals to a sum of three 1D integrals along the edges of the shifted triangle $\widehat{T}-\boldsymbol{\widehat{x}}_0$. For instance, for $I_{-2}$, this yields $$ I_{-2}(h) = -\mathrm{sign}(h)\varphi_0\vert\boldsymbol{\widehat{n}}_0\vert\sum_{j=1}^3\widehat{s}_j\int_{\partial\widehat{T}_j-\boldsymbol{\widehat{x}}_0}\frac{\sqrt{\vert J_0\boldsymbol{\widehat{x}}\vert^2 + h^2}-\vert h\vert}{\vert J_0\boldsymbol{\widehat{x}}\vert^2\sqrt{\vert J_0\boldsymbol{\widehat{x}}\vert^2 + h^2}}ds(\boldsymbol{\widehat{x}}), $$ where the $\widehat{s}_j$'s are the distances from the origin to the edges of $\widehat{T}-\boldsymbol{\widehat{x}}_0$. Step 5. To circumvent near-singularity issues in the computation of $I_{-2}$ and $I_{-1}$, we employ transplanted quadrature when the singularity is close to the edges. We take advantage of the a priori knowledge of the singularities to utilize transplanted rules with significantly improved convergence rates. Numerical experiments We consider the sound-soft scattering of a plane wave $u^i(r,\theta)=e^{ikr\cos\theta}$ by the unit sphere. We choose $\eta=k/2$ and discretize the boundary integral equation with a boundary element method with linear basis functions/planar triangles ($p=q=1$) and quadratic basis functions/triangles ($p=q=2$). We take $k=2\pi$, solve for $\varphi^s$, and evaluate the far-field pattern $$ u_\infty(\boldsymbol{\theta}) = \frac{1}{4\pi}\int_{\partial D} e^{-ik\boldsymbol{\theta}\cdot\boldsymbol{y}}\varphi^s(\boldsymbol{y})dS(\boldsymbol{y}), \;\; \boldsymbol{\theta}\in\mathbb{S}^2, $$ for an increasing number of triangles. We observe quartic superconvergence for quadratic elements as the mesh size $h\to0$. We now take $k\in\{2\pi,4\pi,8\pi,16\pi\}$, solve for $\varphi^s$ and seek the number of degrees of freedom needed to reach a relative error on the far-field pattern around $10^{-3}$ for each $k$. We observe that using quadratic elements reduces the number of degrees of freedom by a factor of about four, while also decreasing computer time by a similar factor at high frequency. This is how the solutions look like. Check out our paper for details!

The linear sampling method for small random scatterers https://hadrien-montanelli.github.io/2024-02-04.html 2024-02-04T00:00:00+00:00

In this post, I talk about an extension of the linear sampling method (LSM) for solving the sound-soft inverse acoustic scattering problem with data generated by randomly distributed small scatterers. For details, check out my paper with Houssem Haddar and Josselin Garnier! Introduction Inverse scattering problems arise across a multitude of fields, ranging from medical imaging and non-destructive testing to radar technology and seismology. At their core, these problems involve the task of deducing the characteristics of an object or medium from the scattered signals it generates. These problems involve several challenges, including nonlinearity, which is particularly pronounced near resonance, making linearization inapplicable; these are also severely ill-posed, raising questions about uniqueness and stability, and necessitating the inclusion of regularization. Additionally, for iterative procedures, reconstruction time can be lengthy and prior knowledge is required for initiation. In this context, fast, data-driven algorithms, such as the LSM, can be useful in reducing computational costs, and initializing more sophisticated algorithms. Two acquisition configurations are commonly used in inverse scattering problems—active and passive. Active imaging involves sending waves through controlled sources and recording the medium's response through controlled sensors. Passive imaging, on the other hand, employs controlled sensors but relies on random, uncontrolled sources (such as microseisms and ocean swells in seismology). In this setup, it is the cross-correlations between the recorded signals that convey information about the medium. Passive imaging is a rapidly growing research topic because it enables the imaging of areas where the use of active sources is not possible due to, e.g., safety or environmental reasons. Additionally, even in scenarios where active sources could be employed, utilizing passive sources helps reduce operational costs and enhances stealth in defense applications. In a previous paper, we introduced an extension of the LSM to address the sound-soft inverse scattering problem involving random sources. This current study builds upon that foundation, focusing on a scenario where a single controlled source, operating at a specified wavelength $\lambda$, illuminates a small random scatterer and an object of comparable size to $\lambda$—the goal is to reconstruct the shape of the latter. The reflection of the wave field transmitted by the point source on the small scatterer serves as a random source, aligning with the context explored in our earlier work. The motivation for this research stems from the necessity, when imaging an unknown obstacle $D$, to illuminate it with a diverse set of incoming waves. For deterministic, controlled point sources, one considers a family of sources located at points $\boldsymbol{z}_m$ generating the incident fields $$ \begin{align} \phi(\boldsymbol{x},\boldsymbol{z}_m) = \frac{i}{4}H_0^{(1)}(k\vert\boldsymbol{x}-\boldsymbol{z}_m\vert), \end{align} $$ and measures the resulting scattered fields $u^s(\boldsymbol{x}_j,\boldsymbol{z}_m)$ at points $\boldsymbol{x}_j$ to populate the near-field matrix $N$ with entries $N_{jm} = u^s(\boldsymbol{x}_j,\boldsymbol{z}_m)$. In the case of random, uncontrolled sources at positions $\boldsymbol{z}_\ell$, as demonstrated in our prior work, the relevant matrix is the cross-correlation matrix $C$. Its entries are given by: $$ \begin{align} C_{jm} = \frac{2ik\vert\Sigma\vert}{L}\sum_{\ell=1}^L\overline{u(\boldsymbol{x}_j,\boldsymbol{z}_\ell)}u(\boldsymbol{x}_m,\boldsymbol{z}_\ell) - \left[\phi(\boldsymbol{x}_j,\boldsymbol{x}_m) - \overline{\phi(\boldsymbol{x}_j,\boldsymbol{x}_m)}\right], \end{align} $$ where $u(\boldsymbol{x},\boldsymbol{z})=\phi(\boldsymbol{x},\boldsymbol{z})+u^s(\boldsymbol{x},\boldsymbol{z})$ is the total field for the incident wave $\phi(\boldsymbol{x},\boldsymbol{z})$, and $\vert\Sigma\vert$ is the area of the surface $\Sigma$ on which the random sources are distributed. Suppose now that we only have a single controlled source located at some given point $\boldsymbol{z}$, as opposed to several positions $\boldsymbol{z}_m$. This is insufficient for reconstructing the obstacle's shape with the LSM (the near-field matrix $N$ would have rank one). To address this limitation, we propose introducing a random medium between the source and the obstacle. We illustrate this concept, in acoustic scattering, by considering a single small random scatterer between the source and the obstacle—a seemingly simple yet powerful model of a random medium that allows us to apply the LSM in a novel manner, with a modified version of $C$. Modified Helmholtz–Kirchoff identity Consider the incident field $\phi(\boldsymbol{x},\boldsymbol{z}_\epsilon)$ generated by a point source located at $\boldsymbol{z}_\epsilon=\lambda\epsilon^{-q}e^{i\theta_z}$ for some scalars $\epsilon>0$, $q>0$, and $\theta_z\in[0,2\pi]$. Here, $k>0$ is the wavenumber and $\lambda=2\pi/k$ is the wavelength. Let $D$ be an obstacle of size proportional to $\lambda$ and independent on $\epsilon$. Without loss of generality, we assume that $D$ is centered at the origin. We also consider a small disk $D_\epsilon=D_\epsilon(\boldsymbol{y}_\epsilon)$ of radius $\rho(D_\epsilon)=\lambda\epsilon$ centered at $\boldsymbol{y}_\epsilon = \lambda\epsilon^{-p}e^{i\theta_y}$ for some scalars $0 We examine the scattering of the incident field \(\phi(\cdot,\boldsymbol{z}_\epsilon)$ by $D$ and $D_\epsilon$, which generates the scattered field $w^s_\epsilon$. More precisely, let $w^s_\epsilon(\cdot,\boldsymbol{y}_\epsilon,\boldsymbol{z}_\epsilon)$ be the solution to the sound-soft scattering problem $$ \begin{align}\label{eq:w^s} \left\{ \begin{array}{ll} \Delta w^s_\epsilon(\cdot,\boldsymbol{y}_\epsilon,\boldsymbol{z}_\epsilon) + k^2 w^s_\epsilon(\cdot,\boldsymbol{y}_\epsilon,\boldsymbol{z}_\epsilon) = 0 \quad \text{in $\mathbb{R}^2\setminus\{\overline{D\cup D_\epsilon}\}$}, \\[0.4em] w^s_\epsilon(\cdot,\boldsymbol{y}_\epsilon,\boldsymbol{z}_\epsilon) = -\phi(\cdot,\boldsymbol{z}_\epsilon) \quad \text{on $\partial D\cup\partial D_\epsilon$}, \\[0.4em] \text{$w^s_\epsilon(\cdot,\boldsymbol{y}_\epsilon,\boldsymbol{z}_\epsilon)$ is radiating}. \end{array} \right. \end{align} $$ We showed that $w^s_\epsilon(\cdot,\boldsymbol{y}_\epsilon,\boldsymbol{z}_\epsilon)$ may be approximated as the sum of three terms, $$ \begin{align} w_\epsilon^s(\cdot,\boldsymbol{y}_\epsilon,\boldsymbol{z}_\epsilon) \approx u^s(\cdot,\boldsymbol{z}_\epsilon) + v_\epsilon^i(\cdot,\boldsymbol{y}_\epsilon,\boldsymbol{z}_\epsilon) + v_\epsilon^s(\cdot,\boldsymbol{y}_\epsilon,\boldsymbol{z}_\epsilon) \end{align} $$ with an error $\mathcal{O}(\vert\log\epsilon\vert^{-1}\epsilon^p\epsilon^{q/2})$ in the $H^1(B)$-norm; the scattered fields in the expansion above are the solutions to the sound-soft scattering problems $$ \begin{align}\label{eq:u^s} \left\{ \begin{array}{ll} \Delta u^s(\cdot,\boldsymbol{z}_\epsilon) + k^2 u^s(\cdot,\boldsymbol{z}_\epsilon) = 0 \quad \text{in $\mathbb{R}^2\setminus\overline{D}$}, \\[0.4em] u^s(\cdot,\boldsymbol{z}_\epsilon) = -\phi(\cdot,\boldsymbol{z}_\epsilon) \quad \text{on $\partial D$}, \\[0.4em] \text{$u^s(\cdot,\boldsymbol{z}_\epsilon)$ is radiating}, \end{array} \right. \end{align} $$ $$ \begin{align}\label{eq:v^i} \left\{ \begin{array}{ll} \Delta v_\epsilon^i(\cdot,\boldsymbol{y}_\epsilon,\boldsymbol{z}_\epsilon) + k^2 v_\epsilon^i(\cdot,\boldsymbol{y}_\epsilon,\boldsymbol{z}_\epsilon) = 0 \quad \text{in $\mathbb{R}^2\setminus\overline{D_\epsilon}$}, \\[0.4em] v_\epsilon^i(\cdot,\boldsymbol{y}_\epsilon,\boldsymbol{z}_\epsilon) = -\phi(\cdot,\boldsymbol{z}_\epsilon) \quad \text{on $\partial D_\epsilon$}, \\[0.4em] \text{$v_\epsilon^i(\cdot,\boldsymbol{y}_\epsilon,\boldsymbol{z}_\epsilon)$ is radiating}, \end{array} \right. \end{align} $$ and $$ \begin{align}\label{eq:v^s} \left\{ \begin{array}{ll} \Delta v_\epsilon^s(\cdot,\boldsymbol{y}_\epsilon,\boldsymbol{z}_\epsilon) + k^2 v_\epsilon^s(\cdot,\boldsymbol{y}_\epsilon,\boldsymbol{z}_\epsilon) = 0 \quad \text{in $\mathbb{R}^2\setminus\overline{D}$}, \\[0.4em] v_\epsilon^s(\cdot,\boldsymbol{y}_\epsilon,\boldsymbol{z}_\epsilon) = -v^i_\epsilon(\cdot,\boldsymbol{y}_\epsilon,\boldsymbol{z}_\epsilon) \quad \text{on $\partial D$}, \\[0.4em] \text{$v_\epsilon^s(\cdot,\boldsymbol{y}_\epsilon,\boldsymbol{z}_\epsilon)$ is radiating}. \end{array} \right. \end{align} $$ This is how the scattered fields look like. We proved that the scattered fields verify the following modified Helmholtz–Kirchhoff identity, $$ \begin{align} u^s(\boldsymbol{x},\boldsymbol{x}') - \overline{u^s(\boldsymbol{x},\boldsymbol{x}')} \approx 2ik\sigma_\epsilon\int_{\Sigma_\epsilon}\overline{v_\epsilon(\boldsymbol{x},\boldsymbol{y}_\epsilon,\boldsymbol{z}_\epsilon)} v_\epsilon(\boldsymbol{x}',\boldsymbol{y}_\epsilon,\boldsymbol{z}_\epsilon)dS(\boldsymbol{y}_\epsilon) - [\phi(\boldsymbol{x},\boldsymbol{x}') - \overline{\phi(\boldsymbol{x},\boldsymbol{x}')}], \end{align} $$ with scaling factor $\sigma_\epsilon = \pi^2\vert H_0^{(1)}(2\pi\epsilon)\vert^2\epsilon^{-q}$. This motivates the introduction of the following modified cross-correlation matrix: $$ \begin{align} \widetilde{C}_{jm} = \frac{2ik\vert\Sigma_\epsilon\vert\sigma_\epsilon}{L}\sum_{\ell=1}^L\overline{v_\epsilon(\boldsymbol{x}_j,\boldsymbol{y}_\epsilon^\ell,\boldsymbol{z}_\epsilon)} v_\epsilon(\boldsymbol{x}_m,\boldsymbol{y}_\epsilon^\ell,\boldsymbol{z}_\epsilon) - \left[\phi(\boldsymbol{x}_j,\boldsymbol{x}_m) - \overline{\phi(\boldsymbol{x}_j,\boldsymbol{x}_m)}\right]. \end{align} $$ Numerical experiments We consider a kite of size $\lambda/2$ centered at $2\lambda + 2\lambda i$ for the wavenumber $k=2\pi$ (wavelength $\lambda =1$). We take $\epsilon=10^{-2}$, $p=1$, $q=2$, and $\theta_z=\pi$ for the asymptotic model, which yields $\boldsymbol{z}_\epsilon=-10000$. For the LSM, we take $J=120$ equispaced sensors on the circle of radius $5\lambda=5$, $$ \begin{align} \boldsymbol{x}_j = 5e^{i\theta_j}, \quad \theta_j = \frac{2\pi}{J}(j - 1), \quad 1\leq j\leq J, \end{align} $$ and $L=120$ different positions of a single small scatterer on the circle of radius $\lambda\epsilon^{-p}=100$, \begin{align}\label{eq:scatterer-beta} \boldsymbol{y}_\epsilon^\ell = 100e^{i\theta_y^\ell}, \quad \theta_y^\ell = 2\pi\beta_\ell, \quad 1\leq \ell\leq L, \end{align} where $\beta_\ell\sim U(0,1)$ is drawn from the uniform distribution on $[0,1]$. Finally, we add some multiplicative white noise with amplitude $\delta=5\times 10^{-3}$ to the the near-field measurements (generating a matrix $\widetilde{C}_\delta$), and probe the medium on a $100\times100$ uniform grid on $[-6\lambda,6\lambda]\times[-6\lambda,6\lambda]$. The defect is well identified by our novel sampling method, based on cross-correlations and a small random scatterer. This illustrates that the LSM can be utilized in this novel passive-imaging setup.

Teaching SciML https://hadrien-montanelli.github.io/2025-03-24.html 2025-03-24T00:00:00+00:00

In this post, I talk about a course I just finished teaching at École Polytechnique on scientific machine learning. Introduction The course was aimed at third-year students at École Polytechnique, who had completed two years of classes préparatoires, making the material equivalent to a graduate-level course in the UK or US. This Scientific Machine Learning (SciML) course bridged the gap between Scientific Computing and Machine Learning. SciML, an emerging field, enables efficient methods for solving complex problems in science and engineering, like partial differential equations (PDEs). This course offered a new, valuable specialization, helping students stand out in the evolving job market, where demand for ML experts with specialized skills is growing. Scientific computing and numerical analysis Scientific Computing and Numerical Analysis are branches of applied mathematics that focus on the development and analysis of numerical algorithms for solving problems in continuous mathematics, such as PDEs. When a PDE is linear, we first discretize it on a grid or mesh and then solve it using linear algebra. This creates a natural connection between PDEs and Numerical Linear Algebra (NLA). Sometimes, when an energy functional exists for a problem (e.g., PDEs with coercive bilinear forms like the Laplace equation), we can also apply optimization techniques to solve it effectively. A typical approach to solving nonlinear PDEs involves transforming the problem into a sequence of linear problems, through linearization and iteration. In this way, nonlinear PDEs are reduced to linear algebra problems. Scientific machine learning SciML is a recent research field based on both machine learning and scientific computing. Its goal is the development of robust, efficient, and interpretable methods to solve problems in science and engineering, such as PDEs, parameter identification, or inverse problems. In SciML, a common approach to solving nonlinear PDEs starts with generating training data, typically using standard numerical solvers, where the training data consists of reference solutions. This data is then used to construct a cost function that quantifies the training error, while the solution or solution operator is discretized using a neural network. The goal is to minimize this cost function through optimization to learn the optimal network weights. This brings optimization back to the core of the numerical solutions of PDEs. Overview Throughout the course, we touched upon all major fields of Scientific Computing and Numerical Analysis, including Numerical Linear Algebra, PDEs, and Optimization, with a ML flavor. Each method discussed was implemented in Python, and every tutorial provided hands-on experience through notebooks. This ensured that students not only understood the theoretical aspects but also gained practical experience in implementing and using these methods. Check out my lecture notes! Supervised Learning Chapter 1 — Supervised Learning without Neural Networks Linear and nonlinear least squares with QR and Gauss Newton's method Trigonometric interpolation with the FFT Chapter 2 — Supervised Learning with Neural Networks Artificial neural networks Training with stochastic gradient descent and backpropagation Chapter 3 — Neural Network Approximation Theory Universality theorems Quantitative estimates in Sobolev spaces Solving PDEs Chapter 4 — Solving PDEs without Neural Networks Finite elements for the 1D Poisson equation Finite elements with finite differences for the 1D heat equation Chapter 5 — Solving PDEs with Neural Networks Physics-informed neural networks Chapter 6 — Solving PDEs with Neural Networks 2 (Samuel Kokh) Solving Parametric PDEs Chapter 7 — Solving Parametric PDEs without Neural Networks Reduced basis methods with the SVD Chapter 8 — Solving Parametric PDEs with Neural Networks Deep operator networks Fourier neural operators

Writing research memos https://hadrien-montanelli.github.io/2025-05-23.html 2025-05-23T00:00:00+00:00

Welcome to the first entry in my Holiday in Sicily series. Over the next 13 days, while vacationing in Sicily, I’ll be posting here every other day. In today’s post, I reflect on the practice of writing research memos—why I do it, how it helps, and how surprisingly enjoyable the process can be. Introduction At his retirement conference from Oxford, Nick Trefethen gave a talk on writing research memos. He shared seven compelling reasons for doing so: Reason #1: Memos create a helpful record for the future. Reason #2: They provide day-to-day motivation. Reason #3: They serve as a repository for your old code. Reason #4: They become nice mementos over time. Reason #5: They help communicate progress to others. Reason #6: They offer great practice in writing and LaTeXing. Reason #7: (Wait a few years… you’ll see.) Nick, who was my PhD advisor, passed that habit on to me, and I’m so grateful he did. I’ve written over 200 memos—60 during my PhD, 50 more at Columbia, and 90 since returning to France. Now my students are picking it up as well: Victor wrote just under 20 in his first year, and Esteban, who just started, has already completed his first one. Nice work, guys! But what is a memo? Before I highlight the reasons I’ve found this habit so valuable—and add a couple of my own—let me explain. What is a research memo? A memo is a short research report, typed in LaTeX and generated as a PDF—I like using a two-column format. It synthesizes the progress (or lack thereof) made over a short period of time, typically a week or two. A memo blends formal mathematics when the work is theoretical, with updates on coding efforts, numerical results, and figures. It’s meant to be concise, rigorous, and practical—a snapshot of where things stand. To keep track of them, I maintain a summary table in Notion (which I love and highly recommend). Each entry includes a brief description of the memo’s content and a tag corresponding to the project it belongs to—e.g., Nonlocal PDEs, LSM with random medium, or SciML—so that everything can be easily sorted and filtered. Now, back to the reasons this habit has been so valuable. Reason #1: A helpful record for the future One of the worst feelings I had during my BSc and MSc was knowing I’d done a computation before, but not remembering how, and being unable to find the draft. We can’t remember everything—and that’s okay. But we must know where to look. Whenever I do a new computation or learn something that requires a bit of digestion, I type it up in a memo. Weeks, months, or years later, when I need it again, I just open the memo and—boom—everything comes flooding back. It’s a wonderful feeling. Reason #2: Day-to-day motivation As Nick mentioned in his talk, books take years, papers take months, but memos take just a day or two. That makes them a perfect tool for staying engaged with research on a daily basis. This is especially valuable during a PhD—particularly in mathematics—when progress can be slow and setbacks frequent. Getting stuck is part of the process, but it can take a toll on your mental health. Impostor syndrome is real. Writing memos regularly, even just to document things that didn’t work, helps maintain momentum. And it ties back to Reason #1: when you revisit a paused project, a well-written memo reminds you exactly what failed and why—saving you from repeating the same mistakes. And beyond the practical benefits, there’s something deeply rewarding about finishing a memo. Printing it out, proofreading it, and adding it to the growing stack gives you a tangible sense of progress—a small but satisfying moment of self-achievement that keeps you going. Reason #5: Communicating progress to others I’ve had some fantastic blackboard sessions—brainstorming with Mikael Slevinsky on nonlocal operators on the sphere, and more recently with Luiz Faria and Pierre Marchand on convergence rates for curved boundary elements. But every productive chalkboard exchange was always followed by something more: a quick photo of the board, then a memo or two. That’s where the real value crystallized. The blackboard is great for sparking ideas, but the memo is where those ideas are clarified, organized, and made durable. When it comes to presenting results—especially to colleagues or supervisors—nothing beats a well-written memo. Writing forces clarity. It sharpens your arguments, ensures consistent notation, and gives structure to your thoughts. You’ve already done the mental work of deciding what matters and how to communicate it. By contrast, unprepared explanations at the board often fall flat. The handwriting is difficult to decipher, the argument becomes unclear, and the notation is often inconsistent or poorly chosen. Too many times I’ve watched someone try to explain a result on the fly, only to get tangled in their own logic or lost in unclear symbols. I’ve had to interrupt just to restore some clarity—and that’s frustrating for everyone involved. A memo avoids all of this. It respects your own time, and it respects the time and attention of your audience. Reason #6: Practicing writing and LaTeX This one is crucial, especially for graduate students. Mathematical writing is hard. English writing (for non-natives) is hard. LaTeXing is hard. And you only get better by doing it often. It’s then the role of the supervisor—as a mentor—to help you improve by pointing out what needs fixing: shaky reasoning, unclear explanations, inconsistent notation, awkward English, LaTeX quirks, formatting issues. That’s why reviewing memos is so valuable. Nick used to read his students’ memos before meetings, and I try to do the same. You get better by practicing; I support that practice by giving feedback. Each memo is an opportunity to sharpen your skills and grow as a researcher. Bonus: Writing papers (and your thesis) becomes fun I usually write papers in a week. Why so fast? Because each paper is built from half a dozen polished memos. Writing the paper becomes a matter of stitching them together into a story. The mathematics, proofs, and most figures are already done. All that’s left is to make it look good—clear, neat, and polished. It’s fun. I wrote my PhD thesis in seven weeks. It had five chapters, plus an introduction and a conclusion—each chapter roughly corresponding to a research paper. Because the memos and papers were already polished, the writing process was smooth—and enjoyable. The thesis was accepted with no corrections (about 1% of maths PhD theses at Oxford achieve that). Bonus: Focus your brainpower where it matters When I’m doing pen-and-paper research, I keep all relevant memos printed on my desk or displayed on my screens. This way, if I need a specific result, I don’t waste mental energy recalling it—I just glance at the memo. Was it $i\pi$ or $-i\pi$? Did the result require a compact operator or just a bounded one? I don’t need to remember—I just look it up. And that frees up my mind to focus on creating new ideas and making deeper connections.

When theory meets practice—and they argue productively https://hadrien-montanelli.github.io/2025-05-25.html 2025-05-25T00:00:00+00:00

Welcome to the second entry in my Holiday in Sicily series. Today, I reflect on the interplay between algorithms and theory—how practice often leads, theory follows, and how the two shape each other in a dynamic loop that drives much of applied math progress. Introduction In many fields of applied mathematics and engineering, the relationship between theory and practice isn’t a one-way street. It’s a conversation. At its best, it’s a lively argument. Often, the most exciting developments come from this back-and-forth—a dynamic feedback loop between algorithms and theory. Practice speaks first: a new method appears, driven by need or intuition. Then theory replies, asking tough questions. Does it work? When? Why? Sometimes, theory pushes back, exposing flaws or failure modes. Practice listens, adjusts, and counters with a better design. And so the exchange continues—design, analysis, failure, improvement. A real dialogue. Each side learning, challenging, shaping the other. The pattern This pattern can be broken down into five stages: It starts with an algorithm.Someone proposes a new method. Maybe it’s computationally efficient, maybe it’s inspired by physics, maybe it’s a neural network architecture copied from another domain. The key point: it’s motivated by what works, not necessarily by what can be proved. Then comes theory.After initial empirical validation, theoretical work seeks to rigorously understand the algorithm: Is it stable? Does it converge? Under what conditions? Is it optimal? These proofs don’t just validate the method—they clarify its domain of applicability and often reveal surprising subtleties. Theory explains failure modes.Theory doesn’t just confirm what works. It explains why it fails under certain conditions. It identifies the edge cases: when the matrix is ill-conditioned, when the PDE becomes singular, when the neural network can’t generalize. These insights often reveal flaws in the original method—and point the way forward. Failures lead to new designs.Armed with new theoretical insight, practitioners redesign the algorithm: they precondition it to handle ill-conditioned matrices; they stabilize the discretization to cope with singular PDEs; they regularize or reshape the architecture to improve generalization in neural networks. Now there’s a new method—a fix, a patch, or sometimes a breakthrough. Theory circles back.The cycle completes as the community begins to rigorously analyze these new approaches, proving stability, convergence, or optimality. The feedback loop spins again—and the field takes another step forward. This cycle is more than just a historical curiosity. It reflects a healthy research ecosystem where algorithms and theory co-evolve, each pushing the other forward. Classic example: the finite element method In the 1960s and 1970s, engineers began using the finite element method (FEM) for solving PDEs in structural mechanics. Early algorithms were developed based on physical intuition and computational feasibility, often with minimal theoretical backing. As usage spread, mathematicians stepped in to analyze convergence, consistency, and error bounds. This theoretical work revealed the importance of mesh quality, element choice, and function spaces—leading to remedies such as adaptive mesh refinement and stabilized formulations. These in turn became subjects of rigorous mathematical analysis, culminating in the powerful FEM theory we now rely on. Modern example: neural networks The past decade has seen a similar cycle in machine learning. Deep neural networks first exploded in popularity due to empirical success in computer vision and natural language processing. The original architectures—like AlexNet—worked well, but few could explain why. Soon after, theoretical work began to address generalization, expressivity, and training dynamics. Researchers identified failure modes: vanishing gradients, overfitting, sensitivity to initialization. Each insight led to remedies like residual connections, batch normalization, and dropout. These new components became standard architecture elements, and new theoretical frameworks arose to understand them—such as neural tangent kernels, overparameterization theory, and mean-field limits. Why it matters Understanding this pattern helps us better appreciate the non-linear path of scientific progress. It reminds us that: Empirical success can precede theory.Not every good idea starts with a theorem. Failures are fertile ground.When algorithms fail, it often sparks the most productive theoretical advances. Theory isn’t just for validation.It can actively shape new methods, sometimes in surprising ways. If you’re a theorist, don’t wait for the perfect question. Look at what practitioners are doing—it’s full of open problems. If you’re a practitioner, don’t be afraid to work with unproven methods. The theory might catch up—and when it does, it’ll make your work even better.

Niche and deep vs broad and new https://hadrien-montanelli.github.io/2025-05-27.html 2025-05-27T00:00:00+00:00

Welcome to the third entry in my Holiday in Sicily series. Today’s reflection comes from a quiet morning walk in Taormina. It got me thinking about how researchers choose between digging deeper into old, technical questions and venturing out into broad, unfamiliar territory. Both paths have their charm—and their challenges. The classic path: niche and deep Much of traditional academic research follows a well-worn pattern. You take a known problem and generalize it—bit by bit. In my corner of applied math, this often means scattering problems, and the progression looks something like this: 1D → 2D → 3D; Constant coefficient → smooth coefficient → non-smooth coefficient; Time-harmonic → time-dependent; Laplace equation → Helmholtz equation → Maxwell’s or elasticity equations. This is the incremental path. It’s rigorous, methodical, and deeply satisfying—like a well-reduced ragù. You start with something simple and add layers until you have something rich, complex, and complete. Pros: You become an expert. You know the literature, the open problems, the tools. You can guide students and propose research topics with confidence. You know what’s too easy, what’s too hard, and what might become a PhD or postdoc subject. You can make sharp conjectures. You see structure and patterns others miss. The theory is well developed. You have energy estimates, compactness arguments, variational formulations—tools that work. Cons: It gets harder to publish. Journals want novelty, not the fifth lemma on yet another variant of a known method. The community gets smaller. Your work may be elegant, but fewer people notice. The impact fades. It can feel slow. Progress is incremental, and big leaps are rare. The adventurous path: broad and new Then there’s the other route: jumping into something new. A new model, a new field, a new intersection. Maybe it’s inverse problems meets neural networks. Maybe it’s PDEs meets large language models. It’s less like a slow-simmered sauce, more like wandering through a Sicilian street market—you don’t know what you’ll find, but it’s exciting. Pros: It’s easy to publish. Everything is new. Even basic observations can be contributions. The community is large and curious. People are watching. If you hit the right nerve, your ideas can spread quickly. It can feel fast. Breakthroughs happen early and often, because no one’s explored the space yet. Cons: You’re not an expert. You don’t know the terminology, the norms, or the landscape. It’s hard to guide students or propose research topics—you’re unsure what’s feasible, what’s too simple, or what’s completely out of reach. You struggle to make good conjectures. There’s little structure, and few known results to build on. The theory is missing. You're improvising without a reliable framework or classical tools. The sweet spot There’s no universal answer. But good research often lives at the boundary between the two: old methods in new contexts, new methods applied to old problems. Like blending Sicilian ingredients with northern technique. You don’t need to abandon your expertise—but you might need to take it on a little vacation. As for me, I like to keep one foot in the familiar and one in the unknown. It’s a good way to stay grounded—and stay curious. One of my PhD students is working on neural operators to regularize the linear sampling method for inverse problems—a new tool meeting an old friend. Another is following a more traditional path, extending my work on inverse problems in random acoustic media to the elastic setting.

Mathematics near the volcano—rigor, intuition, and Bourbaki https://hadrien-montanelli.github.io/2025-05-29.html 2025-05-29T00:00:00+00:00

Welcome to the fourth entry in my Holiday in Sicily series. Starting with this post—and even more so in the next few—my opinions will become progressively more controversial. Call it the mathematical equivalent of hiking closer to Mount Etna: things might get heated, and by blog seven, expect a full eruption. Today, I reflect on the influence of Bourbaki on French mathematics, and the tension between rigor and intuition in how we learn, teach, and research. The legacy of Bourbaki The legacy of Nicolas Bourbaki—an anonymous collective of mostly French mathematicians formed in the 1930s—still looms large over the teaching and practice of mathematics in France. Whether celebrated or criticized, Bourbaki’s influence has shaped not just what French mathematicians learn, but how they think about mathematics at its deepest levels. At the heart of Bourbaki’s project was a radical vision: to rebuild all of mathematics on a rigorous, axiomatic foundation. The result was a monumental, multi-volume treatise covering much of pure mathematics in a uniform and abstract style. The language was formal, the structure logical, and the aesthetic minimalist. Proofs were king; diagrams, figures, and concrete examples were largely ignored. The Bourbaki approach made rigor a key priority. This had clear benefits: it clarified definitions, brought different areas of math together, and trained generations of French students to avoid sloppy reasoning. It also helped establish France as a powerhouse of pure mathematics in the 20th century. But this rigor came at a cost. In many French curricula, particularly in the post-war decades, intuition and visual understanding were often treated with suspicion. Concrete examples were rare. Applied mathematics was sidelined. A generation of students learned the French equivalent of New Math and abstract algebra in high school—where even addition was defined via group theory—while numerics and algorithms remained sidelined in much of French mathematical culture and seen as less elegant than theory. Rigor in mathematical statements A mathematical statement can often be expressed in more than one language—plain English, formal logic, or a blend of both. Take the definition of a limit, for example. In plain English: As $x$ gets close to $a$, $f(x)$ gets close to $L$. In formal mathematical language: $\forall\epsilon>0,\,\exists\delta>0,\,\vert x − a\vert\leq\delta\,\Longrightarrow\,\vert f(x) − L\vert\leq\epsilon$. And in practice, most mathematicians use a mix of both when thinking or teaching: For any small error $\epsilon$, we can find a range $\delta$ so that whenever $x$ is within $\delta$ of $a$, $f(x)$ stays within $\epsilon$ of $L$. Each version captures the same core idea, but they speak to different audiences and serve different purposes—intuition, rigor, or communication. The Bourbaki style favored the most formal, symbolic version, where ambiguity is impossible but accessibility can suffer. In high school, we usually encounter the plain English version. We talk about “getting close” or “approaching a value,” and that’s often enough to grasp the core idea. By the time we reach the first years of university, we learn how to rewrite those ideas using the precise machinery of logic and quantifiers. It can feel like learning a new language—one with no room for vagueness, but also less space for intuition. Then, in graduate school, something surprising happens: we start mixing the two. A well-trained mathematician will switch fluidly between rigor and intuition, between formal statements and verbal sketches. This blend isn’t a failure of rigor—it’s a mark of fluency. And yet, the Bourbaki tradition often treats only the formal register as truly mathematical. Rigor in mathematical writing That mindset has consequences—not just in how mathematics is taught in France, but also in how it’s written. Both French textbooks and research papers tend to adopt an extremely terse, notation-heavy style, with minimal exposition or story-telling. The result is dense, compact, and often impenetrable to the uninitiated. It’s not just a question of clarity—it’s a question of style, culture, and what we think mathematical writing should do. In the French tradition, if a paragraph contains more words than symbols, it starts to feel suspiciously verbose. In contrast, Anglo-American papers often aim for a balance between notation and narrative, guiding the reader through the logic rather than expecting them to reconstruct it from raw expressions. Of course, compactness has its virtues—it can be elegant, efficient, and precise. But when it comes at the expense of accessibility, communication suffers. Mathematics is already hard. We don’t need to make it harder with a style that hides ideas behind walls of notation. The cult of definitions Beyond the obsession with formal logic, there’s another hallmark of the French mathematical style: a deep, almost compulsive need to define everything. In a typical French undergraduate program, we start from scratch. Week one: propositional logic, quantifiers, truth tables. Week two and three: elementary group theory. Then we painstakingly construct the real numbers from the rationals, which themselves were constructed from the integers, which came from the naturals. Only after all that groundwork, do we get to start analysis or algebra. Compare that to Baby Rudin—the classic American textbook for analysis. (Its real title—pun very much intended—is Principles of Mathematical Analysis by Walter Rudin.) Rudin opens with a refreshingly honest sentence: “We shall not enter into any discussion of the axioms that govern the arithmetic of integers, but assume familiarity with the rational numbers.” No apologies, no constructions, no set theory detours. The point isn’t that the construction of $\mathbb{R}$ is unimportant—it’s that analysis is the main story. It’s a small example, but it reveals a broader cultural difference. In the French style, you don’t touch a theorem until you’ve built the entire scaffold beneath it. In the Anglo-American style, you’re allowed to see the landscape before learning how to pave the road. Rigor vs. reality This difference in approach isn’t just pedagogical—it reflects a deeper philosophy about what mathematics is. In the French tradition, there’s a sense that we must define every object, justify every step, and rebuild every tower from its foundations. No shortcuts, no assumptions, no room for ambiguity. But in practice—especially in research—we don’t always work that way. We write for a community that shares a common background, a shared mathematical culture. We assume our readers know what a Banach space is, or what “compactly supported” means. We don’t re-define continuity every time we use it. As a recent Quanta Magazine article puts it, “You can think of a proof as a social compact — a sort of mutual agreement between the author and their mathematical community.” That compact means I don’t need to define every term or prove every lemma from scratch—because I trust that you, the reader, already speak the same language. Sometimes, mathematical rigor isn’t about saying everything—it’s about knowing what doesn’t need to be said. This philosophical split extends beyond how we teach and write—it shapes how we do research. In the French tradition, a typical approach might start from a known theorem and attempt to generalize it. You begin with theory and formal rigor, and only later explore algorithms, numerical experiments, and applications. In contrast, the Anglo-American style tends to lead with intuition. You try an idea, run a numerical experiment, and interact with your code. If the algorithm works—great. The priority is pragmatic: do something first, and only then figure out why it works. Theory can come later. (I touched on this in a previous blog post on theory and practice.) Of course, most of us live somewhere between the two extremes. But it’s worth recognizing that there are multiple valid paths through mathematics. One begins with formalism and builds upward. The other starts with a feeling, an idea, a sketch—and only afterward puts the house in order. Both can lead to beauty. Both can lead to truth. Anecdotes from the front lines Over the years, I’ve had countless reminders—sometimes funny, sometimes frustrating—of just how deep these cultural patterns run. One that sticks with me happened during an informal conversation with some colleagues. I was describing an idea—nothing too technical, just sketching out the motivation and rough structure of a problem. After a few minutes of me talking about the setup, one of them looked impatient and finally said, “Yes, but what are the equations?” They could not engage with the idea until it was expressed in symbolic form. The story, the context, the motivation—all were just noise until the math appeared. I also remember my first-year undergraduate professor, who refused to draw a figure of an orthogonal projection. “It’s in $n$ dimensions,” he said. “There’s no point trying to visualize it.” That response has always stayed with me. It wasn’t just a personal quirk—it was a pedagogical stance, born of a tradition that treats geometry and visual intuition as dangerously imprecise. And then there was the time I was giving a talk—aimed at a general audience of applied mathematicians—and someone interrupted me mid-slide. “You haven’t listed all your assumptions!” they said. But it was a warm-up slide, and the assumptions were entirely standard for the context. Still, the expectation was that every detail be written explicitly, even at the cost of disrupting the flow or obscuring the main idea. Why diversity matters These moments can be annoying in the moment, but they also reflect a deeper truth: we are shaped by the mathematical cultures we grow up in. That’s why it’s so important to travel—to work in different countries, to listen to different voices, to be humbled by different ways of doing things. It’s one of the most important things I’ve learned. It opens your eyes to different mathematical cultures, different teaching styles, different assumptions about what needs to be said and what can go unsaid. It teaches humility—the ability to recognize that there are other ways of doing mathematics, and that yours isn’t necessarily the best or only one. A France 2 video about Oxford captures this beautifully: a researcher explains that in her lab, there are about 15 to 20 researchers and nearly as many nationalities. This diversity isn’t just cosmetic—it brings a diversity of ideas, habits, and intuitions. It’s valuable for French students and researchers to spend time abroad, to gain new perspectives and experience different ways of doing mathematics. But just as important is the need for French labs to be more open and welcoming to researchers trained elsewhere. (Given the current situation across the Atlantic, now may be a particularly good time.) Diversity of background brings diversity of thought—and that can only enrich the mathematical community. More on this in post seven, when Mount Etna finally erupts.

The volcanic rise of SciML and PINNs https://hadrien-montanelli.github.io/2025-05-31.html 2025-05-31T00:00:00+00:00

Welcome to the fifth entry in my Holiday in Sicily series. As I sip a strong espresso under the Etna sun, I can’t help but feel it: the eruption is getting closer. Not just from Etna—though it’s rumbling again—but from the growing volcano in computational science. I’m talking about Scientific Machine Learning (SciML) and its most explosive crater: Physics-Informed Neural Networks (PINNs). What are SciML and PINNs? At its core, SciML is the blend of traditional scientific computing (think numerical linear algebra, differential equations, optimization) with machine learning techniques. The goal? Learn from data and physics. It’s a natural step forward for computational scientists in the era of data-driven models. Among its many algorithms, PINNs are arguably the flashiest. Introduced by Raissi et al. in 2019, PINNs solve PDEs by turning them into loss functions for neural networks. Instead of assembling a stiffness matrix or computing fluxes across volumes, you just minimize a loss that penalizes deviation from the PDE—using automatic differentiation and PyTorch or TensorFlow. Explosive popularity How hot is it? PINNs are scorching. They’ve already surpassed the Fast Multipole Method (FMM) in citations and are catching up to the Fast Fourier Transform (FFT)—two of the most celebrated algorithms of the 20th century, and personal favorites of mine: Cooley and Tukey’s FFT (1965): ~21,000 citations. Raissi et al.’s PINNs (2019): ~16,000 citations. Greengard and Rokhlin’s FMM (1987): ~7,000 citations. But it’s not just citations. On LinkedIn, debates over PINNs are as passionate as they are polarizing. You’ll find postdocs promoting “PINN 2.0” architectures, industry labs claiming real-world applications, and grumpy seniors complaining about their instability. Why are they so popular? Let’s try to be rational. Here’s my take: Generic over specialized.Classical methods—FEM, FVM, FD—are tailored to specific PDEs and geometries, and they require careful discretization each time. PINNs promise a one-size-fits-all approach. Just rewrite the PDE loss—Laplace, Helmholtz, Navier–Stokes—same codebase. Simple math over advanced math.Classical methods rely on advanced tools from PDEs, functional analysis, and numerical approximation. PINNs can be taught after a basic calculus and Python course. (It doesn’t help that FEM—especially in France, under the long shadow of Bourbaki—is still taught abstractly, starting with distributions and Sobolev spaces, with little focus on implementation.) Accessible tooling.With PyTorch and TensorFlow, it’s easier than ever to experiment. Funding and fame.Let’s be honest—it’s the first time numerical analysts get this much attention. The buzz brings grants, citations, and job offers. In a world where success is often measured by adoption and citations, PINNs seem to be a win. But that brings us to a deeper question: what is science for? Is it about writing technically flawless, Bourbaki-style papers that are hard to read and understand—and that few people actually do? Or is it for spreading ideas, teaching, and energizing the community? PINNs undeniably brought people in. That matters. But… they struggle The reality is more complex. Despite the buzz, PINNs often fail to deliver on benchmarks. A recent methodical benchmarking study highlighted serious issues: weak baselines, inconsistent reporting, and disappointing performance compared to traditional solvers. Or consider this honest blog post by a grad student in astrophysics who fell for the hype—then ran into a wall. Following that post, things got especially heated on LinkedIn, with people openly clashing. And yet, the papers keep piling up—like villagers reinforcing old walls, hoping they’ll hold when the eruption comes. One patch after another on an architecture many suspect is fundamentally unstable. PINNs are still great for teaching Let’s not throw everything away. PINNs shine in the classroom. We still teach FD methods for space discretization—even though few people actually use them in practice. Why not teach PINNs? They offer: A clear link between applied math, physics, and ML. A way to get students fluent in Python and PyTorch, and in thinking about PDEs and differential operators. Motivation for studying consistency, stability, and convergence—albeit in new contexts. My personal journey with SciML I came to SciML not via PDEs but through approximation theory. It started with approximations in Korobov spaces, inspired by Yarotsky’s work in Sobolev spaces. Later, I explored the Kolmogorov–Arnold superposition theorem. These works felt theoretical at the time—but they later inspired practical architectures like KAN, which like PINNs, exploded in popularity almost overnight and generated heated debates. (This is an example where theory precedes practice, and talk to each other.) I missed the practical trick that made implementation efficient: stacking multiple layers and iterating. As a numerical analyst, I should’ve guessed—iteration is the heart of so many methods. Then, this year, I launched a SciML course at École Polytechnique. It was a rewarding experience. Today, I’m working with a brilliant PhD student, Victor, on regularizing the Linear Sampling Method using neural operators, a class of SciML algorithms designed to learn mappings between function spaces. It works surprisingly well. Victor’s writing up his first paper now. Rational optimism, not dogma In science, we must remain rational, not emotional. Some people are irrationally for PINNs—hyped beyond evidence. Others are irrationally against—rejecting anything AI-flavored on principle. At work, I often hear: “AI consumes too much electricity! ChatGPT is catastrophic!” But let’s put things in context. Say a single ChatGPT query gives you the answer you need. It consumes about 1.5 Wh on the server side. (This is a very conservative estimate—GPT-4o appears to be significantly more efficient.) If it takes 10 seconds to read and process on your laptop (drawing 50 W), that adds around 0.14 Wh. Total: ~1.64 Wh. Now compare that with Google. Each search uses about 0.3 Wh server-side. Suppose you need 3 queries and spend 3 minutes browsing—totaling 0.9 Wh server-side, plus 2.5 Wh from your laptop. Total: ~3.4 Wh. Used efficiently, ChatGPT can be up to twice as energy-efficient for certain tasks. The real issue isn’t the tool—it’s the usage. If you fire off 10 prompts a minute, yes, it adds up. But thoughtful use can save both time and energy. More broadly, let’s consider history. The printing press democratized access to information. The Industrial Revolution increased productivity through mechanization. The telegraph and telephone collapsed distances and accelerated communication. The Internet transformed how we share and retrieve knowledge. Now, AI is speeding up technical workflows—from writing code and analyzing data to formulating and testing scientific models. It’s not an anomaly; it’s a natural continuation. Yes, AI must be efficient and responsible—but we shouldn’t dismiss progress just because it’s imperfect. Going back to SciML, I do believe that neural operators can make a real impact. That said, too many papers and talks still avoid fair comparisons: they often benchmark against single-instance solvers like FEM, rather than their true competitor—Model Order Reduction. Watching the volcanos So, here I am in Sicily, watching both Etna and SciML rumble. Both are beautiful, unstable, and sometimes dangerous. Both inspire awe and fear. And both may shape the future—even if we don’t know how. Let’s stay curious, critical, and open. Not irrationally for, not irrationally against. Just rationally engaged.

Touching the volcano—French meritocracy is burning out https://hadrien-montanelli.github.io/2025-06-02.html 2025-06-02T00:00:00+00:00

Welcome to the sixth and penultimate entry in my Holiday in Sicily series. Coincidentally, as I write this, Etna is erupting, just in time for today’s post, where I finally turn to the main volcano: the French academic and meritocratic system. The one I approached two posts ago, and which is now showing signs of violent activity. The magma chamber is fracturing. The smoke is rising. The eruption has begun. The views expressed in this post are entirely my own and do not represent the official position of any institution I am affiliated with. This is a personal reflection based on my individual experience across different academic systems. It is written in a deliberately informal and critical tone, with the intent to spark discussion, not to discredit or disrespect any person or institution. Meritocracy in theory French academia, industry, and the state itself have long been built and steered by engineers. Not just any engineers, but those trained in the elite grandes écoles including Polytechnique, ENS, Mines, Ponts, ENSTA, and ISAE. These schools are more than institutions—they are pipelines into power. École Polytechnique alone has produced an extraordinary roster of alumni: three Nobel laureates, legendary mathematicians such as Cauchy, Cholesky, Jordan, Liouville, Poincaré, and Poisson, as well as heads of state like Borne, Giscard d'Estaing, Lebrun, and Sadi Carnot. And that’s just the beginning—many more appear on this long Wikipedia list. The gateway to this ruling class? National competitive exams. Thousands of students spend two grueling years in classes préparatoires (or prépas, for short) for a shot at the top. Those who pass enter the grandes écoles; those who don’t often fall off the radar entirely. It’s a system built on selection, stress, and prestige. On paper, it’s brilliant. Two intense years, then the concours, a written exam followed by oral exams if you pass the first stage. The best succeed, regardless of wealth, or so we’re told. From there, the Republic rewards its young elite. The schools are free for low-income students and very affordable for others, typically around €5,000 per year, and completely free at institutions like Polytechnique and ENS. Careers are effectively shaped by these degrees. The school you attend often determines your entire professional trajectory, including access to specific career paths and salary scales tied directly to your alma mater. I know—I walked that path. I’m from a (lower) middle-class family. I received state aid to cover tuition fees. The Republic lifted a part of the burden. For that, I am grateful. But gratitude is not silence. The volcano looks majestic from afar. The slopes are clean, the summit sunlit, and the climb subsidized. But as you ascend, the air thins. The paths narrow. The terrain reveals its traps. Meritocracy in practice Let’s be honest: diversity isn’t Polytechnique’s strong suit. According to a 2018 study, 50% of successful candidates in 2013 (MP and PC tracks) come from just two preparatory schools: Louis-le-Grand (Paris, public) and Sainte-Geneviève (Versailles, private Catholic). These elite prépas can afford to run targeted preparation programs for the Polytechnique entrance exams. In contrast, other schools struggle to even cover the syllabus. The result? Students from Louis-le-Grand and Sainte-Geneviève are five times more likely to make it to the oral exams than those from “intermediate” prépas. The numbers on diversity are bleak. In 2024, only 16.7% of admitted students were female, and that figure is declining (source). In terms of socioeconomic background, that same 2018 study, commented on in Libération and Le Figaro Étudiant, showed 1.1% of students came from working-class backgrounds compared to 29.2% nationally. And the system reinforces itself. While written exams are held in regional centers, oral exams require students to travel to Paris at their own expense. Travel costs and logistics become invisible barriers. Professors from the most prestigious prépas have often been involved in designing the exams—or still are. That’s not a leak. That’s structural capture. Past written exams are made available online, but past oral questions are tightly guarded and circulated internally within top prépas. The problem isn’t just Polytechnique: a broader study shows that most grandes écoles in France remain bastions of social reproduction. Polytechnique French population Women (2024) 16.7% ≈50% Working class (2018) 1.1% 29.2% Look abroad While France takes pride in its egalitarian ideals, the numbers suggest otherwise. When it comes to diversity in its elite institutions, we’re not doing better than the UK or US models we so often criticize. Oxford and Columbia—despite operating within systems often portrayed in France as elitist and commodified—show stronger inclusion on key metrics. At Oxford, recent admissions to mathematics show nearly 28% female students, rising to 44% for Math & Stats (source). In terms of social background, 21.2% of Oxford students across all courses came from areas of social and economic disadvantage and from areas of low progression to higher education, a strong indicator of outreach and inclusion beyond the elite schools pipeline (source). Columbia’s data shows that 39% of math and stats graduates in 2022 were women (source), and while access by income remains a challenge, 5.1% of students came from the bottom 20% of family income—five times higher than Polytechnique’s proportion of working-class students (source). Oxford (Overall) Oxford (Math) Oxford (Math & Stats) UK Population Women (2023) 52.2% 27.9% 44.0% ≈50% Disadvantaged backgrounds (2023) 21.2% [N/A] [N/A] [N/A] Columbia (Math & Stats) US Population Women (2022) 39.0% ≈50% Low income (2013) 5.1% 20.0% The lava is moving fast elsewhere. We’re still admiring the symmetry of our volcano. Inside the crater What is life like inside a grande école? At first glance, it’s an impressive ecosystem. The students are brilliant. I’ve taught at Oxford and Columbia, and while those institutions attract extraordinary minds from across the world, I’d argue that, on average, a Polytechnique student is stronger technically. France’s top engineering schools produce sharp, driven individuals with an admirable ambition. But that’s only half the story. The academic schedule is intense. More than 30 hours of classes per week with lectures followed by two-hour tutorials. It’s immediate, heavy, and gives students little time to digest what they’ve learned. The format is rigorous, by design. As a consequence, attendance is often poor. A turnout of two-thirds is considered good. Some students treat their allowed absences as paid leave. Once, a student told me “Since I haven’t missed anything yet, I’ll skip next week’s tutorial and enjoy a long weekend.” At Polytechnique, where I currently teach, and at ENSTA, where I taught previously, this attitude is not uncommon. The entitlement can be striking. And when policies change—when, for example, the administration at ENSTA decided to tighten rules around absences—students went on strike. They sat in protest outside the library. It was all very French. And very privileged. It’s easy to forget just how fortunate we are when it comes to education. At the start of each term, once I’ve had a chance to get to know the group, I sometimes mention how much tuition costs at a university like Columbia. Students are usually surprised—they simply haven’t had much reason to think about it. The education they receive here is world-class, funded by the state, and often taken for granted—not out of neglect, but because it’s just part of the landscape. Still, every now and then, it’s worth stepping back and appreciating what that means. And it’s expensive. Very. The lectures are usually given by a single, senior professor, and then students are split into small groups of about 20 for tutorials—each led by another academic, often less senior. It’s a fantastic pedagogical structure, but also a costly one: one lecturer, a dozen tutors, plus funding for sports facilities, student clubs, and countless associations. I’m not criticizing, I benefited from this system myself, but it’s a real investment of public money. At Polytechnique, for example, a single engineering student can cost the state up to €40,000 per year (source). That’s about four times the national average for higher education students (source). This figure does not include the salary students receive. Yes, unlike most students in the world, Polytechnique students not only study for free: they are paid a monthly salary. It’s one of the most generous public education systems on Earth. That generosity comes with responsibility: in a time of tight public budgets, how do we make sure this investment continues to make sense—not just for the students, but for society? Rebuilding the landscape The volcano is no longer dormant. Its foundations are cracking. Our model of meritocracy is eroding from within. It feeds on the myth of neutrality while entrenching inequality. It’s time to redraw the map. Here are proposals—not reforms, but tectonic shifts: Create regional centers of excellence; starting in high school or earlier. Free tuition for low-income students at these centers; others contribute. Hold both written and oral exams locally, at these centers. Introduce quotas for women and students from modest backgrounds; at these centers, not in the exams. Make these centers attractive for top professors, with competitive salaries and benefits. Abolish private (and especially religious) prépas; public means public. Publish a national database of all past written and oral questions. Dormant no more From a distance, the volcano was beautiful—perfect symmetry, noble ascent. But I’ve stood on its rim, and what I saw wasn’t glory. Smoke can be spun. Exams can be dressed in neutrality. But lava doesn’t lie, it flows where inequality has cracked the ground. The eruption has begun. And in forty-eight hours, the mountain will speak in flames.

The French research Mafia https://hadrien-montanelli.github.io/2025-06-04.html 2025-06-04T00:00:00+00:00

Welcome to the seventh and last entry in my Holiday in Sicily series. Here we are. The full eruption. It’s no coincidence I’m writing this from Sicily, flying back over Etna’s crater. I call it Mafia, and I’m a quarter Italian, so I get a pass. But this isn’t a vendetta; it’s a reflection. And yes, I love my lab, I love my job, and I know how incredibly privileged I am. Tenure with no grant pressure, no publishing quotas. Most researchers land a permanent job after one or two postdocs. Then you’re a civil servant—and short of criminal activity, you can never be fired. Sounds amazing, right? The views expressed in this post are entirely my own and do not represent the official position of any institution I am affiliated with. This is a personal reflection based on my individual experience across different academic systems. It is written in a deliberately informal and critical tone, with the intent to spark discussion, not to discredit or disrespect any person or institution. The Mafia doesn’t like outsiders People warned me: the French academic system is incredibly closed. I didn’t fully understand what they meant until I saw it. In many labs, people come from the same advisor or their academic offspring. Me? I’ve been called “the American,” even though I’m French and did my undergrad in France. I just spent a few years abroad: four in the UK, three in the US. Someone once told me not to speak English in the coffee room, “We won’t hire internationals otherwise.” Not a joke. More broadly, coming back to France was a cultural shock. I missed the COVID-era chloroquine debates. I wasn’t prepared for the hijab discourse. Bias here is subtle but pervasive, and you don’t realize how saturated the environment is until you’ve seen the outside world. Most people haven’t, I can’t blame them. It wasn’t easy to infiltrate the Mafia. But I finally got in two years ago. Today, I’m a permanent researcher at Inria, i.e., a civil servant for life, and I also hold a teaching position at École Polytechnique as a Professeur Chargé de Cours. I landed my dream jobs, and I’m genuinely grateful to those who supported me along the way. Now, let’s dive into the different types of positions and how the Mafia protects its own. Ranks and roles in the family The French academic Mafia operates through two main channels: CNRS and Inria. These are the elite research-only positions. There's also the Maître de Conférences (MCF), the backbone of university teaching. A few new positions exist, such as CPJ, Chaires de Professeur Junior, but I won’t go into those here. CNRS & Inria These are the top research roles in France: CNRS: You apply to a department within a university or engineering school affiliated with CNRS. The position is officially research-only, but you're allowed to teach and get paid extra for it. In practice, especially when you're early in your career, you often teach to support the lab. Inria: You apply directly to a project team (typically 5-10 people). These teams may be hosted by a university, engineering school, or by Inria itself in one of its research centers. Again, it's technically research-only, but teaching is common and appreciated. Salary is decent. If you add a bit of teaching, it becomes competitive, usually better than UK starting salaries. And the freedom is real: no mandatory teaching, full research focus, and appreciation (plus cash) if you do help out. MCF (Maître de Conférences) This is the standard university role: 192 hours of teaching per year, plus research. The pay is low, the teaching load heavy, and the conditions aren’t great, often at underfunded universities, with little support and students who struggle. It’s an unappealing option compared to the research-only tracks. Joining the family Let me now explain how you get made in the French research Mafia. The CNRS process is mostly the same as Inria’s, with a few small differences. Inria Hiring Process Step 1: Dossier. About 20 pages. Inria provides a well-structured template (I’m happy to share mine). At Saclay, for example, each year: 100+ applicants, 15 shortlisted and interviewed, three hired until 2023 (two now). Team support is crucial. You need the backing of the project team. They'll be consulted and asked to write an evaluation. They may also help with your dossier. Without their support, your application is dead on arrival. If you’re a total outsider, you have almost no chance. Step 2: Oral. A 15-minute presentation plus questions. You present to a panel of about ten people. One of them knows your field; the others don’t. Your job is to win over that one expert as they’ll make the case for you behind closed doors. Step 3: The Re-Ranking. After the orals, a smaller committee of three people (only one of whom saw your talk) can re-rank the candidates based on internal criteria. Let me illustrate Step 3 with actual data. What’s nice is that, for all these civil-servant jobs, the final rankings are publicly available online. The table below shows the number of visible re-rankings, i.e., candidates who changed positions between oral and final results. This does not include people who disappeared entirely from the list (likely because they accepted other offers). Inria Center 2024 2023 2022 2021 Saclay [N/A] 1 3 0 Paris [N/A] 1 1 1 Bordeaux 1 1 1 1 Nice [N/A] 1 0 1 MCF Hiring Process It’s closer to the UK/US model but with very French twists: Application: A dossier and a single, short oral interview. No seminar, no visit day, no real chance to meet the department. Letters of recommendation: Technically illegal under French law. But you’re still expected to send them unofficially to the jury president. The problem? Their name is often missing from the job ad. So you have to guess. Email lab members. Piece clues together. Classic French coutume. Once, a colleague told me the custom was actually to wait: the jury would ask for letters after shortlisting. Supposedly, this avoids students with no chance from bothering senior people for letters. Reasonable. So I waited. But then I got an interview and was grilled by a member for not having any letters: “Why does no one want to recommend you?” Che cazzo? Ranking system: On a website straight out of the 90s. Candidates rank jobs, jobs rank candidates. The top match gets the offer. If they decline, it goes to the next one, and so on. But labs fear the job will fall to someone who’s already accepted elsewhere, so they rank only those who seem fully committed. If you’re too strong (say, shortlisted at Inria or CNRS, info that’s public), they might not rank you at all. Just to avoid “wasting” the offer. And yet, another custom: Inria juries often check how you fared with MCF jobs. So you still need to apply to ten or more, just to show you’re competitive. Make it make sense. Bonus anecdote: In Toulouse, I got scolded because my slides were in English. Truly a vibe. Life inside the family Once you’re inside the system, it’s easy to understand why many people never leave. The work-life balance is extraordinary: long lunches, generous vacation time, and flexible work arrangements. That’s a gift, but it can also come with a hidden cost: insulation. In many labs, productivity norms are self-referential. A strong PhD here might publish one or two papers. In the US or UK, that would often be the baseline. This isn’t a question of metrics for their own sake. It’s about the ambition to push boundaries and the cultural energy that supports that ambition. I had the luck of being shaped by environments where people combined rigor, drive, and joy in their work. That shaped my expectations, probably forever. So yes, I sometimes find it hard to adjust to the local tempo. But I’m learning that excellence can wear many masks: with publications, quality can matter more than quantity. French admin as the Mafia’s iron hand I couldn’t not mention French administration. As civil servants, we deal with it constantly and its rigidity is part of daily life. I used to think all bureaucracy was terrible. Then I left France and realized: French admin is uniquely dysfunctional. Take letters of recommendation for MCF. When I called HR because I didn't know what the rule was, the person on the line said, with a condescending tone, “By law, you can’t submit letters of recommendation. Period. I can’t help you.” It was astonishingly unhelpful—and frankly, dismissive. Not to mention that reimbursements for conferences or payments for extra teaching hours can take anywhere from three to five months and that very same HR can't be bothered to speed it up. Hiring an intern is a full-blown bureaucratic saga. One of the administrative assistants told me, almost apologetically: “You need to submit the hiring request at least two months in advance.” Not because of her, of course, she’s just following the rules, but because the process requires seven different people to sign off, and each signature takes about a week. Two full months to rehire someone we already knew: a brilliant student from our Master’s program and a former intern of mine. The delays were especially stressful for him, as he's on the autism spectrum. Thankfully, one extraordinary administrative assistant helped us navigate around the system and hire him through a different channel. She’s incredible, and the thought of her retiring soon honestly terrifies me. She’s spent her career quietly fighting her own version of the Mafia—namely, the patriarchy. She reminds me of my mother: a strong woman who raised five kids on her own. (Funnily enough, as I write this, another administrative assistant just emailed me to say the books I ordered back in November finally arrived—a month ago. But they still haven’t been stamped or catalogued. Classic.) French admin isn’t much better outside of work. My carte vitale (national health card) took four months. A passport photo was rejected while I was applying in person because the date printed on the sheet was from 13 months ago, just over the official 12-month limit. I cut the date off with the clerk’s scissors—“Voilà!”—but she simply replied, “They’ll know.” I asked how. She said the government can track photobooth systems. Honestly, go touch grass. The real issue isn’t the rules themselves, it’s the robotic, blind enforcement of them. On the road here, no one follows rules: bikes run red lights, cars never stop at zebra crossings, pedestrians cross wherever they want. In admin, it’s the opposite: zero flexibility. After the eruption The ground has stopped trembling, the lava has cooled, and the ash has settled. In the stillness, it’s easier to see the landscape for what it is. Scarred, yes, but fertile too. I don’t blame them. Most people haven’t seen how it works elsewhere, and maybe it’s better that way. But we can do better. The Mafia is strong but not unbeatable. Despite all this, I still believe in the French system. We’re not obsessed with metrics. We don’t treat academia like war. Most of us just want to make good math, good science. Let’s keep that spirit but show up a bit more, stay open to the world, and realize how privileged we are.

Mental health and mentorship https://hadrien-montanelli.github.io/2025-10-21.html 2025-10-21T00:00:00+00:00

Mental well-being in academia is a topic that has gained more visibility in recent years. Youth mental health issues have been on the rise, partly due to COVID, but also because more people are talking about it. PhD students, in particular, face intense pressure—imposter syndrome, long hours, and high expectations can take a serious toll. The views expressed in this post are entirely my own and do not represent the official position of any institution I am affiliated with. This is a personal reflection based on my individual experience across different academic systems. It is written in a deliberately informal and critical tone, with the intent to spark discussion, not to discredit or disrespect any person or institution. My own experience has shown me that the way we manage, mentor, and give feedback can make a huge difference. I’ve seen very different approaches across countries and stages of my career, and it shaped my own philosophy of teaching and supervision. First lessons in positive management My first teaching assistant job was at Oxford, grading homework for a Numerical Linear Algebra course. I remember one particular incident: I marked a student’s answer with a red pen, crossing out the wrong parts and writing “0” next to it. The student complained, and the course instructor very kindly explained that this was not the way to give feedback. No red pens, no crossing out, always highlight what was good and what could be improved. Explain how to do better next time. At first, I was surprised. I came from a French prep school and grande école culture where feedback was harsh and direct—if you weren’t perfect, you were shown in red ink. Positive reinforcement wasn’t common. But the instructor’s approach taught me something crucial: encouragement and constructive feedback make students engage, learn, and grow much faster than criticism alone. Learning it again through mentorship The lesson became even more tangible during the same period, throughout my PhD. I was often nervous and unsure of my work—but my advisor always began by highlighting what was done well and what progress I had made. Only after acknowledging the positives would he point out what could be improved, always with detailed suggestions. His approach was rigorous but deeply supportive, and it helped me grow tremendously. I saw the same pattern at Columbia with my postdoctoral advisor. Experiencing this kind of mentorship firsthand profoundly influenced how I would later mentor my own students, teaching me the value of starting with encouragement before addressing corrections. The French PhD experience Back in France as a permanent researcher, I see that the old culture persists in many labs. Many PhD students I meet, including at a recent workshop in the south, report that their advisors rarely compliment their work, are not encouraging, and are very tough. Some of this is the “old-fashioned” French model: the harder the training, the stronger the student. Sometimes, perhaps, it’s simply the only model the advisor knows—or maybe a tiny bit of revenge for past experiences. On my side, I try to maintain positivity in teaching and mentoring. I give detailed, encouraging feedback and highlight improvements, while still keeping high expectations. I will ask a student to redo a figure ten times if it’s not clear—but I do it constructively. My students seem to respond well; motivation is higher and stress is lower, even when standards remain rigorous. Why positive management matters Encouragement doesn’t mean lowering standards. On the contrary, it allows students to reach higher, because they know their effort is seen and valued. Mental well-being, especially in early-career researchers, is deeply connected to how feedback is delivered. Positive management can reduce anxiety, build confidence, and combat imposter syndrome. Final thoughts Wherever I go, I try to spread this approach. It’s 2025, and the way we mentor and manage students can—and should—change for the better. Encouragement and rigor are not mutually exclusive. Positive management helps students thrive, keeps mental health in check, and ultimately leads to stronger science. Let’s make this a standard, not an exception.

A nonlocal Stokes theorem on the sphere https://hadrien-montanelli.github.io/2026-04-21.html 2026-04-21T00:00:00+00:00

Some good news: our paper “Nonlocal vector calculus on the sphere,” joint with Mikaël Slevinsky and Qiang Du, has just been accepted for publication in M3AS. I’m particularly happy about this one—it’s a project where the mathematics turned out cleaner than expected, and the main idea is quite satisfying. Check it out here! This post is a short attempt to explain what we did, and in particular what I think is the most interesting result: a nonlocal version of Stokes’ theorem on the sphere. Why nonlocal? Classical vector calculus is local. Gradients, divergences, curls—they all depend on derivatives, and therefore on what happens infinitesimally close to a point. This works beautifully when functions are smooth and the geometry is simple. But in many applications—materials science, peridynamics, data analysis—you don’t want locality. You want interactions over a finite distance. Points “talk” to each other across a neighborhood, not just through derivatives. This leads to nonlocal operators: instead of derivatives, you integrate differences over a region. The price you pay is that the clean structure of vector calculus starts to break down. The gain is that you can model much richer phenomena. In flat space, this nonlocal framework is by now relatively well understood, and analogues of classical identities—including Stokes-type theorems—have already been developed. The question we ask here is slightly different: what happens when you try to do the same thing on a curved surface? Doing this on the sphere Extending nonlocal calculus from flat space to a curved surface like the sphere is not just a technical step—it fundamentally changes the problem. On the sphere, everything is constrained to the tangent plane. Vectors at different points live in different spaces, and there is no canonical way of comparing them. Even defining something as simple as a “difference between two points” requires care. This is where most of the difficulty lies. In flat space, geometry is invisible; on the sphere, it is everywhere. What we do in the paper is to define nonlocal analogues of the classical operators—gradient, divergence, curl—using carefully constructed integral kernels that respect the geometry of the sphere. The key idea is that these operators can still be analyzed using scalar and vector spherical harmonics. In fact, one of the main technical results is that they are diagonal in that basis. This makes the analysis much cleaner than one might expect. A nonlocal Stokes theorem The highlight of the paper is the following: we prove a nonlocal Stokes theorem on the sphere. In the classical setting, Stokes’ theorem relates the circulation of a vector field along a boundary to the curl of the field inside the surface. It is one of the cornerstones of vector calculus. Nonlocal versions of this theorem already exist in Euclidean space. The novelty here is to establish such a result on a curved surface. There is no a priori reason why the same structure should survive: curvature introduces additional geometric constraints. What we show is that, with the right definitions, a nonlocal Stokes-type identity still holds on the sphere. The mathematics of the nonlocal Stokes theorem Let $\Sigma \subset S^2$ be a spherical cap with boundary $\partial \Sigma$, and let $\boldsymbol{s}$ denote the unit tangent vector along $\partial \Sigma$. In the classical setting, Stokes’ theorem reads \[ \int_{\Sigma} (\nabla \times \boldsymbol{V}) \cdot \boldsymbol{x}\, d\Omega = \int_{\partial \Sigma} \boldsymbol{V}\cdot \boldsymbol{s}\, d\omega. \] In our paper, the nonlocal operators are built from kernels adapted to the geometry of the sphere. We restrict attention to antisymmetric kernels of the form \[ \boldsymbol{\beta}(\boldsymbol{x},\boldsymbol{y}) = \gamma'(\boldsymbol{x}\cdot \boldsymbol{y})\,(\boldsymbol{y}-\boldsymbol{x}), \] where $\gamma$ is a scalar kernel depending only on the dot product $\boldsymbol{x}\cdot \boldsymbol{y}$. To localize interactions to a finite horizon $\delta>0$, we choose \[ \omega_\delta(\boldsymbol{x},\boldsymbol{y})=\chi_{[0,\delta]}(|\boldsymbol{x}-\boldsymbol{y}|), \qquad \gamma_\delta(\boldsymbol{x}\cdot \boldsymbol{y}) = \gamma(\boldsymbol{x}\cdot \boldsymbol{y})\,\omega_\delta(\boldsymbol{x},\boldsymbol{y}), \] and then set \[ \boldsymbol{\beta}_\delta(\boldsymbol{x},\boldsymbol{y}) = \gamma_\delta'(\boldsymbol{x}\cdot \boldsymbol{y})\,(\boldsymbol{y}-\boldsymbol{x}). \] With this notation, the weighted nonlocal curl of a vector field $\boldsymbol{V}$ is \[ C^\delta\{\boldsymbol{V}\}(\boldsymbol{x}) = \int_{S^2} \boldsymbol{\beta}_\delta(\boldsymbol{x},\boldsymbol{y}) \times \bigl[\boldsymbol{V}(\boldsymbol{y})-\boldsymbol{V}(\boldsymbol{x})\bigr]\, d\Omega(\boldsymbol{y}). \] The key result is that this nonlocal curl satisfies a Stokes-type identity on the sphere. More precisely, if we define the averaging operator \[ A^\delta\{\boldsymbol{V}\}(\boldsymbol{x}) = \int_{S^2} \gamma_\delta(\boldsymbol{x}\cdot \boldsymbol{y})\, \boldsymbol{V}(\boldsymbol{y})\, d\Omega(\boldsymbol{y}), \] then one proves that \[ \int_{\Sigma} C^\delta\{\boldsymbol{V}\}\cdot \boldsymbol{x}\,d\Omega = \int_{\partial\Sigma} A^\delta\{\boldsymbol{V}\}\cdot \boldsymbol{s}\,d\omega. \] This result can be viewed as a nonlocal analogue of Stokes’ theorem on the sphere. The structure is very similar to the classical case, but with an important difference: the boundary term involves the averaged field $A^\delta\{\boldsymbol{V}\}$, reflecting the nonlocal nature of the model. The key mechanism behind this result is that the nonlocal operators are diagonal in the basis of vector spherical harmonics. This makes it possible to prove the identity \[ C^\delta\{\boldsymbol{V}\}\cdot \boldsymbol{x} = (\nabla\times A^\delta\{\boldsymbol{V}\})\cdot \boldsymbol{x}, \] and once this is established, the result follows by applying the usual local Stokes theorem to the averaged field. Finally, as $\delta\to 0$, the interaction range shrinks, the averaging disappears, and the nonlocal operators converge to the classical ones. In that limit, we recover the usual Stokes theorem on the sphere. Final thoughts This project was a nice mix of geometry, analysis, and spectral methods. What I find most interesting is that the main difficulty is not the nonlocality itself—this is by now well understood in flat space—but the interaction between nonlocality and curvature. Many classical structures in mathematics are surprisingly robust. But they are not automatic: on a curved space, you have to work to make them reappear. And of course, it’s always nice when a paper finds a home. Thanks to my coauthors for a great collaboration, and to the editors for handling the submission.

Learning only what needs to be learned https://hadrien-montanelli.github.io/2026-05-10.html 2026-05-10T00:00:00+00:00

Our paper “A neural operator framework for solving inverse scattering problems”, joint with Victor Chenu and Houssem Haddar, has been available on arXiv for a couple months. This post is a short attempt to explain what we did, but also why I find this direction interesting. The paper is about inverse acoustic scattering, neural operators, and the linear sampling method. More broadly, it is about a philosophy of scientific machine learning that I care about: machine learning should not replace reliable numerical algorithms when good mathematical methods already exist. It should help them. Two extremes There are two extreme ways of thinking about numerical algorithms in the age of machine learning. The first extreme is to keep using standard methods exactly as before. This is the safe option. The algorithms are interpretable, connected to the theory, and often come with rigorous mathematical justification. In inverse scattering, the linear sampling method belongs to this world. It is a beautiful and robust method, but it can be computationally expensive in practice. The second extreme is to replace the whole inverse problem by a neural network. The input is the measured data, the output is an image of the obstacle, and the intermediate mathematics disappears. Once trained, such a model can be very fast. But the training itself may be expensive, the method may require a lot of data, and it is often difficult to say precisely what has been preserved from the original inverse problem. Our position is in between these two extremes. We do not want to replace the linear sampling method. We want to keep it. But we also do not want to ignore the fact that some parts of the numerical pipeline are expensive and repetitive. The question is therefore: can we use a neural network only for one carefully chosen part of the algorithm? The guiding principle is simple: learn only where learning is useful, and only where imperfection is acceptable. The inverse scattering problem The setting is acoustic scattering. One sends waves toward an unknown object and measures the waves scattered back. The direct problem is: given the obstacle, compute the scattered wave. The inverse problem is the opposite: given measurements of the scattered wave, recover information about the obstacle. This is difficult for two reasons. First, the inverse problem is nonlinear. The relation between the shape of the obstacle and the measurements is not simple. Second, it is severely ill-posed. Small perturbations in the data, such as measurement noise, can lead to large changes in the reconstruction. This is why classical inverse problem methods are valuable. They are not just numerical recipes; they are built around the structure of the equations. They encode mathematical information about the problem. But they also have bottlenecks. In the linear sampling method, one of the main practical issues is regularization. The linear sampling method The linear sampling method is a classical method for reconstructing obstacles from far-field measurements. Very roughly, the method probes the domain point by point. For each sampling point $z$, one solves a regularized linear problem involving the far-field operator. The result is an indicator function: its values tell us whether $z$ is likely to be inside or outside the scatterer. This is attractive because the method does not require a parametrization of the obstacle. One does not need to assume in advance that the scatterer is a disk, an ellipse, or a polygon. The method produces a qualitative reconstruction directly from the data. However, the regularization parameter matters. In the standard approach, one often uses Tikhonov regularization together with Morozov’s discrepancy principle. This is robust and mathematically meaningful, but it requires knowledge of the noise level and can be computationally expensive. This is the point where we insert learning. Learning only the regularization The key idea of the paper is to use a neural operator to help regularize the linear sampling method. This is not the same as training a neural network to solve the full inverse problem. The neural network does not produce the final reconstruction by itself. Instead, it produces information that is used inside the classical method. This distinction is important. Regularization does not need to be perfect. Its role is to guide the inversion process, not to solve the entire problem alone. If the true obstacle is a kite or an ellipse, the learned regularization does not need to be a perfect kite or a perfect ellipse. It only needs to provide useful spatial information about where the obstacle is likely to be. This is why the approach can remain lightweight. We train using simple shapes, in particular circles. Circles are easy to generate, the corresponding scattering data can be computed analytically, and the training set can remain small and high-quality. The network learns a useful regularization pattern, not a complete inverse solver. Then this learned regularization is injected into the linear sampling method. The final reconstruction still comes from the classical algorithm. The neural network helps with one part of the pipeline, but the mathematical backbone remains the linear sampling method. This is perhaps the main message of the paper: We train the network not to solve the inverse problem, but to help regularize a method that already knows how to solve it. The hybrid method More concretely, the input of the learned model is the noisy far-field matrix. From this data, we learn two quantities. First, a neural operator produces a preliminary indicator function. This indicator gives a rough estimate of the location and shape of the obstacle. Again, it does not need to be perfect. It is only used as a guide. Second, another neural network estimates the noise level in the data. This is important because classical regularization strategies often require information about the noise level. The two learned outputs are then combined to define a regularization function for the linear sampling method. In simplified notation, one may think of it as \[ \alpha_\theta[F^\delta](z) = \delta_\theta[F^\delta]\, I_\theta[F^\delta](z), \] where $F^\delta$ is the noisy far-field matrix, $I_\theta$ is the learned indicator, and $\delta_\theta$ is the learned noise estimate. This regularization function $\alpha_\theta$ is then used in the linear sampling method. The final reconstruction is therefore not just the output of a neural network. It is the output of a classical inverse scattering method whose regularization has been informed by learning. A deliberately simple neural operator The neural operator we use is a DeepONet with a fixed radial-basis-function trunk. The branch network reads the far-field matrix and produces coefficients. The trunk gives a fixed spatial basis, made of radial basis functions, in which the indicator function is represented. This design is intentionally simple. We are not trying to build the largest possible architecture. We want a model that is expressive enough to be useful, but structured enough to remain understandable. A nice feature of the construction is that the network is essentially determined by the wavelength. Once the wavelength is fixed, we choose a spatial discretization scale, for instance $h=\lambda/2$ or $h=\lambda/4$. This in turn determines much of the architecture and training setup: the number of radial basis functions in the trunk (and hence the output dimension of the branch network), the branch network's width, the number of training samples, and the quadrature size. This is quite different from the usual trial-and-error approach to neural network design. We are not tuning a large architecture by hand until it works. The geometry and physics of the scattering problem determine the scale at which the network should operate. There remains essentially one important architectural parameter: the width of the radial basis functions in the trunk. This controls how much neighboring basis functions overlap. If the width is too small, the basis is too localized; if it is too large, the representation becomes too smooth. In the paper, we use an NTK analysis to guide this choice. Why the NTK appears The neural tangent kernel, or NTK, is one way of analyzing the training dynamics of neural networks. Very roughly, it describes how the network changes during training in a regime where the dynamics can be approximated by a kernel method. For us, the NTK is not used as an abstract theoretical decoration. It has a practical purpose: it helps understand the effect of the radial-basis-function width in the trunk. This is exactly the kind of analysis I would like to see more often in scientific machine learning. If a neural component is inserted into a numerical method, its design should not be arbitrary. The architecture should be constrained by the mathematical problem, and the remaining parameters should be chosen using analysis whenever possible. Noise estimation The second learned ingredient is the noise level. Instead of feeding the full far-field matrix to the noise-estimation network, we feed it singular values. This is natural for two reasons. First, the singular values contain useful information about the noise level. In ill-posed inverse problems, noise often appears as a plateau in the singular spectrum. Second, they are invariant under translations of the obstacle. This is a very useful feature: the noise-estimation network can be trained only on disks of varying radius centered at the origin, and still generalize to translated obstacles. Again, the point is not to use a neural network blindly. The input representation is chosen because it matches the structure of the inverse problem. What the numerical experiments show The numerical experiments are in two dimensions. The main observation is that the learned regularization can significantly accelerate the linear sampling method workflow while producing accurate reconstructions. After training, the neural networks provide the regularization information quickly, avoiding the repeated parameter search associated with Morozov’s discrepancy principle. For single obstacles, the method gives good reconstructions. For multiple obstacles, the learned indicator alone can be less precise, which is not surprising since the training data are deliberately simple. But once this imperfect indicator is used inside the linear sampling method, the final reconstruction improves substantially. This is an important point. The neural network output by itself is not the final product. It is allowed to be imperfect because the classical method corrects and refines it. That is the advantage of the hybrid approach. Interactive scientific machine learning There is also a practical aspect that matters a lot to me. I am not especially interested in methods that require enormous datasets, huge networks, and specialized hardware just to reproduce a result. Large models have their place, but in computational mathematics there is great value in methods that are small enough to understand, fast enough to experiment with, and simple enough to modify. This is close in spirit to Nick Trefethen’s idea of “ten digit algorithms”: programs that aim for “ten digits, five seconds, and just one page.” The exact numbers are not the point. The point is the attitude: a good computational idea should be accurate, fast enough to explore interactively, and simple enough to be read, modified, and understood. I like to think of this as interactive scientific machine learning: models that are scientifically useful, fast enough for real-time experimentation, simple enough to be understood and modified, and small enough to remain at least potentially amenable to mathematical analysis. This matters scientifically, not just practically. If a learned component is modest in size and has a clear role inside a numerical pipeline, one has a better chance of understanding what it represents, how it trains, how errors propagate, and how it interacts with the rest of the algorithm. I think scientific machine learning should learn something from this. A good hybrid method should not require a gigantic model and a massive dataset whenever the underlying numerical problem already contains structure. Ideally, a student should be able to run experiments on a laptop, change a parameter, see what happens, and develop intuition. Interactivity is not only a computational convenience. It is part of the scientific method. Final thoughts What I like about this project is that it sits in a middle ground. It is not classical inverse problems as usual, because learning genuinely enters the algorithm. But it is not black-box AI either, because the learned components are embedded into a mathematical method with a clear purpose. The neural network is not asked to solve the whole problem. It is asked to do something more modest, and more useful: provide a regularization function for the linear sampling method. Thanks to Victor and Houssem for a very enjoyable collaboration. I am especially happy that this project connects naturally with the broader research direction I hope to develop in the coming years: making hybrid scientific machine learning more predictable, more interactive, and more useful for computational science.

Hybrid SciML, in hybrid mode https://hadrien-montanelli.github.io/2026-05-13.html 2026-05-13T00:00:00+00:00

This year was the second edition of my SciML course at École Polytechnique, and the first time we taught it in a hybrid EuroTeQ format. The course notes are available on HAL. The course itself is about hybrid SciML: methods that combine classical numerical models with data-driven components. This year, the teaching format was hybrid as well: students followed the course both in the classroom and remotely through EuroTeQ. The course was organized with Victorita Dolean, Rémy Hosseinkhan, and Loïc Gouarin. The audience was a mixture of third-year Polytechnique students, EuroTeQ Master students, and IP Paris Master and PhD students. The format was simple on paper: one two-hour lecture and one two-hour tutorial every week, from January to March. The reality, of course, was slightly more complicated. This post is a short reflection on what worked, what did not, and what I learned from teaching SciML in a hybrid setting. Why hybrid? SciML is a natural topic for this kind of experiment. It sits at the intersection of numerical analysis, scientific computing, and machine learning. No single institution has all the expertise, all the students, and all the viewpoints. A collaborative course makes sense because the field itself is collaborative by nature. In that sense, EuroTeQ worked well. The course gave students access to expertise and teaching styles they might not have encountered locally. It also created mobility opportunities: for example, a couple of students went to TU Eindhoven afterwards. This is exactly the kind of concrete outcome one hopes for in a collaborative European teaching format. The course itself The course was designed as an introduction to SciML for students with a background in applied mathematics. The goal was not to teach machine learning in general. The goal was to explain how learning-based methods can be used for scientific computing: solving PDEs, learning operators, accelerating numerical methods, and thinking about the interaction between data-driven models and mathematical structure. As usual, I tried to keep the course close to computation. The lectures introduced the ideas, but the tutorials were essential. In SciML, one only really understands what is going on when one starts training models, changing parameters, plotting errors, and seeing what works and what fails. This is especially true because SciML can easily become too abstract. Terms like physics-informed neural networks, or operator learning sound impressive, but students need to see what these methods actually do on concrete examples. For this reason, the tutorials were not secondary. They were the place where the course became real. What worked The overall feedback was very positive: 98% of students gave positive feedback, which was of course encouraging. For students, the main benefit was exposure. They were exposed to different scientific cultures, different styles of teaching, different expectations, and complementary expertise. This is particularly valuable in an emerging field like SciML, where the boundaries are still moving and where different communities approach the same objects in different ways. For instructors, the benefits were also real. A collaborative course gives access to a broader student pool and creates interactions across institutions. In our case, the collaboration with Victorita Dolean also fed directly into a PhD project. She suggested including an Neural Tangent Kernel analysis in one chapter, and this later became part of a PhD paper, improving the paper itself. This was a nice example of teaching and research feeding into each other. This is something I like about teaching advanced courses. If the topic is alive enough, teaching is not just dissemination. It becomes a way of testing ideas, finding students, starting collaborations, and clarifying what one really thinks. What was difficult The difficult part was not the mathematics. The difficult part was the system around the course. A hybrid international course is a multi-actor system. It requires synchronization between instructors, administration, digital platforms, classrooms, and students across institutions. Everyone has to understand the framework, the constraints, and the deadlines. In practice, this is much harder than it sounds. The first issue was scheduling. Academic calendars do not align perfectly. Terms start and end at different times. A two-hour lecture plus two-hour tutorial format may be natural at Polytechnique, but not elsewhere. These differences look minor on paper, but they affect everything: registration, attendance, project deadlines, exams, and communication. The second issue was digital access. We used Zoom, Moodle, and the IP Paris Media Player. Access for external students was not immediate. Zoom took days, Moodle took weeks, and some Media Player access issues were not fully resolved. The third issue was the classroom itself. A good hybrid classroom is not just a room with a camera. The camera has to point to the right place. The microphone has to capture questions. The instructor needs a second screen to see remote students. The setup has to be reliable enough that the teacher is not thinking about it while teaching. In our case, the camera had to be adjusted each time by a technician. This is the kind of small technical detail that becomes large when repeated every week. Passive hybrid does not work The main pedagogical lesson is simple: passive hybrid does not work. By passive hybrid, I mean a format where one teaches almost as usual in the classroom, while remote students quietly follow the stream. This is easy to set up, but it is not enough. Remote engagement was hard. Remote students asked fewer questions, gave less feedback, and were harder to involve during tutorials. Interaction between Polytechnique and EuroTeQ students also remained limited. This was one of the main frustrations of the format. There was also a clear attendance issue. Once lecture videos were uploaded, attendance dropped. This is not surprising, and I do not blame students for it. If a course is available as a recording, and if no specific interaction is expected during the live session, then the rational decision for many students is to watch it later. But watching later is not the same as learning together. This is especially true for tutorials. A tutorial is not a video. It is a place where students try things, get stuck, ask questions, compare approaches, and slowly build intuition. If the format becomes purely passive, much of this disappears. So one lesson for next time is clear: engagement has to be designed. Assessment in the age of remote teaching and LLMs Assessment was another challenge. In a hybrid international format, in-class exams are impossible. Students are not all in the same room, under the same conditions, at the same time. This pushes the evaluation toward projects, reports, oral presentations, or take-home assignments. I actually think this is quite natural for SciML. A project is often a better assessment than a written exam. It shows whether students can implement an idea, test it, interpret results, and explain what happened. But projects also raise another issue: LLMs. In practice, remote monitoring is impossible, and LLM usage is effectively fully allowed. Again, I do not think this should be treated only as a problem. If students will use these tools anyway, then the assessment should be designed accordingly. There is also a real advantage to LLMs in this context. They allow students to attempt much more ambitious projects than would otherwise be realistic in a short course. In our case, the final project for all groups went beyond a standard implementation exercise: students had to read a research paper, reproduce part of it, test the method, and explain the results. In a traditional setting, this would often be considered a task for PhD students rather than master’s students. This does not mean that the LLM did the work for them. Rather, it changed the level at which the work took place. Students could use the tool to get past technical barriers faster, but they still had to understand the paper, make decisions, debug experiments, interpret outcomes, and present the ideas clearly. The question becomes: what do we want to evaluate? Not whether a student can produce boilerplate code. Not whether they can write a generic introduction to neural networks. But whether they understand the method, can make choices, can explain results, can identify failure modes, and can defend their work orally. In other words, assessment has to move upward. Less emphasis on producing text or code from scratch, more emphasis on understanding, judgment, and interpretation. Level mismatch Another predictable difficulty was the heterogeneity of backgrounds. Some students were comfortable with PDEs but less familiar with machine learning. Others knew neural networks but had less experience with numerical analysis. Some were strong programmers; others were not. In a local course, one can sometimes assume a shared curriculum. In a European hybrid course, this is much less true. This is not necessarily bad. In fact, it reflects the nature of SciML itself. The field requires people to cross boundaries. But it also means that one should not assume too much implicit common knowledge. In my case, I did not really write down formal prerequisites for the course. I mostly relied on what I thought would be shared background knowledge, and I should probably have made those expectations more explicit. One has to decide what is truly essential, provide bridges, and accept that not everyone will enter the course from the same direction. What I would change Next year, I would change a few things. First, I would coordinate digital access much earlier. External students should have access to Zoom, Moodle, recordings, and course material before the first lecture. This sounds obvious, but in a multi-institutional setting it requires much more anticipation than expected. Second, I would give remote students a clearer role during the live sessions. For example, they could work in small tutorial groups, answer short questions during the lecture, or give brief updates on their projects. The goal would not be to police attendance, but to make the live course feel less passive and more useful. Third, I would make the expected background more explicit and more modular. I would also look more carefully at the curricula of the different universities involved, so that the course better matches what students have actually studied before. Instead of listing a long set of formal prerequisites, I would provide short preliminary notebooks on Python, automatic differentiation, PDE discretization, and basic neural networks. This would help students identify what they already know and fill in gaps before or during the course. Final thoughts I am glad we taught the course in this format. It was not perfect, and hybrid teaching is clearly not just “put a camera in the room.” It requires design, coordination, and active engagement. It also requires accepting that the administrative and technical layers are part of the course, whether one likes it or not. But the benefits are real. Students get access to a broader scientific environment. Instructors interact with new students and colleagues. Courses in emerging fields can circulate more easily across institutions. And sometimes, concrete collaborations and mobility opportunities follow. For SciML, this feels particularly appropriate. The field itself is hybrid: between mathematics and computation, between models and data, between theory and experiments. Teaching it in a hybrid international format is therefore both natural and challenging. The main lesson I take from this year is simple: hybrid teaching can work, but only if it is not passive. A good hybrid course is not a recorded lecture with a chat window. It is a course where interaction has been deliberately designed. That is the challenge for next year.

Convergence rates of curved boundary elements https://hadrien-montanelli.github.io/2026-08-09.html 2026-08-09T00:00:00+00:00

Welcome to the first episode of my Retreat in Lozère series: a few short posts written during a 10-day retreat in a quiet chalet where I spent many of my childhood holidays. I came here to work on my ERC Starting Grant and go trail running every day. My ERC project will be about hybrid scientific machine learning for forward and inverse scattering. Working on the proposal, I have been going again through some of my recent work on boundary integral equations. Curved boundary elements Boundary element methods are useful when solving the Laplace or Helmholtz equation. Instead of solving the equation in the whole domain, one rewrites the problem as an integral equation on the boundary. This is very useful for scattering problems, where the domain outside the obstacle is unbounded. In practice, the boundary is discretized by a mesh. If the obstacle is smooth, we wish to use curved elements to better approximate the geometry. This raises the following question: how fast does the method converge when the mesh size $h$ goes to zero? Two sources of error There are two main sources of numerical error. First, the unknown density on the boundary is approximated by polynomials of degree $m\geq0$. Second, the geometry is approximated by curved elements of order $\ell\geq1$. The point of the paper is to understand how these two choices interact. If the geometry is too rough, the geometric error dominates. If the polynomial space is too small, the approximation error dominates. One should choose $m$ and $\ell$ in a balanced way. Main result In the paper, we prove improved convergence rates for curved boundary elements for the 3D Laplace/Helmholtz Dirichlet problems. For the single-layer formulation, the pointwise error away from the boundary behaves like $$ |u(x)-u_h(x)| \leq c_x\left(h^{2m+3}+h^{\ell+1}\right). $$ For the double-layer formulation and for the combined-field integral equation, the rate is $$ |u(x)-u_h(x)| \leq c_x\left(h^{2m+2}+h^\ell\right). $$ When using an interpolated normal instead of the element normal, the geometric part improves to $h^{\ell+1}$. Numerical experiments We also tested the rates numerically, using curved triangular elements up to order four. We considered both a sphere and a more complicated bean-shaped object. The experiments confirmed the theoretical rates. They also showed an interesting superconvergence effect: for even values of $\ell$, namely $\ell=2$ and $\ell=4$, the geometric error often behaved like $h^{\ell+2}$. This happened not only for the sphere, but also for the bean-shaped geometry, so it does not seem to be only due to symmetry. Check out our paper!

What will remain? https://hadrien-montanelli.github.io/2026-08-12.html 2026-08-12T00:00:00+00:00

Welcome to the second episode of my Retreat in Lozère series. Today I want to talk about a question that is becoming uncomfortable for us mathematicians: what will remain when AI becomes better than us at doing our job? The views expressed in this post are entirely my own and do not represent the official position of any institution I am affiliated with. What will remain? This may have sounded strange a few years ago, when AI was sloppy and failed at high-school problems. But the line keeps moving—high-school problems, then undergraduate problems, then graduate problems, and now AI is starting to help prove conjectures that humans could not solve for decades. A recent example in my field of numerical analysis is the proof of Crouzeix’s conjecture by a neurosurgery resident a couple of days ago, with help from GPT-5.6 Sol. So what will be left for mathematicians? Us mathematicians, we prove things. But what is a proof? What is a proof? A proof is a sequence of logical steps that starts with assumptions and proceeds to a conclusion. We mathematicians like to think of mathematics as the noblest of sciences—once the assumptions have been established, everything follows from logic. Things are either right or wrong. What we do is pure science. Or is it? A proof is also a social compact. I discussed this in a previous blog post about mathematical rigor. When we write a proof, we do not define everything. Instead, we give enough details so that another mathematician can follow the argument and fill in standard gaps. Without this compact, every proof would have to start from the foundations of mathematics. Nobody wants that. This social compact also means that many proofs are not checked line by line. If the author and/or the institution is trusted, and if the paper looks correct at first sight, many people will believe it, including reviewers! Of course, this is different for a major theorem. A very important result will be checked carefully by experts. But most papers are incremental and reviewers will not check every line. Let's be a little bit provocative. Sometimes I feel that academic mathematics works like an old gentlemen’s club. (And entering this club is hard; see my previous post on The French Research Mafia.) But once you have a seat at the table, people trust you almost blindly. Your papers look serious because your name is on them. You review your peers, your peers review you. In the end, everyone protects the same system. And I think AI is about to change this. Most proofs contain mistakes I would bet $100 that if you pick a math paper at random and run a strong model on it, it will find at least one mistake. Try me! Most mistakes will probably be small and fixable, but sometimes the problem is more serious. The more technical the paper is, the easier it is to make mistakes. Moreover, if the paper is also hard to read, fewer people will check it carefully, then the author gets less help and the mistakes stay hidden. I also think that, in 20 or 30 years, people may be surprised by how we used to do mathematics. They may ask how we could publish so many formal arguments without AI. They may look at us as artisans, much like we look at people who built the pyramids without cranes or any kind of machinery. A small uncomfortable anecdote Recently, I took a very technical draft that I found almost impossible to read. To my surprise, some people around me had the opposite reaction: they said the draft was deep and impressive! However, when I asked more questions, it became clear that almost nobody had really read it carefully—this is the kind of hypocrisy that is everywhere in mathematics. Complicated writing creates the impression of depth and a dense proof can make the writer look smart and serious. It can also make the reader feel that the result must be important. Sometimes the reader also wants to sound smart and serious by saying that they understood it! I'm not even joking, this is real. Anyway, I ran a strong language model on the draft. Ouch. It found many mathematical problems: not only style issues, but also some very serious gaps. The main result may actually be wrong. In the future, it may become much harder to hide weak mathematics behind complicated writing. And I think this is a good thing. My provocative posts are not always appreciated by the community, and this one may hurt a bit. Sorry! Mathematicians as editors Where does this leave us? If AI becomes better at producing and checking proofs, then one thing may become much cheaper: correctness. This feels and sounds strange because correctness has always been the heart of mathematics. But since you can give a proof to an AI and get a detailed report in a few minutes, writing a correct proof may no longer be enough to impress anyone. And I think this will make clear writing even more important! Suppose an AI gives you a 40-page proof of a conjecture. Great. But somebody still has to read those 40 pages and understand what really happened. Where is the main idea? Is there a simple example that explains what is going on? Going back to Crouzeix’s conjecture, only a few days later, a couple of mathematicians produced a five-page proof instead of the original 24-page one. Thanks, humans! I think part of our job may therefore move from producing arguments to digesting them. We may spend more time taking long AI-generated or AI-assisted proofs and turning them into mathematics that humans can understand. In other words, mathematicians may become a little more like editors. I think I actually like this future. Let me say this one more time: clear writing will matter a lot. Mathematics can become more about ideas again, and less about putting complicated symbols everywhere. I have never been a big fan of the French Bourbaki style, or of the way mathematics is sometimes taught here. Bourbaki and I broke up—let's make math fun again! So, what will remain? Of course, there is another possibility: maybe humans will remain useful for coming up with strange questions and completely bold and unexpected ideas and connections. I hope so but I would not bet $100 on that one! By now we realize that every time we draw a line and say, “AI will never do this,” then the line seems to move a few years (if not months!) later. So perhaps the better question is what we still want humans to do. I still want to learn mathematics from another human being. I still want somebody to tell me why a theorem is beautiful. I still want a teacher or a mentor who knows when I am confused. I still want a colleague who gets excited by an idea and spends time finding the clearest way to explain it to me. I still want to think about mathematics at my own pace without looking at a computer screen. Mathematics is also a culture. It is about transmitting rigor, critical thinking, intuition, curiosity, and the joy of discovery. Maybe this is what will remain. “The scientist does not study nature because it is useful to do so. He studies it because he takes pleasure in it, and he takes pleasure in it because it is beautiful.” — Henri Poincaré, Science and Method

Writing a grant in Lozère https://hadrien-montanelli.github.io/2026-08-15.html 2026-08-15T00:00:00+00:00

Welcome to the third and last episode of my Retreat in Lozère series. Today I want to write about the experience of preparing a research grant. The views expressed in this post are entirely my own and do not represent the official position of any institution I am affiliated with. A special place in my heart I am writing this from a chalet in Lozère. This place has a special place in my heart. I spent two weeks here almost every summer from age zero to age 20. I came back less regularly after that, but I still come whenever I can. This chalet is where I prepared some of my math exams during prep school. Between my two years of classes préparatoires, I spent one month here working hard and running on the trails. I know the forests and paths around here very well. Within a radius of 15 kilometers, almost every turn feels familiar. This is also where I read Saint Augustine as a teenager and felt something close to faith for the first time. It was also the last time, but it still mattered to me. Some places keep the memory of who we used to be. The chalet has now passed from my grandparents to my uncle and aunt. I am very grateful that they let me stay here. For ten days, I could work, eat, sleep, run, and think. This is a luxury. Why write a grant? In France in particular, and I'm sure elsewhere but I will not over-generalize, the idea of writing grants is very sensitive. Researchers are very attached to permanent positions and long-term research with no immediate pressure. Who wouldn't? I understand this and I like this model—it gives people time to think and protects research from short-term fashion. However, there is another side: there is less and less money, and funding is getting much harder. Us, young researchers cannot simply pretend this is not happening; we need PhD students and postdocs! We need resources if we want to build something larger than a few papers. I sometimes feel that permanent researchers who reject grants on principle are defending a system that no longer really exists. (Yes, I know, another provocative post. This is my style!) I try and say this with respect, but also with great frustration. They lived in a different system with a very different funding landscape. The discussion is difficult because it often becomes political. On the left, grants are seen as a problem, while on the right, they are seen as a solution. Reality is more complicated. I felt the same thing when I wrote about AI: the discussion immediately became political. Some people did not want to discuss what AI can or cannot do. They only wanted to say that it eats bad text, digests bad text, produces bad text, and then eats its own bad text again. (I'm writing this in a much more polite way than what was actually said!) Other people immediately talked about AI water and electricity consumption. These are very important points but they absolutely do not remove the scientific question. For grants, I feel the same. Yes, we can criticize the system, and we should! But we also have to live in it. A new goal My first reason is that I'm a driven person and I need goals. After I got my permanent position, I sent an email to senior people in my lab. I asked them for advice about what to do next. Some of them basically answered: “You have tenure now: enjoy it! Why are you already thinking about the future?” Well, yes, this is me, hello! Anyway, so I got a permanent position three years ago. Since then, I have completed around 70 or 80% of the research program I had in mind at the time; so, yes, I needed a new goal. Writing a grant is a difficult exercise, but it is also a wonderful one. It forces you to read the current literature. And with AI, scientific machine learning, and numerical analysis moving so fast, this is already a huge task. It also forces you to step back and ask a hard question: what do I really want to do next? Leaving the niche It is easy to stay in a niche! As I wrote in a previous post, this is one way to do research: you take a known problem and generalize it. You write a few papers, you slowly improve the numerical method, you prove sharper convergence theorems, you apply it to harder and harder application problems. This is useful but I do not want to do only that. So another reason is leaving the niche to properly think about where applied mathematics is going. In particular, I want to understand how AI can be used in numerical analysis and scientific computing without losing mathematical rigor. What I really want are mathematical criteria that tell us when these methods work. I will not say much more here, because this is the grant proposal and the core of the project. The pleasure of writing There is also another reason. I like writing. It was passed on to me by my mother, who was a pharmacist but also loved writing and wrote a couple of novels in her free time. It was also passed on to me by my PhD advisor, whose writing skills always impressed and inspired me. So now, I really like putting documents together. I like taking messy ideas and turning them into something clear. And editing a document until it's polished. I do this in my research memos. I do this in my papers. I do this in my lecture notes. I do this on this blog! A grant proposal is another version of the same exercise. It remains a hard process. Like trail running for a few hours with some good D+. Yes, it is hard, but it is the kind of hard I like. A good retreat This retreat has been good for me. I spent my days reading, writing, thinking, running, and eating delicious local food. I was hosted and fed by my family, so I could put almost all my mental energy into the proposal. Again, that is a luxury. When the writing became too much, I ran on the same paths I knew as a child. I recognized the trees and the stones. I felt tired, but I felt safe and at home. I do not know whether this grant will succeed and, frankly, most grants do not. Nevertheless, the experience has already been useful. It forced me to think seriously about the next years of my research. It forced me to ask how I want numerical analysis to look like in this new AI era. For those reasons, I am glad I did it anyway. Finally, I am grateful to this place, once again, for giving me the space to think.