Solvent subtraction

Waves

Method

Solvent denoising

Result

Compare methods

Methods and equations

S(x) is the sample spectrum, R(x) the solvent reference interpolated onto the sample's x axis (natural cubic spline), x the Raman shift. B(x) = b₀ + b₁t + b₂t² + b₃t³ is the optional baseline, a polynomial in t = (x − x_c)/w with x_c and w the centre and half-width of the fit regions. Fits use only the points inside the fit regions and outside the excluded ranges (mask the solute bands, roughly ±25 cm⁻¹ around each). The result is D(x) = S(x) − M(x), with M the fitted solvent contribution; the baseline is subtracted too only if that box is ticked.

1. Least-squares scale factor

   min_{a, b} Σ_{i ∈ fit} [S_i − aR_i − B(x_i)]²,  M = aR

   Linear least squares by QR. The ± values on this page are standard errors from the fit covariance s²(JᵀJ)⁻¹ with s² = RSS/(N − p). Neighbouring pixels are often correlated and the model is never exact, so treat them as lower bounds.

2. Peak area ratio

   a = Σ_regions ∫ [S − ℓ_S] dx  /  Σ_regions ∫ [R − ℓ_R] dx

   ℓ is the straight line through the first and last three points of each region (averaged), so each region should hold one isolated solvent band with a little baseline on both sides. The table also lists the ratio of each region on its own; they should agree.

3. Minimum roughness

   a = argmin Σ [Δ^k(S − aR)]² = Σ Δ^kS Δ^kR / Σ (Δ^kR)²,  Δy_i = y_i+1 − y_i,  k = 1 or 2

   Broad solute bands and backgrounds hardly change the point-to-point differences; the narrow solvent bands dominate them. The scale that leaves the smoothest result removes the solvent, without fit regions free of solute. Works best when the solute bands are broader than the solvent bands.

4. Minimum total variation

   a = argmin Σ |Δ(S − aR)| = weighted median of ΔS_i/ΔR_i with weights |ΔR_i|

   The L1 version of minimum roughness: less sensitive to noise spikes and to strong solute bands. The weighted median is the exact minimum.

5. Taylor expansion (linear)

   ∫ p(s) R(x − s) ds = R − μ₁R′ + (μ₂/2) R″ − (μ₃/6) R‴ + …,  μ_n = ∫ sⁿ p(s) ds
   S ≈ a₀R + a₁R′ + a₂R″ + a₃R‴ + B,  M = a₀R + a₁R′ + a₂R″ + a₃R‴
   δ = μ₁ = −a₁/a₀,  σ² = 2a₂/a₀ − δ²,  κ₃ = −6a₃/a₀ − 3δσ² − δ³,  FWHM = 2√(2 ln 2) σ ≈ 2.355 σ
   R′ ≈ [R(x+h) − R(x−h)]/2h,  R″ ≈ [R(x+2h) − 2R(x) + R(x−2h)]/4h²,  R‴ ≈ [R(x+3h) − 3R(x+h) + 3R(x−h) − R(x−3h)]/8h³  (np.gradient)
   R″ ≈ [R(x+h′) − 2R(x) + R(x−h′)]/h′²,  R‴ ≈ [R(x+2h′) − 2R(x+h′) + 2R(x−h′) − R(x−2h′)]/2h′³,  h′ = h/2  (fine)

   A small shift δ and a symmetric broadening of any shape (variance σ²) of the solvent bands, expanded to second or third order, is linear in the coefficients a_k: one least-squares fit, no starting values. The subtracted terms a₁R′ and a₂R″ are exactly the dispersive, derivative-shaped solvent residues that a shift or width change leaves behind (for example in transient pump-on − pump-off FSRS spectra). σ² < 0 means the solvent bands are narrower in the sample than in the reference, which the Taylor form handles and a convolution cannot. κ₃ measures asymmetry. By default the derivatives are central differences at the reference's point spacing h, applied once, twice and three times (as np.gradient does). The "fine" stencils at h/2 are less biased on narrow bands but pass more of the reference's noise into R″; noise in a fitted column biases its coefficient toward zero (regression dilution), so with a noisy reference a₂ and the width come out low. Use fine with a clean or denoised reference. The expansion holds while δ and σ are small compared with the solvent band widths. "Iterate" removes that limit: it replaces R by the reference shifted by the fitted δ and convolved with a Gaussian of the fitted variance σ², R → [R ⊗ G_σ](x − δ), and refits until the new a₁ and a₂ terms vanish (Gauss–Newton steps). It converges to the exact shift + Gaussian broadening result; a net narrowing stays in the a₂R″ term, since a convolution cannot sharpen. Smoothing the solvent helps when its noise is amplified by R″ and R‴.

6. Shift and broadening (nonlinear, exact)

   M(x) = a [R ⊗ V](x − δ − ε(x − x_c)),  V = Gaussian(FWHM w_G) ⊗ Lorentzian(FWHM w_L)
   ℱ[R ⊗ V](f) = ℱ[R](f) exp(−2π²σ²f² − 2πγ|f|),  σ = w_G/2.355,  γ = w_L/2

   The reference is resampled on a uniform grid at half its point spacing, padded, broadened by FFT and read at the shifted positions by cubic interpolation, so shifts of any size are exact. For each trial δ, ε, w_G, w_L the linear parameters a and B come from least squares (variable projection); the nonlinear ones from a grid search over the shift and the widths followed by Nelder–Mead. A convolution can only broaden: w_G, w_L ≥ 0. Widths add as √(w² + w_G²) for Gaussian and w + w_L for Lorentzian broadening. ε is a stretch of the x axis (a calibration error that grows across the window): the shift is δ + ε(x − x_c), and the table gives it at both ends of the fit regions.

7. Fit on differences

   min Σ [Δ^k(S − M − B)]²,  k = 1 or 2

   An option for the least-squares, Taylor and shift-and-broadening methods. It weights the narrow solvent bands and ignores smooth backgrounds. Baseline terms of degree below k drop out (they have no differences), so the baseline is then only partly determined.

8. Denoising the reference: HSVD

   x(n) = DFT⁻¹[R](n) ≈ Σ_k A_k e^{(i2πν_k − γ_k)n},  H_ij = x(i + j),  i < N/4,  i + j ≤ N/2,  H → U_r S_r V_rᴴ

   In the time domain each Lorentzian band is a damped complex exponential, and K of them make the Hankel matrix exactly rank K, while white noise spreads over every rank. Keeping a hard rank r, averaging the anti-diagonals back into a signal, rebuilding its conjugate-symmetric half and transforming back removes most of the reference's noise. A hard rank keeps weak bands whole; soft weights near the noise edge shrink them. The rank errs high by default: 3 × the number of singular values clearly above the noise edge (> 1.5 τ), and never fewer than 40. A rank too low cuts weak bands (Voigt-shaped bands need several ranks each: with a Gaussian instrument width σ = 3 cm⁻¹ the weak cyclohexane bands lose 30–40 % at rank 8), and it shows as dips at the band positions in loaded − denoised. A rank too high costs little: from 20 to 40 the denoised solvent's noise goes from about 1.2 to 1.7 × 10⁻³ (from 3.0) in the notes' simulation, and that adds in quadrature to the sample's own noise. The Check button compares the result at r − 10 and r + 10. The noise edge drawn is τ = σₜ(√m + √n) for the m × n matrix, with σₜ from the decayed end of the signal. The reference is first put on an even grid at its median spacing; if its two ends differ by more than their noise, a straight line through them is taken out before the transform and added back after, so the periodic transform sees no jump. The products with H go through FFTs, and the leading 100 singular triplets come from a randomized range finder with power iterations, once per solvent; any rank up to 80 is then rebuilt in milliseconds. Single noise spikes are undamped exponentials, so the ranks above the bands keep the largest of them: a small rest of the noise stays.

9. Denoising the reference: Savitzky–Golay

   R̃_i = value at i of the least-squares quadratic through the N points around i

   Keep N well below the number of points across a solvent band, or the bands themselves get broadened. Either denoiser is applied to the solvent before every method and before its derivatives are taken.

Files

• Igor binary waves (.ibw): versions 1, 2, 3 and 5, Mac or Windows byte order, any real numeric type (complex waves: the real part). For a 2D wave pick the column (spectra in columns, x along the rows).
• Igor text (.itx): any number of waves per file; SetScale x and Note commands are read; 2D waves written with /N=(rows,cols).
• Packed experiments (.pxp) and templates (.pxt): every numeric wave is listed with its data folder.
• Delimited text (.txt, .csv, .dat): one wave per column; a header line gives the names. Two columns of the same file are offered as x and y.
• X axis: the wave's own scaling (SetScale), the point number, or any monotonic wave of the same length, from either file or from the separate x wave file. The solvent is interpolated onto the sample's x values, so the two may have different points and ranges; only the overlap is used. The x axis may run either way.
• Raman-loss spectra with the peaks pointing down need no flipping: the scale comes out positive when both spectra point the same way. The shift δ is in the units and direction of the x axis, so on a negative-cm⁻¹ axis its sign is reversed relative to |cm⁻¹|.
• Not read: HDF5 experiments (.h5xp) and text waves. Save the waves as .ibw or .itx (Data ▸ Save Waves) or the experiment as .pxp.
• Everything runs in your browser; the files are not uploaded anywhere.