Neumann spectral problem in a domain with very corrugated boundary

Let Ω⊂Rn be a bounded domain. We perturb it to a domain Ωε attaching a family of small protuberances with “room-and-passage”-like geometry (ε>0 is a small parameter). Peculiar spectral properties of Neumann problems in so perturbed domains were observed for the first time by R. Courant and D. Hilbert. We study the case, when the number of protuberances tends to infinity as ε→0 and they are ε-periodically distributed along a part of ∂Ω. Our goal is to describe the behavior of the spectrum of the operator Aε=−(ρε)−1ΔΩε, where ΔΩε is the Neumann Laplacian in Ωε, and the positive function ρε is equal to 1 in Ω. We prove that the spectrum of Aε converges as ε→0 to the “spectrum” of a certain boundary value problem for the Neumann Laplacian in Ω with boundary conditions containing the spectral parameter in a nonlinear manner. Its eigenvalues may accumulate to a finite point.