Differences of the Selberg trace formula and Selberg type zeta functions for Hilbert modular surfaces

We present an example of the Selberg type zeta function for non-compact higher rank locally symmetric spaces. This is a generalization of Selberg's unpublished work [26] to non-compact cases. We study certain Selberg type zeta functions and Ruelle type zeta functions attached to the Hilbert modular group of a real quadratic field. We show that they have meromorphic extensions to the whole complex plane and satisfy functional equations. The method is based on considering the differences among several Selberg trace formulas with different weights for the Hilbert modular group. Besides as an application of the differences of the Selberg trace formula, we also obtain an asymptotic average of the class numbers of indefinite binary quadratic forms over the real quadratic integer ring.