Smartphone-based audiometry provides an opportunity to perform hearing diagnostics when there is no direct access to a clinical audiologist. Besides providing access to a healthcare service in the poor areas of the world, reliable smartphone application can reduce the risk of visiting the clinics in the times of a global pandemic. Numerous studies assess various implementations of automated audiometry. They typically perform automated tests under controlled, sound isolated conditions and compare them with clinical audiometry. In this work we make a step towards analyzing how reliable such tests are in practice under the conditions intended for these type of solutions: at patient's homes. We used the data collected by Jacoti Hearing Center and evaluated the test-retest reliability of the smartphone audiometry under real-life conditions. The data base provides threshold measurements of more than 9000 users, but the test sessions are occasionally incomplete or performed in noisy environments, in which case the built-in noise monitoring aborts the test. Using data from 5946 ears, we formed pairs of test-retest sessions containing several frequencies in common. We computed threshold deviation for each frequency independently, and session-to-session deviation averaged across frequencies. We excluded data points originating from the borders of the measurable scale, as they lead to underestimated variability. We analyzed two diagnostic tests of Jacoti Hearing Center – screening test and full audiometry. For both tests, in all frequencies, at least 75 % of threshold differences were within the margin of 10 dB HL. The averaged session-to-session difference in 84 % of session pairs did not exceed 10 dB HL. For a subset of fully completed sessions from 1676 ears, the numbers increased to 85 % across individual frequencies and 90 % in averaged session-to-session differences. These results indicate that the smartphone-based audiometry can achieve a high reliability, even when performed at home.