I have heard that there is an obligation to have a viable Beis Din in a town. How big does the community have to be for that obligation to take effect?

Indeed, there is an obligation to have a Beis Din in a Jewish town as Horav Eliashiv Zt’l and others point out (See Vayishma Moshe p. 373). Horav Shlomo Miller’s Shlit’a opinion is that there are many difficulties in establishing a correct Beis Din, as it is unusual to find and engage the instructed and reliable Talmidei Hachamim in our common cities and towns, especially outside of Eretz Yisroel. Therefore, one must rely on the well established Batei Din and recognized courts in some of the closest big cities.