The Jamul Casino is the closest casino to downtown San Diego Three of these are less than 30 miles away from the heart of downtown. In total, there are ten casinos within an hour’s drive of San Diego. While there are 24 casinos in Southern California total, 9 of these are located within San Diego County.