Penn Yan First Baptist Church Church of Penn Yan, NY. The location of Penn Yan First Baptist Church Church is 14527, 224 Main St - marked with a pin on Google Map.