
Akhalkalaki is one of the bigger cities in southern Georgia. It is something of a travel hub, with connections to Tbilisi and Akhaltsikhe. It is not ideal to visit Vardzia from here, but is is possible. The easiest way is probably to take a marshutka minibus to Khertvisi and try to catch one there to Vardzia.
From Akhalkalaki you can cross into Armenia to Gyumri. There are minibuses and taxis. A taxi to Gyumri should cost about 20 dollars.