Maybe in rural areas of the country, not in the more "modern" parts, that's for sure. I know that there should still be a military unit in the Caucasus where horses are present as a full-fledged transport, but I'm not sure about that now. I read about them too long ago.