Until Microsoft adds a native command that easily connects virtual directories to Azure Web apps, we're going to have to do it ourselves.

Currently, App Service Static Web Apps are available in the Central US, East US 2, West US 2, East Asia, and West Europe Azure regions and as preview starts with a free plan.