User-agent: * Disallow: /self-api/ Disallow: /common-api/ Disallow: /*/v1/* Disallow: /v1/* Disallow: /*?utm_source= Disallow: /*?ref= Disallow: /login/ Disallow: /page-preview Allow: / Sitemap: https://www.short.ai/sitemap.xml crawl-delay: 10