Beginning around 16:24 UTC on July 16, 2018 latency on requests to the API nodes that back the Apps Manager developer console jumped significantly. Investigation narrowed the problem to an issue with DNS resolution on one API instance. The issue was resolved by restarting
bosh-dns on the affected instance.
The issue was a bug in
bosh-dns that manifested itself on a single API instance. From that instance
uaa.service.cf.internal could not be resolved.
curl --cacert uaa_ca.crt https://uaa.service.cf.internal:8443/token_keys curl: (6) Could not resolve host: uaa.service.cf.internal
bosh-dns on the instance, the bosh team extracted logs from the instance for further analysis.
Customers may have noticed occasional long page load times in Apps Manager.
After the bosh team had analyzed the affected instance, the
bosh-dns job was restarted, which has mitigated the issue for now. The fix was deployed on 2018-07-26.