Identified - We’ve identified a potential Linux kernel regression around networking performance. We initially suspected that the impact was limited to Docker applications. However, while our initial mitigation of separating Docker and Buildpack applications lowered the rate of failure, we continue to see failures downloading resources in the Buildpack specific infrastructure.

We are rolling out an update that we believe mitigates the regression to a portion of the platform. We intend to monitor the update for a short time before applying it to the remainder of the platform.
Oct 12, 15:45 PDT
Monitoring - Our mitigation has successfully controlled the error rate and only Docker apps should experience turbulence. We are monitoring the platform for further symptoms. A permanent resolution to Docker-based app issues will be forthcoming.
Oct 8, 17:53 PDT
Update - We have finished applying a temporary mitigation and the platform appears to have stabilized as of 2018-10-07 02:00 UTC for buildpack-based applications. The root cause is still being investigated but is related to an OS update adversely affecting Docker-based applications. Docker-based applications have been moved to separate infrastructure as to not negatively impact buildpack-based applications so we expect Docker-based applications to continue to have intermittent issues until a fix can be put in place.
Oct 8, 08:39 PDT
Identified - The issue has been identified and we are installing a temporary mitigation while we address the underlying cause.
Oct 5, 10:14 PDT
Investigating - We've rolled out some fixes, but are continuing to see staging failures and are investigating.
Oct 4, 06:59 PDT
Monitoring - The deploy to mitigate this issue has completed and we are seeing successful application pushes. We'll continue to monitor.
Oct 3, 17:32 PDT
Identified - We believe we have identified the issue and are deploying a fix now.
Oct 3, 16:19 PDT
Investigating - We are currently seeing failures when staging and or starting application (e.g., `cf push`) and are investigating.
Oct 3, 15:31 PDT

About This Site

Pivotal Web Services is the latest public release of the OSS Cloud Foundry Project.

Apps Manager (formerly Developer Console)   Operational
Pivotal Web Services API   ? Operational
Application Execution Pool   ? Partial Outage
Marketplace Service Instances   ? Operational
Marketplace Services API   ? Operational
Router   ? Operational
PCF Metrics (Console for Application Monitoring)   ? Operational
App Autoscaler   ? Operational
UAA (Authentication Server)   Operational
Application logging (Loggregator)   ? Operational
Pivotal Apps Operational
90 days ago
99.99 % uptime
Today
support.pivotal.io   Operational
90 days ago
100.0 % uptime
Today
login.run.pivotal.io   Operational
90 days ago
100.0 % uptime
Today
network.pivotal.io   Operational
90 days ago
99.99 % uptime
Today
pivotal.io   Operational
90 days ago
100.0 % uptime
Today
Billing and Account Management, Pivotal Web Services   ? Operational
90 days ago
100.0 % uptime
Today
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
API Response Time
Fetching
Past Incidents
Oct 16, 2018

No incidents reported today.

Oct 15, 2018

No incidents reported.

Oct 14, 2018

No incidents reported.

Oct 13, 2018

No incidents reported.

Oct 11, 2018

No incidents reported.

Oct 10, 2018

No incidents reported.

Oct 9, 2018

No incidents reported.

Oct 7, 2018

No incidents reported.

Oct 6, 2018

No incidents reported.

Oct 3, 2018
Resolved - From approximately 2018-10-03 21:00 to 2018-10-04 00:30 UTC, application developers deploying to Pivotal Web Services (PWS) would have experienced failures in all attempts to push or scale applications. The underlying cause was that a set of TLS certificates used by platform-internal scheduling and placement components had expired, resulting in communication failures between those components. The monitoring PWS has in place to warn of upcoming certificate expiration missed the certificates in question; that monitoring has been updated.
Oct 3, 01:00 PDT
Oct 2, 2018

No incidents reported.