Issue with cloud processing platform
Incident Report for PRX
Resolved
This incident has been resolved.
Posted Oct 09, 2014 - 16:17 EDT
Monitoring
We have replaced the master server for our cloud processing platform. At this time, all jobs that had been delayed by the original outage have been updated and follow-on jobs have been scheduled.

If you continue to see issues with tasks that usually happen in the background (such as deliveries, validation, and processing), it may be unrelated and should be reported to the help desk.
Posted Oct 09, 2014 - 15:08 EDT
Identified
Several minutes ago, were notified of an issue preventing our cloud processing platform from correctly updating the status of jobs for prx.org and networks.prx.org. While the jobs are being processed, the fact that the progress of the job is not being updated is preventing additional work from being scheduled.

The source of the issue has been traced to a corrupted filesystem used for storing transient logs. We need to restart the affected server to complete necessary repairs to the filesystem, so we have temporarily halted processing and will resume once the server has been repaired.

We expect the process to take several minutes.
Posted Oct 09, 2014 - 14:21 EDT