Monday September 22, 2025
Outage
Reports of US customers unable to access Blueprint OMS
Issue summary:
Dear Customers,
Please accept my sincere apologies for the service disruption earlier this week.
Blueprint OMS was unavailable for all US customers between 11:15am and 1:15pm (EST) on Monday, Sep 22.
We are committed to providing a reliable service that you can depend on, and we take this responsibility very seriously. We have already implemented changes that will prevent a similar occurrence in the future, and the details of our investigation are shared below.
Once again, I sincerely apologize for the inconvenience this caused for your staff and patients.
Kind regards,
Aleem Sunderji
Founder, CTO
--
Investigation results:
While our infrastructure partner was deploying a new server, an automated tool unexpectedly terminated several existing application servers, which brought the Blueprint OMS application offline.
We began restoring the servers from backups right away, and brought them online as soon as the restoration was complete. That process took 2 hours.
Action plan:
We have made several changes to ensure that this doesn’t happen again, including:
- Implementing access restrictions to prevent automated tools from deleting servers
- Requiring an additional review by a second engineer before executing any plan generated by an automated tool
We have also reiterated to our partner the importance of performing any infrastructure changes outside of business hours, to avoid any potential impact for our customers.
Dear Customers,
Please accept my sincere apologies for the service disruption earlier this week.
Blueprint OMS was unavailable for all US customers between 11:15am and 1:15pm (EST) on Monday, Sep 22.
We are committed to providing a reliable service that you can depend on, and we take this responsibility very seriously. We have already implemented changes that will prevent a similar occurrence in the future, and the details of our investigation are shared below.
Once again, I sincerely apologize for the inconvenience this caused for your staff and patients.
Kind regards,
Aleem Sunderji
Founder, CTO
--
Investigation results:
While our infrastructure partner was deploying a new server, an automated tool unexpectedly terminated several existing application servers, which brought the Blueprint OMS application offline.
We began restoring the servers from backups right away, and brought them online as soon as the restoration was complete. That process took 2 hours.
Action plan:
We have made several changes to ensure that this doesn’t happen again, including:
- Implementing access restrictions to prevent automated tools from deleting servers
- Requiring an additional review by a second engineer before executing any plan generated by an automated tool
We have also reiterated to our partner the importance of performing any infrastructure changes outside of business hours, to avoid any potential impact for our customers.
Sep 22, 12:15 PM CDT
Blueprint OMS is beginning to come back online, and full functionality is expected to be restored within the next hour. Our team continues to make steady progress, and we’re closely monitoring the system as services are restored. We’ll provide further updates here. Thank you for your continued patience and understanding.
11:40am CDT
Our engineers have identified the cause and are currently implementing a fix. We expect Blueprint OMS to be unavailable for the next 1-2 hours. We will post updates here.
11:00am CDT
We have received reports of customers unable to access Blueprint OMS (all services unavailable).
Our engineering team is actively investigating the issue and working to restore services as quickly as possible. We will continue to keep you informed here as we learn more.
Sep 22, 10:33 AM CDT