Get a free AI visibility report
One of our vendors that provides hosting some of our databases, servers, and other infrastructure disclosed they had an incident with unauthorized access to their systems that store customer environment variables. The vendor was Railway and they notified Cognizo at 7:00 AM ET on October 22 that Cognizo may or may not be affected. As a precaution, Cognizo immediately revoked all potential access to our systems that could have been affected. Additionally, Cognizo performed an in-depth analysis and monitoring our systems and logs. The result of these actions is that the Cognizo team does not believe Cognizo or any Cognizo customers are affected by this incident but it is our policy to share this information with you. At this time, Cognizo has found no malicious access and our services remain secure.
To further minimize any future incidents, Cognizo will be migrating off of Railway to AWS (Amazon Web Services) within a 30-day period. This migration was already planned, but Cognizo is now prioritizing it.
Status: As it stands today, Cognizo and Cognizo customers are not impacted by this incident. Railway's investigation is still ongoing (learn more here), and the Cognizo team will update this post-mortem as Railway shares more information.
All times are Eastern (GMT-4).
| When | Event |
|---|---|
| Oct 21, morning | Railway detected activity on the systems that store customer environment variables. Railway paused all deployments. |
| Oct 22, 7:00 AM | Railway notified Cognizo of the incident, and the Cognizo team began the necessary security analysis and actions as per our internal protocols for vendor security leaks. |
| Oct 22, morning | The Cognizo team rotated every API key and secret stored in Railway. All logs and access were reviewed by the team. |
| Oct 22, 9:44 AM | Cognizo opened an incident on the Cognizo Status Page (status.cognizo.ai) |
| Oct 22, 11:10 AM | Status update: Cognizo found no malicious activity on our side. Users may see intermittent outages and unexpected sign-outs while new credentials roll out. Cognizo will continue to monitor for the next 72 hours and will follow the Railway incident until it is closed by them. |
Cognizo found no unauthorized access to Cognizo systems or customer data. The impact was limited to some service downtime while we rotated credentials.
The incident started at Railway, not at Cognizo. Railway reported that an unauthorized third party may have accessed the encrypted environment variables of some customers, and decrypted some of them. These variables can hold secrets such as API keys, database credentials and registry credentials.
To learn more about the Railway incident click here.
Cognizo followed Railway's recommended actions within hours of learning about the incident.
We will migrate our infrastructure from Railway to AWS within a 30-day period as an added security measure.
.svg)
.svg)
