Those of you with live products - how painful are Production Bugs to you?

Tal Buchshreiber
6 replies
Production bugs are often inevitable, and usually result in some angry users. While this is a mess on any step of the way, it could be devastating for a relatively new product in the market. Please share if how often does your product face such bugs, and how do you handle the situation? How do you prevent them?

Replies

Coro
Every new version stays on the staging server for some time for the testers. The codebase has tons of automated tests and telemetry set-up so that I can easily debug potential issues in the production system. And even if something bad makes its way to production, I usually have the Docker images of the previous version handy so that I can rollback to the last version with minimal downtime. The only way to prevent them is to have a consistent codebase with automated tests, and make real users test the deployment before it makes its way to production.
Tal Buchshreiber
@zignis a pro answer, thank you very much. On which layer are the automated tests? Unit/component/api/ui?
Coro
@tal_buchshreiber1 for the backend there are unit and integration tests across the api endpoints, utility functions, background jobs, and postgres functions & triggers (pgtap). for the front end there are component tests and integration tests using jest and playwright.
Gurkaran Singh
Production bugs are like uninvited guests at a party - never a pleasant surprise! ๐Ÿž The key is to tackle them head-on and learn from each occurrence. As a tech enthusiast myself, I understand the struggle! Share your insights on managing those pesky bugs effectively. ๐Ÿ’ก
Neel
Difficult to avoid them especially at an early stage when you have limited developers and practically no testing/QA team. We do some testing ourselves; yet there are minor issues which escape.