How Hacker News crashed our service with 2.83 million server hits (what not to do!)

I've been building and shipping products for years, and had launched about 5 products on Hacker News before this. Most of the time I'd get 2 or 3 upvotes and very few people cared. For this reason, when I launched PrettyPolly (https://www.prettypolly.app) on Hacker News, I hadn't even considered how to scale it up. The website was set up on a pitiful $5 per month PythonAnywhere hosting plan.

The product also used two of OpenAI's APIs. When you sign up for an OpenAI developer account, there's a $120 hard limit on spending which you can only increase by contacting the OpenAI support team (it can take up to a week to get a response). Again, that didn't bother me because $120 was enough for millions of tokens and many hundreds of hours of speech-to-text transcription. I expected it to be many months before I'd hit that.

So I posted (https://news.ycombinator.com/item?id=36973400) and it got zero upvotes. I checked on another device and my post wasn't visible. I'd created an account with the handle 'prettypolly' which I later found out Hacker News didn't like, because it prefers you to post from a personal account (ideally with some post/comment history) rather than spamming from a company account. I emailed the moderator of Hacker News who explained that to me. He offered to create a new account with a personal handle and to make sure the post went live. I really owe him for this. Moderating Hacker News must be hard, but he was so damn friendly and helpful.

The post went live and it got to the top of Hacker News within 20 minutes. The first day was manageable. I just upgraded the hosting plan and was pleased to get about 1,000 signups on the first evening. I went to bed happy.

When I woke up the next day, the traffic had doubled. The app was regularly giving users '500 internal server' errors so I had to upgrade the servers again. I did a long day of customer support and tried to ship some improvements based on things customers were contacting me about. That evening, it got knocked off of the top spot on Hacker News so I assumed it was past the peak.

I woke up the next morning and the traffic had doubled again! This scared the hell out of me.. I realized that it was going to hit the $120 spend cap limit later that day if I didn't do anything. If it did hit the limit, the app would go completely down and no one would be able to use it (including paying customers). By this point, I had reached out to OpenAI through multiple channels about increasing the spend cap with no response. I decided to put everything behind a paywall temporarily until the spend limit was increased so our paying customers could continue to use it.

Even with that restriction, users ended up using 23 million tokens (words, sort of) on PrettyPolly in one day (image attached)! The server got 2.83 million hits in the first week.

Even after putting everything behind the paywall, it still hit the spend cap and the app went down completely on Day 5. Luckily, a friend/ex-colleague introduced me to his contact at OpenAI. He was able to increase the spend cap 10 mins later and the PrettyPolly was back up. I owe both of them a lot too.

We've now scheduled our Product Hunt launch for Wednesday 27th March 2024 (https://www.producthunt.com/products/prettypolly) and will hopefully not make all of the same mistakes as we did last time, but let's see!

Thanks for reading,