12
6 Comments

Coinbase spent $16,000,000 on an ad that brought down their website

👋 Hey Indie Hackers,

First of all, I find it unbelievable that something like this has happened to a company the size / worth of Coinbase.

Brought to my attention by Mr Snowden:

Now, how do you prevent such occurrence, it should be easy, right, especially for a well funded company like Coinbase?

Here is how I think all Indie Hackers can ensure that their websites won't be brought down when you spend that $16,000,000 on your solo-SaaS:

  1. Build your frontend using one of the static-frontend frameworks like Next.js, Nuxt.js - or whatever is your favorite
  2. Host your frontend for FREE with Cloudflare - sure can host with Vercel but they'll bill you for bandwidth and AWS-like GB-hours per invocation pricing.
  3. On the backend, utilise memory based caching (Redis, Dynamo etc.), not disk (even if SSD) as memory r/w can be up to 100x faster than the SSD. And, of course host it with a serverless service such as AWS Lambda, Cloudflare Workers or similar.
  4. For the actual database, any serverless option would do but if I was Coinbase I'd use AWS Aurora. As an Indie Hacker who loves to keep costs down I'd use AWS RDS with auto-scaling enabled.

Now, I don't think that ad would bring down Coinbase if they had a setup as above.

I've not seen a write-up on what actually brought it down as the bottleneck or issue might be in various places in the request flow, but I'd be really interested to find out.

Afaik, Coinbase runs their backend on Rails and React on the frontend.

What do you think? How would you ensure that a $16,000,000 ad wouldn't bring down your website?

Cheers!

  1. 6

    They spents 16M$ for international publicity and branding.
    Bad publicity is still publicity

    don't worry, they are still winning :p

  2. 3

    Hmm yeah coming from a place where scaling web-apps was one of the issues, I'm not able to wrap my head around why this tanked that much.
    They did an awesome job with their ad - and for sure they must have anticipated a spike in traffic - because... I mean.... superbowl :-)

    One additional thing which I find highly interesting: The site most probably didn't crash due to logged in users - simply due to the fact that so many people were accessing the homepage. So actually I would even go so far and say that the database was not really involved at all. Maybe really only the frontend and mainly the backend.
    All speculation, but I guess at least somewhat reasonable. By the way, I'd love to see an incident report from coinbase, this would be a great learning opportunity.

    Ok, all of that being said and assuming coinbase uses Rails and React as you suggested, I'd go for the following:

    • Make sure the frontend is appropriately horizontal scalable. Either use one of the cloud providers auto-scaling options (Azure ACI or AWS ECS) or if you have virtual-machine-based deployments, you will have Kubernetes anyhow. Make sure to have autoscaling enabled for your frontend at least.
    • Have a proper load balancing in place. I don't know the load balancing options for ACI or ECS by heart because we had our self-hosted HAProxy load-balancer back than. However I guess they all have a similar feature set. With HAProxy you have multiple settings to prevent server downtime. Having settings like maximum connections and throttling actually should guarantee that you don't run into downtimes. Dynamically change max connections with the number of horizontally scaled services.
    • In the case of coinbase, I'd make sure to have at least some operations folks do manual service monitoring. Beforehand, prepare easy to install worst-case-mechanisms, like setting throttling or again max. connections, if the servers are hitting a limit.
    • Make sure you have enough bandwidth available. This one is actually a big one. Depending on which cloud service you use, you have limited network (and disk!!) bandwidth. Do some calculations and make sure you have enough provisioned - or at least have the option to easily provision more bandwidth.
    • Overprovision. I mean, look, it was clear, that a lot of users will hit coinbase during the superbowl ad. Make sure to have 100x or more of the resources you would normally have. If you are coinbase, your application (if it's at least somewhat state of the art) will easily scale to 100x and scale back 2 days after the superbowl without a massive increase in costs.
    • I'd also look into from where my content is delivered. As the superbowl is a global event, I would definitely look into content delivery networks which can cache and serve my site around the globe. As you already hinted, cloudflare is one of the good solutions here.

    Writing all of that, I realize it really comes down to planning. I'm sure coinbase has IT operations people who are smarter on more experienced than me. They of course would have known the above points. But I guess they were simply not expecting this HUGE load of traffic. Maybe they even were not informed about the nature of the advertisement they did. Coinbase simply had a QR code displayed on-screen, nothing else. This is genious from a marketing perspective, because everybody will scan the QR code, just to look what's behind. So naturally, this ad will provide many many times more website visits compared to "normal" superbowl ads. Maybe the IT operations simply calculated a normal superbowl ad conversion?

  3. 2

    I saw a Twitter post which tried to analyse the ROI of that Super Bowl ad.

    With some sketchy metrics it was estimated the the $16M ad generated ~$22.5M for the year.

    Full post: https://twitter.com/montemagno/status/1493292188482027531

  4. 1

    This comment was deleted a year ago.

    1. 1

      Good points @Primer

      Other news / media outlets reported similar things but said it was $14M and not $16M.

      Surely Superbowl ads don't come cheap.

      1. 1

        This comment was deleted a year ago.

        1. 1

          FYI, this comment is a post on social media.

    2. 1

      He probably just opened up PRISM or Xkeyscore to check. Easy Peasy.

      1. 1

        This comment was deleted a year ago.

Trending on Indie Hackers
Passed $7k 💵 in a month with my boring directory of job boards 39 comments Reaching $100k MRR Organically in 12 months 33 comments 87.7% of entrepreneurs struggle with at least one mental health issue 14 comments How to Secure #1 on Product Hunt: DO’s and DON'Ts / Experience from PitchBob – AI Pitch Deck Generator & Founders Co-Pilot 11 comments Competing with a substitute? 📌 Here are 4 ad examples you can use [from TOP to BOTTOM of funnel] 10 comments Are you wondering how to gain subscribers to a founder's X account from scratch? 9 comments