Scenario

We are attempting to make a post request that involves calling a third-party API, the application is serverless and an EDA, so everything.
Nevertheless, this API occasionally experiences unexpected outages. We are currently exploring possible solutions to address this issue.
A typical "trust no one" problem can arise when integrating with external partners or suppliers, and the first thing that occurred in my mind was whether I should use a circuit breaker, However, this is only used in synchronous applications, and my application is entirely asynchronous. How can I achieve it? First of all, let's recap.

What is a circuit breaker?

Visualise your app as a rock star holding an arena concert, and the Circuit Breaker is that guy backstage who knows when you've just gone too far. You'll fry the stage wiring if you keep up like that.

So, your application is at the top of its lungs, each note is a service call. But then suddenly, the guitar amp (your database) wants to rest. Rather than having your app crying for more solos, like a cool roadie, the Circuit Breaker watches out after things, saying:

"Calm down there. The amp needs time to breathe."

It's like making sure your concert doesn't descend into a riotous frenzy, allowing the guitar amp (your company) to cool down.

Who wants to hear the tale of a concert fiasco?

In the curtain call, the Circuit Breaker is your backup plan for staying gigable even when glitches strike you out midair. So, you can rock on, but with a safety net.

Backing to the tech buzzwords

In a nutshell, a circuit breaker is designed for synchronous applications because it can interrupt the sequence of operations when one fails swiftly. In synchronous settings, operations occur linearly, allowing the circuit breaker to prevent cascading failures effectively.

In contrast, asynchronous applications, with concurrent and independent operations, often rely on different error-handling mechanisms better suited to their non-linear and non-blocking nature

Is it possible to accomplish an asynchronous circuit breaker?

I've been brainstorming and coming up with some out-of-the-box ways to achieve our goal.
I'm excited to share these two approaches with you!

1 - A self-healing Lambda function that adapts its throughput based on performance

This unbeatable article goes through a way how to handle third parties in an approach using kinesis and a ventilator lambda function, below it's a small piece of this treasure written by the marvellous serverless hero Yan Cui at the OG BurningMonk Blog.

The ventilator function can, therefore, self-adjust its batch size by updating the Kinesis event source mapping.
As the provider API’s response time goes up, the ventilator can respond by reducing its batch size. This gives the API a chance to catch its breath and recover. When the response time returns to acceptable levels, then the ventilator can gradually increase its batch size back to previous levels

2 - Take a break for your downstream 😅

After a quick catch-up with a 🇵🇱 Polish Wizard a.k.a Łukasz Kiedrowski 🧙‍♂️, giving me some wisdom, he sent me a quote that basically blew my mind. This quote, written by him, clarified the perspective I was wondering about, a way to achieve the wanted async circuit breaker using basically only a DLQ

In EDA architectures DLQ (maxReceiveCount + dynamic visibility timeout) will act as a circuit breaker in Synchronous architectures

This approach will protect you from the third party but won't act, in fact, as a "breaker". It will reduce the invocations instead of breaking it, protecting you from third-party failure.

This post presents some alternatives for implementing a circuit breaker within an asynchronous serverless environment.
Please let me know if you would like to see an implementation of this.

Of course, I would highly appreciate constructive discussion. If you don't agree with anything mentioned here, please feel free to leave your comments below.

Cheers!