In 2021, I built a startup called Linvo, a LinkedIn automation tool. It generated good money but also had a high burn rate.

I can tell you that I lost most of my money as a young entrepreneur (entirely bootstrapped) 😆

The problem with my system is that it kept on failing; I am not the best programmer in the world, but let me elaborate.

Puppeteer sucks your memory

Linvo went into a few iterations during its lifetime. I used the tech that I know: React and NestJS.

Basically, any automated cloud scraping goes into making tasks and then solving them.

It looks something like this:

• Cron runs every 10 minutes, looks for new stuff to do, and sends it to the queue.
• Workers take tasks from the queue and process them.
• Save everything to the DB.

This is a very naive solution that was running on multiple EC2 (AWS) and Redis.

In theory, it should work - Yes, you would need to scale the workers horizontally, but that was not my problem.

I started to experience something else - memory leaks.

As you probably know, Puppeteer - a chromium automated browser, is heavy (same as Chrome on your computer), and I was running lots of them - they sucked both my memory - causing the servers to stop working and took the internet bandwidth.

Needed a solution

Refactoring sounds like - yeah, let's do that, but my Intercom support was filled with support tickets (I already had many users), and I knew I had to create some crazy refactoring.

I had an idea for a solution - let's use AWS lambda.

• Lambda start
• Do the task
• Die (most of the time)

It doesn't suck your internet.
It will die and not cause a memory leak.

It's a bit more expensive because Lambdas were not meant to run indefinitely—that would remove the whole concept, but I was desperate. (Well, the good thing is that you don't need to think about scale.)

Here is the new infrastructure:

• Event bridge cron running every 1 minute (that's the minimum)
• Lambda that creates six events for the queue (making it like the cron runs every 10 seconds)
• SQS queue (bye bye, Redis)
• Lambdas shooting out with auto-scaling (yay?)

Vendor Locking

As you probably understand from the context - now I am all in with AWS, but I didn't mind - only how I would test it on my computer.

Back in the day, I used Sam. It's almost the same as Serverless to create the entire environment quickly.

And yes, they do bring some local testing. They can take your lambdas and turn them into endpoints! Yay!

But I was using Event Bridge, SQS, and Lambdas. That was just not possible.

I have encountered localstack. It was supposed to help me build an entire AWS stack on my computer, but many things were not supported back then, such as Lambda layers. And I was overwhelmed in using it.

So I started to create my "own" infrastructure where I deployed one "test" SQS on AWS and ran local code with it (a function that calls a function), and it was - bad.

I did not work well in production; I felt like the word was squeezing up on me back in the day.

I just wanted to get rid of this startup; I even offered it to people for free, taking the customers and rid myself of that problem.

Long story short, I managed to sell the system for a low price, but lately, I discovered something interesting.

Winglang - Typescript like dialect for the cloud

I guess my pain was real, and even though I would be happier to find this back in the day - I am glad it's alive today.

Winglang helps you to build the same cloud infrastructure as I used, but the nice thing is:

• It doesn't require you to use any configuration files. You basically define the entire infrastructure with code (you can check their entire preflight and inflight documentation)
• You can run it locally - thank you, god, 🙏🏻🙏🏻🙏🏻🙏🏻 with all the infrastructure included.
• You can deploy it to any cloud provider, and you don't have to use AWS. They can literally generate any Terraform or cloud formation for easy deployment.

I talked to the founder of Winglang, Elad (builder of the famous CDK), and he said something that made me twitch.

You can literally work on your projects when you are on an airplane and write tests for it 🤯

I can't give them enough stars myself - but do me a favor and star their repository and of course, use it for your next project.

https://github.com/winglang/wing

Final words

I am angry at Winglang. I wish they had done it sooner. Now they show up. When it's all about AI?

I started a new YouTube channel mostly about Tech, open-source, and marketing :)

If that's something that interests you, feel free to subscribe to it here:
https://www.youtube.com/@nevo-david?sub_confirmation=1