What It's Like To Code For Amazon

Adam Nathaniel Davis - Jul 12 '22 - - Dev Community

I haven't blogged in four months. I've been really busy because... I started working for Amazon. (You can read about my hiring process here: https://dev.to/bytebodger/how-i-got-hired-at-amazon-3ajl) While this post won't contain any of my usual this-is-what-I-hate-about-programming rants, or any useful how-to tips-n-tricks, I've been thinking that some of you might be interested to know just what it's like to work for a tech behemoth like Amazon. I know I always wondered - and now I can share some of the (occasionally jaw-dropping) things I've experienced in my short time here.

For the record, I'm a self-taught dev with a quarter-century of experience. I've kinda "done it all" during my long career, but I'm mostly focused now on frontend dev - and that's what I was hired to do for Amazon. Specifically, I work on a team building the tools used by our marketers to send billions of messages (usually: email) to our millions of customers around the world.

It may also be useful to remember that I work entirely remotely. My team is based in Seattle. But even they are almost all working from home. And there are no plans to force us back into the office.


Image description

Disclaimers

First, let's get a few things out of the way:

I've only been here for four months. Of course I don't know everything there is to know about working for Amazon. In my short tenure (so far), I've had the chance to work on and interact heavily with two separate teams. But obviously, there are millions of Amazon employees and hundreds of teams. Even if we "only" narrowed it down to just the programming teams, there are still So Many Teams. So nothing I write here is the end-all/be-all of what it's like to work for Amazon.

I've also discovered that Amazon is no monolith. This means that, while there are certainly some aspects of our culture that probably apply to many/all teams, it's also inevitable that some teams operate in a manner that's entirely different from mine. So some of the things written here may have no bearing on you, if you're working (or will eventually be working) for Amazon. This is true even if you're a frontend software engineer, just like me.

That being said, I've definitely started to realize that some of the things I've experienced are absolutely driven by Amazon's culture. So I'd imagine that other devs would at least be able to nod along to some of the things I'm outlining here.

I'll also state, right up front, that I've been pretty darn happy in the early goings of my Amazon career. So none of this should be taken as a "rant" or as me "bitching" about anything at Amazon. There are some things that I'd define as, umm... suboptimal. But overall, I'd take this job again in a heartbeat.

So let's dive into those... "quirks".


Image description

(Lonely?) Autonomy

When I interviewed, one of my last interviews was with a senior architect type. He's not in my current group. But he was definitely well-versed with what I could expect if I were to be hired. So I asked him, quite simply, "What is it like to work at Amazon??"

I ask this question a lot, because I'm usually not too concerned with any particular tech stack. And I don't often care much about the nitty-gritty details of the health benefits. Instead, I wanna know - preferably, from someone who's done my job (or been on a team similar to mine) - what the day-to-day worklife is really like.

At the time, his answer really struck me. And it's only continued to resonate in my head as I've become integrated into Amazon's processes. He said,


"Well... you have a lot of autonomy. A ton of autonomy. Almost "too much" autonomy. It's the kind of autonomy that can actually get some people in trouble."



Now that I'm "in the fold", I know exactly what he was talking about. This is not a place that will babysit you or hold your hand. This is not a place that will try to dictate your agenda. Yes, we have sprints and tasks, and you're obviously responsible for delivering on those tasks. But exactly how you go about accomplishing those tasks is almost completely up to you. So much so that I don't think I've ever seen anything quite like it.

Now you may be thinking, "That's great!!!" And to be sure, it is! But it also requires you to be inherently self-driven. It requires that you speak up - loudly, and frequently - if anything has you "stuck". It requires that you learn - most likely, by yourself - how to navigate the company's (incredibly, mindblowingly) massive databanks of internal documentation.

I've also noticed that this can leave me, at times, feeling like I'm on a bit of an island. Not that this is necessarily bad, per se. But if you thrive on constantly chatting with your coworkers about last night's Big Game, you may find yourself feeling a bit... alone.

Like so much of corporate America, we use Slack. And of course, I'm tied into about a dozen Slack channels that are central to my team and/or my job function. But I've gone entire days without seeing a single message posted to any of those Slack channels. Even when I post something there, it can sometimes be a bit of a wait before anyone responds.

To be fair, whenever I've reached out to anyone individually, I almost always receive a timely response. I'm not claiming that I feel ignored here - not at all. But at times, it's almost jarring to see the dearth of mindless chatter that I've grown accustomed to with other companies. (Of course, it's also incredibly refreshing - but it can be jarring as well.)

Also, on many days, I have one meeting. (Our regular standup.) I don't get blindly dragged into a dozen meetings - all having many dozens of near-useless attendees - just for the sake of saying that I was there. I don't get "looped into" an endless stream of emails on which I have no meaningful/useful input. My time is respected in a way that I'm, quite frankly, very unaccustomed to.


Image description

Dev-Driven Culture

I've worked for companies that were founded by devs. I've worked for companies where the devs were treated as some kinda "necessary evil". I've worked for companies that operated everywhere along that spectrum. But I don't think I've ever worked someplace where nearly all of our initiatives seemed to emanate almost directly from... the devs.

To be sure, we have execs higher up our chain who don't code and they certainly drive/approve/oversee our initiatives. But nearly every goal we've discussed has emanated - from us. And I don't just mean that the execs are leaving us alone.

My main team has more than 10 people on it. They're all devs. We don't have an embedded business analyst - or any BA "group" through which we must coordinate our activities. We don't have an embedded QA person - or any QA "group" through which we must validate our work. We don't have a product owner who tends to our queue and prioritizes fixes/features. We don't even have anyone who's a formal "project manager". Our dev manager (who doesn't really code anymore - but he definitely knows how to code) is basically our "project manager".

Do other teams have such roles? Probably. But I haven't met them yet.

Think about that for a minute. Our code drives a massively-important business function. It's responsible for the delivery of billions of outbound messages. And yet we don't have a single person who's constantly looking over our shoulder or trying to tailor our agenda to broader company/customer objectives.

Our timelines are driven by us. Our deliverables are defined by us. It's amazingly awesome. But it's also borderline-intimidating to think of the trust that's placed upon us.

They also give individual teams a tremendous degree of autonomy to pick their own toolsets. Do you think that new app should be built in Svelte? Well, if you can convince your own team that it's the best choice, then you'll probably be building it in Svelte. Do you hate (or love) Redux? Well, as long as your peers have bought in, you can probably ditch (or adopt) Redux. As far as I can tell, there are almost no overriding corporate "rules" stating that you must use (or forego) a given technology.


Image description

Owning The Business Objectives

While I love this approach, I also have to say that this can sometimes be a double-edged sword. What I mean is that, sometimes there's a certain security as a programmer in knowing that you're just the guy who "builds the tools". Because if you just "build the tools", it really doesn't matter what other people do with those tools - so long as your tools work.

For example:

We just finished this exercise in codifying our objectives for the coming quarter/year. Some of those objectives are things that are decidedly... "outside" the bounds of programming. Like, we're working on objectives to increase click-through rates, and to lower opt-out rates.

When I first read these, I thought, "I don't actually craft any of our marketing campaigns. How in the world can I control whether any particular campaign has a low click-through rate? Or a high opt-out rate?"

I mean, if a marketer uses our tools to develop a poorly-targeted, shoddily-worded, overly-ambitious email campaign that's misguidedly sent out to 500 million people, what can I possibly do about that??? I can't possibly be tasked with approving their campaigns. Or editing them. Or, in the most extreme example, killing them. So how can I be expected to drive metrics such as click-through or opt-out?

Of course, the reality is somewhere in the middle. No, I'm not expected (nor am I even wanted) to start sticking my nose in the middle of some campaign that's being crafted by a marketing group in Berlin. But I am empowered to think proactively about how our tools are used - and trying to strategize about every possible thing we could do in the software that could increase the success of our (many) marketing teams.

That can be a little overwhelming at times. But if I'm being totally honest, it's also pretty friggin cool.


Image description

The Endless Proliferation Of Custom Tools

If this is all sounding like a developer's wet dream, I do think there are some, umm... downsides. For example, Amazon has a seemingly-endless supply of in-house, custom-built tools - tools that do all the same stuff you've already learned to do with "industry standard" tools. Amazon doesn't just re-invent the wheel. They create a dozen specialized variants, all of which feature an insane level of over-engineering. And nearly every one of those wheels basically does the same thing you've come to expect from... any other wheel.

Are you really familiar with deployment tools like Jenkins? Well that's too bad. Because you won't be using any of them here. You'll be using Pipelines and Apollo. And you'll be using something called Brazil to manage all of your build artifacts.

Are you a master of virtualization tools like Kerberos or Docker? Well, you can just shove all that knowledge to the back of your mind. Because here you'll be using Rapid Development Environment (RDE).

Are you accustomed to running your apps locally with good ol' chestnuts like npm start or yarn run? Not at Amazon! You'll probably need to deploy/run all of your code on one of their EC2 machines (known internally as a Cloud Desktop).

Do you know ticket-management systems (like Jira) like the back of your hand? Won't do you a bit of good here. They built their own tool called Simple Issue Manager (SIM).

You've probably spent countless hours mining and managing company knowledgebases like Confluence. But Amazon wrote their own custom version (Of course!) called Quip.

Do you think you understand pull requests front-to-back? Well, you won't be using git to submit them here. You'll be creating a "change request" (CR) in their own custom tool - CRUX.

I could go on, but I can't even begin to tell you just how many tools they've created to do the exact same things that you've always been doing in other jobs.

In fact, they've created so many of their own custom tools, that there's little point in trying to Google for answers if you ever get stuck. But not to worry! They created their own custom version of Stack Overflow, called Sage!

To be fair, I get it. A lot of this actually makes perfect sense. Because it's been many years since Amazon made the majority of its money from online book sales. In fact, Amazon doesn't even make the majority of their money from selling any kinda tangible "products" online. Amazon's biggest cash cow is, hands down, AWS.

And what is AWS? It's a massive conglomeration of network/code management tools that they built for their own internal use - and then they productized them by opening them up to the public. But it can still be hella-frustrating when you have to learn, from scratch, how to do something that you mastered many years ago with "standard" open-source tools.


Image description

Cultural Quirks

Like all corporations, Amazon definitely has their own unique culture. Some of those cultural quirks are great. And some of them are, ummm... culture.

Automation

I suppose it stands to reason that a company the size of Amazon could never survive with a plethora of manual processes. But it can be outright exasperating when you run into a "problem" and that problem can only be fixed by submitting a ticket. To someone... on the other side of the world (literally). Who may-or-may-not get back to you, in any way, in the next week-or-more.

I'm not just talking about your standard help-desk kinda stuff. I'm talking about really stressful stuff, like - when there's a problem with your pay. Or when you don't know where to sign up for a certain benefit. Or when you've been (mistakenly) locked out of the network.

For some of the most pressing personal/HR issues, there is simply no one who can/will talk to you in real time.

Shared, Silent Reading Time

I don't know if I'll ever truly get comfortable with this. But I've already witnessed it over and over again.

Someone schedules a meeting where everyone's supposed to review some lonnnnnng document. So they send you a meeting request. But there's no link to the document in the request. Instead, they get many people on the call, and only then do they send everyone the link, and then they sit there, on a live call, and tell everyone to silently read the document. This often takes 10, 15, or 20 minutes. A quarter hour (or more) of everyone sitting in silence while they all, independently, read a document.

Why they can't simply send everyone the link before the call, I'll never understand. But I've already witnessed this so many times that I'm convinced it's an embedded - and probably unchangeable - part of Amazon's culture.

A Java/Backend Mindset

This probably doesn't apply to some teams. Heck, maybe it doesn't apply to most teams. But from the apps I've been exposed to, it's painfully clear that most of the devs are "backend driven". Specifically, they're incredibly Java-driven.

Not that there's anything inherently "wrong" with this. There are millions of Java LoC at Amazon. So of course their dev focus is gonna be largely centered on Java. But I've noticed that general frontend proficiency can be... lacking.

On my team, I'm (currently) the only frontend-focused dev. I was farmed out to another team on a temporary project - on which I was the only frontend-focused dev. My friend also just got hired on an entirely different team (in AWS), on which he's the only frontend-focused dev.

The "problem" with this is that, sometimes, I find that teams have very little knowledge about their own frontend apps. And when they make new design decisions, they're often driven from the mindset of a "backend dev".

For example: Just today I had a really strange conversation with a colleague when I tried to submit code and realized that it wouldn't compile - because the build engine is configured to only allow code that's ES5 (or earlier) compliant.

Lemme be clear on this. We're building a new app from scratch with a significant frontend presence. But everyone else on the team can't fathom why we should possibly consider dropping the use of var. Yes... really. I've also been met by the digital equivalent of blank stares whenever I suggest that a particular app/page/feature shouldn't require a complete round-trip back to the server every time the user provides any sort of input. In fact, I've specifically been told that we should continue building new functionality in an old-skool client-server architecture because most of the people on the team are Java devs and this is what they'll understand. Umm...


Image description

All About The Benjamins (Sorta)

Of course, any "right" job or "wrong" job is about more than just money. But I'd be remiss if I didn't at least mention this.

You're probably aware that FAANG companies tend to pay quite well. Extremely well. And while money alone can't make a crappy job great, the simple fact is that Amazon can be extremely... lucrative for experienced devs. And lemme tell ya. That's definitely a factor in making this "good" job into a great job.

I've been slinging code now for a quarter century. I've been privileged to receive a very generous salary from all of my employers over the last coupla decades. But Amazon?? Well... lemme tell you. They have deep pockets. And I'd be lying if I said that it didn't make any of the corporate "quirks" far easier to laugh off.

No, I'm not gonna spell out my exact compensation. But lemme put it to you like this:

  • I now make almost double my previous-highest salary.

  • They paid me a "signing bonus" (that isn't really a signing bonus, because it gets paid in 12 monthly installments throughout the year - so it's much more akin to "additional salary"). I get a signing bonus in Year 1. And another signing bonus in Year 2 (assuming that I haven't quit or been fired). The amount of each "signing bonus" is... insane.

  • I get stock grants. They'll be fully invested in three years. And the value of the grants alone is massive.

Again - none of that alone makes Amazon a good job. Even the best pay can't make up for, say, a crappy manager or a poorly-run company. But luckily, those "issues" have not been present in my situation. And since the job itself is genuinely rewarding, the pay is just a (massive, generous, bank-account-padding) cherry on top.

Would any of these observations apply to you, if you were to work at Amazon (or if you already work at Amazon)? I have no idea. Maybe my experiences are incredibly unique? Or maybe my viewpoints will change drastically after I've worked here for another four months? Or after a coupla years? Who knows?? But I'd imagine that many devs working here may experience some (or all) of the same things.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Terabox Video Player