This is a space where I'll digest, report, aggregate, and analyze re:Invent 2021. A tremendous amount of announcements come out every year with this conference, so having a space for making sense of it is necessary for me, but I hope it will also be useful for you, dear reader 😉.
re:Invent 2021 was my first time visiting the conference, despite having worked with AWS for several years. I was dumbstruck by the sheer amount of preparation, effort, care, thoughtfulness, planning, design, and empathy that the community and Amazonians put into the conference. I had a great time with old and new friends, met lots of people, and learned a ton.
This year, there were a few important themes that came out of the conference, either as commonalities across multiple product/feature announcements, as thru-lines in the messaging, or as explicitly stated ideas (such as The Everywhere Cloud.)
Over the years the Serverless offerings at AWS have become increasingly mature. The most striking expansion of Serverless offerings this year had to do with the expansion of Serverless offerings as the compute modality for existing managed services. Before we jump into some of the new developments, I'll just leave this here:
https://twitter.com/landau_charles/status/1461133038084145160
MSK Serverless is a powerful example here. Ultimately, the Kafka API (and Kafka Connect ecosystem) is the "thing of value" that we usually want. For some organizations running your own Kafka cluster will be mission critical, but for everyone who can offload it to a trillion-dollar company and only pay for data throughput, that's very compelling. I've read enough gripes about the pain of running your own Kafka cluster to understand that this can be not just a point of technical leverage, but really a quality of life benefit.
Jonathan Ellis recently did an excellent deep-dive into Pulsar and how to reason about the Kafka ecosystem on Software Daily. In it, he pointed out that many users are interacting with Kafka through prebuilt connectors, rather than the APIs, and for users like this the underlying implementation is not relevant.
One important thing to note about this development is that the product design embodies the trend: the Serverless offering is more opinionated, with more constrained config available to the end user. Namely, the broker settings are fully locked-in, and you have a constrained set of topic configs.
EMR Serverless: managed Spark with a Serverless bent is obviously very compelling. Running a Spark cluster is not particularly fun. Spark shares a lot in common with Kafka in that we often only want the cluster for the sake of the Spark APIs and ecosystem. Having a near-zero-ops Spark cluster is therefore "the dream".
It bears mentioning that Serverless Spark offerings are not necessarily new. Having said that, as the largest player in the market it's a different proposition for AWS to add feature parity with the other cloud vendors.
One key point here is that the underlying infrastructure is not fully abstracted, because the worker compute and storage sizes are configurable. (The service is essentially magic already, so this shouldn't be interpreted as a gripe — I'm only pointing it out as "less-Serverless" on the spectrum of things.)
Serverless Inference for SageMaker Model Endpoints: this one is big news for a certain class SageMaker users. Deploying models prior to this offering would mean standing up at least one inference instance with sufficient resources to serve the API requests. GPU instances are pricey.
To be sure, users with deployed SageMaker models under constant load can achieve high utilization and better unit economics on those instances, but that's a headwind for adopters. Essentially, each new endpoint deployment would have a marginal cost associated with it. Now, it will be interesting to see what new usage patterns can be enabled when that marginal cost is vanishingly small.
There were even more Serverless announcements:
AWS continues to develop custom silicone. Graviton3 is their latest ARM offering, and this year they've also launched a partner program to validate software that's optimized for Graviton. This, combined with Apple's M1 line of system-on-chip offerings, is a compelling reason for vendors to support ARM chips, or cross-compiled languages that can easily be built for different architectures (e.g. Golang ).
The partner program is an extension of the Service Ready program, and a pretty clear signal to the market. If your customers are interested in the price performance of your software in the cloud, and you don't have a plan for supporting ARM architectures, AWS is dropping you a hint. Optimizing for Graviton specifically is an interesting topic which I would like to learn more about.