BRIEF DELIGHTS

Decisions API is in public beta

Article URL: https://developers.openai.com/api/docs/guides/decisions Comments URL: https://news.ycombinator.com/item?id=49984025 Points: 360 # Comments: 200

Amazon EC2 R9g and R9gd instances powered by AWS Graviton5 processors are now generally available

Amazon EC2 R9g and R9gd instances powered by AWS Graviton5 are now generally available, delivering up to 25% better compute performance than R8g, ideal for databases, in-memory caches, and real-time analytics.

Amazon Aurora PostgreSQL now supports direct querying of Apache Iceberg and Parquet data in your data lake

Amazon Aurora PostgreSQL now lets you directly query Apache Iceberg and Parquet data stored in your data lake alongside live operational data—no ETL pipelines required. Powered by DuckDB embedded within Aurora, this capability enables single queries that join transactional and historical data using familiar PostgreSQL syntax. It supports AWS Glue Data Catalog, S3, and S3 Tables, with optimizations like predicate pushdown and caching for efficient performance.

A deep dive into Vercel’s build infrastructure

Vercel has a new low-level untrusted and ephemeral compute platform—designed to give us the control needed to securely and efficiently manage and run builds. Since November 2023, this compute platform, internally codenamed "Hive", has powered Vercel’s builds, enabling key improvements like enhanced build machines and a 30% improvement in build performance.The platform operates on the fundamental assumption that we’re executing potentially malicious code on multi-tenant machines, requiring it to be safe, reliable, performant, and cost-effective. It’s architected to handle multiple use cases and can be composed in different ways depending on what’s needed. Most recently, Hive allowed us to reduce provisioning times for Secure Compute customers from 90 seconds to 5 seconds, while also improving their build speeds.We built Hive because we needed finer control and more granular management to continuously improve Vercel’s infrastructure, to meet the growing demands of our customers and to fulfill our vision of delivering the best development experience in the world. Read more

Fly Kubernetes does more now

Eons ago, we announced we were working on Fly Kubernetes. It drummed up enough excitement to prove we were heading in the right direction. So, we got hard to work to get from barebones “early access” to a beta release. We’ll be onboarding customers to the closed beta over the next few weeks. Email us at sales@fly.io and we’ll hook you up. Fly Kubernetes is the “blessed path"™️ to using Kubernetes backed by Fly.io infrastructure. Or, in simpler terms, it is our managed Kubernetes service. We take care of the complexity of operating the Kubernetes control plane, leaving you with the unfettered joy of deploying your Kubernetes workloads. If you love Fly.io and K8s, this product is for you. What even is a Kubernete? So how did this all come to be—and what even is a Kubernete? You can see more fun details in Introducing Fly Kubernetes. If you wade through all the YAML and CNCF projects, what’s left is an API for declaring workloads and how it should be accessed.

But that’s not what people usually talk / groan about. It’s everything else that comes along with adopting Kubernetes: a container runtime (CRI), networking between workloads (CNI) which leads to DNS (CoreDNS). Then you layer on Prometheus for metrics and whatever the logging daemon du jour is at the time. Now you get to debate which Ingress—strike that—Gateway API to deploy and if the next thing is anything to do with a Service Mess, then as they like to say where I live, "bless your heart”.

Finally, there’s capacity planning. You’ve got to pick and choose where, how and what the Nodes will look like in order to configure and run the workloads.

When we began thinking about what a Fly Kubernetes Service could look like, we started from first principles, as we do with most everything here. The best way we can describe it is the scene from Iron Man 2 when Tony Stark discovers a new element. As he’s looking at the knowledge left behind by those that came before, he starts to imagine something entirely different and more capable than could have been accomplished previously. That’s what happened to JP, but with K3s and Virtual Kubelet. OK then, WTF (what’s the FKS)? We looked at what people need to get started—the API—and then started peeling away all the noise, filling in the gaps to connect things together to provide the power. Here’s how this looks currently:

Containerd/CRI → flyd + Firecracker + our init: our system transmogrifies Docker containers into Firecracker microVMs Networking/CNI → Our internal WireGuard mesh connects your pods together Pods → Fly Machines VMs Secrets → Secrets, only not the base64’d kind Services → The Fly Proxy CoreDNS → CoreDNS (to be replaced with our custom internal DNS) Persistent Volumes → Fly Volumes (coming soon)

Now…not everything is a one-to-one comparison, and we explicitly did not set out to support any and every configuration. We aren’t dealing with resources like Network Policy and init containers, though we’re also not completely ignoring them. By mapping many of the core primitives of Kubernetes to a Fly.io resource, we’re able to focus on continuing to build the primitives that make our cloud better for workloads of all shapes and sizes.

A key thing to notice above is that there’s no “Node”.

Virtual Kubelet plays a central role in FKS. It’s magic, really. A Virtual Kubelet acts as if it’s a standard Kubelet running on a Node, eager to run your workloads. However, there’s no Node backing it. It instead behaves like an API, receiving requests from Kubernetes and transforming them into requests to deploy on a cloud compute service. In our case, that’s Fly Machines.

So what we have is Kubernetes calling out to our Virtual Kubelet provider, a small Golang program we run alongside K3s, to create and run your pod. It creates your pod as a Fly Machine, via the Fly Machines API, deploying it to any underlying host within that region. This shifts the burden of managing hardware capacity from you to us. We think that’s a cool trick—thanks, Virtual Kubelet magic! Speedrun You can deploy your workloads (including GPUs) across any of our available regions using the Kubernetes API.

You create a cluster with flyctl:

Wrap text

Copy to clipboard fly ext k8s create --name hello --org personal --region iad When a cluster is created, it has the standard default namespace. You can inspect it:

Wrap text

Copy to clipboard kubectl get ns default --show-labels

Wrap text

Copy to clipboard NAME STATUS AGE LABELS default Active 20d fly.io/app=fks-default-7zyjm3ovpdxmd0ep,kubernetes.io/metadata.name=default The fly.io/app label shows the name of the Fly App that corresponds to your cluster.

It would seem appropriate to deploy the Kubernetes Up And Running demo here, but since your pods are connected over an IPv6 WireGuard mesh, we’re going to use a fork with support for IPv6 DNS.

Wrap text

Copy to clipboard kubectl run \ --image=ghcr.io/jipperinbham/kuard-amd64:blue \ --labels="app=kuard-fks" \ kuard And you can see its Machine representation via:

Wrap text

Copy to clipboard fly machine list --app fks-default-7zyjm3ovpdxmd0ep

Wrap text

Copy to clipboard ID NAME STATE REGION IMAGE IP ADDRESS VOLUME CREATED LAST UPDATED APP PLATFORM PROCESS GROUP SIZE 1852291c46ded8 kuard started iad jipperinbham/kuard-amd64:blue fdaa:0:48c8:a7b:228:4b6d:6e20:2 2024-03-05T18:54:41Z 2024-03-05T18:54:44Z shared-cpu-1x:256MB

This is important! Your pod is a Fly Machine! While we don’t yet support all kubectl features, Fly.io tooling will “just work” for cases where we don’t yet support the kubectl way. So, for example, we don’t have kubectl port-forward and kubectl exec, but you can use flyctl to forward ports and get a shell into a pod.

Expose it to your internal network using the standard ClusterIP Service:

Wrap text

Copy to clipboard kubectl expose pod kuard \ --name=kuard \ --port=8080 \ --target-port=8080 \ --selector='app=kuard-fks' ClusterIP Services work natively, and Fly.io internal DNS supports them. Within the cluster, CoreDNS works too.

Access this Service locally via flycast: Get connected to your org’s 6PN private WireGuard network. Get kubectl to describe the kuard Service:

Wrap text

Copy to clipboard kubectl describe svc kuard

Wrap text

Copy to clipboard Name: kuard Namespace: default Labels: app=kuard-fks Annotations: fly.io/clusterip-allocator: configured service.fly.io/sync-version: 11507529969321451315 Selector: app=kuard-fks Type: ClusterIP IP Family Policy: SingleStack IP Families: IPv6 IP: fdaa:0:48c8:0:1::1a IPs: fdaa:0:48c8:0:1::1a Port: 8080/TCP TargetPort: 8080/TCP Endpoints: [fdaa:0:48c8:a7b:228:4b6d:6e20:2]:8080 Session Affinity: None Events: You can pull out the Service’s IP address from the above output, and get at the KUARD UI using that: in this case, http://[fdaa:0:48c8:0:1::1a]:8080.

Using internal DNS: http://.svc..flycast:8080. Or, in our example: http://kuard.svc.fks-default-7zyjm3ovpdxmd0ep.flycast:8080.

And finally CoreDNS: ..svc.cluster.local resolves to the fdaa IP and is routable within the cluster.

Pricing The Fly Kubernetes Service is free during the beta. Fly Machines and Fly Volumes you create with it will cost the same as for your other Fly.io projects. It’ll be $75/mo per cluster after that, plus the cost of the other resources you create. Today and the future Today, Fly Kubernetes supports only a portion of the Kubernetes API. You can deploy pods using Deployments/ReplicaSets. Pods are able to communicate via Services using the standard K8s DNS format. Ephemeral and persistent volumes are supported.

The most notable absences are: multi-container pods, StatefulSets, network policies, horizontal pod autoscaling and emptyDir volumes. We’re working at supporting autoscaling and emptyDir volumes in the coming weeks and multi-container pods in the coming months.

If you’ve made it this far and are eagerly awaiting your chance to tell us and the rest of the internet “this isn’t Kubernetes!”, well, we agree! It’s not something we take lightly. We’re still building, and conformance tests may be in the future for FKS. We’ve made a deliberate decision to only care about fast launching VMs as the one and only way to run workloads on our cloud. And we also know enough of our customers would like to use the Kubernetes API to create a fast launching VM in the form of a Pod, and that’s where this story begins.

Trading a Cloud Identity for Your Own: Workload Attestation on Managed Compute

By Dhruv PratapIntroductionOrganizations that have been around for a while usually run two identity systems side by side. One belongs to the cloud provider: IAM roles, instance profiles, execution roles. The other is your own, and it is the one your internal services actually check when they decide whether to answer a request.On infrastructure you build yourself, you can bootstrap your own identity however you like. On managed compute you cannot. The provider hands your process a cloud identity and nothing else.This post describes how we close that gap for Apache Spark workloads running on Amazon EMR. A workload that starts with only an AWS identity has to end up holding a first-class internal identity. Doing the exchange is straightforward. Making it trustworthy is the part that took the design work. Very little of what follows is specific to Spark or to EMR.Two identity systemsInternal service-to-service authentication at Netflix runs on a private PKI called Metatron. Every workload gets a short-lived X.509 certificate, and services authenticate each other with mutual TLS. A workload cannot simply ask for a certificate. It is issued only after the identity service has satisfied itself that the requester really is the workload it claims to be. That verification step is called attestation. In each environment we support (VMs, containers, functions) attestation rests on some environment-specific proof that the platform can check on its own.The second concept is the Data Project, which is our unit of ownership for data. A Data Project owns tables, has a set of authorized users, and has an identity of its own. Jobs run as the Data Project rather than as the person who launched them. That is what keeps table-level access control and audit consistent across scheduled and interactive use.So the problem is this. A Spark job on managed compute starts with an AWS execution role and no internal identity. Everything it needs from inside the company (reading an encrypted column, evaluating table ACLs, holding an interactive session open longer than a short-lived token lasts) requires the internal one.One identity, one roleThe design rests on a decision made outside the Spark stack entirely. Each Data Project identity maps 1:1 to a dedicated IAM role, and the service that owns Data Project metadata records that mapping.The mapping is what makes translation possible. It lets a statement in the provider’s vocabulary (“this process is running as role R”) become a statement in ours (“this process is workload W”). Without it there is nothing to translate into, and no amount of cryptography helps. The hard part of bridging two identity systems is rarely the protocol. It is committing to a mapping and then keeping it authoritative.One practical objection arrives immediately. A single AWS account cannot hold tens of thousands of IAM roles, and we expect data projects in the order of ten thousands. We handled this by sharding data project roles deterministically across a small pool of dedicated accounts, which scales well past the projected number of projects and has the useful side effect of keeping workload roles on the far side of an account boundary from the control planes that launch them.ComponentsFive components take part.The control plane is the only service allowed to launch Spark jobs. It resolves the Data Project’s IAM role, builds and signs a workload metadata payload, and submits the job with that role as the execution role. It holds a signing key issued to it for this purpose alone.The Data Project service is the authority for the identity-to-role mapping.The Identity service receives attestation requests, verifies them, and issues certificates.A Spark plugin provides the hook. Spark 3.0+ added a plugin interface with driver-side and executor-side components that are initialized while those processes are bootstrapping.AWS STS acts as a notary, which is not its usual job.Step one: the control plane makes a signed claimWhen a job is submitted, the control plane validates the caller, looks up the Data Project’s IAM role, and builds a small payload describing the workload it is about to launch. The payload names the application identity, the mapped role, and the environment, stack and detail that the resulting credentials should be scoped to. The control plane signs the payload and passes it and the signature through as ordinary job configuration.Two properties of this step matter. First, only one service can produce a valid signature, so the set of things that can assert “this workload is legitimate” is small and auditable. Second, the payload is a claim rather than an authority. On its own it proves nothing, because job configuration travels through infrastructure we do not fully control and anything that can read it can replay it.Step two: the workload proves it holds the cloud identityWhen the managed service launches the driver process, it does so under an OS user that has the execution role’s credentials available. The driver-side plugin initializes and uses those credentials to sign, but not send, a request to the provider’s identity endpoint (sts:GetCallerIdentity). The result is a short-lived pre-signed URL.That URL is a transferable proof of possession. Anyone can fetch it. Only the holder of the role’s credentials could have produced it. And the response comes from AWS rather than from the workload, stating which role signed the request. The workload cannot lie about the answer because the workload does not supply the answer.The pattern is not new. It is the same idea behind AWS IAM authentication in HashiCorp Vault and in AWS’s own function attestation flows. It generalizes well: any environment that gives a process cloud credentials and nothing else can still produce a verifiable statement about its own identity.The plugin sends one attestation request carrying both artifacts, the pre-signed URL and the signed metadata.Step three: corroborationThe identity service does four things.It fetches the pre-signed URL against AWS STS and reads back the role that signed it. This is the provider’s statement.It verifies the metadata signature against the key issued to the control plane. This is the platform’s statement.It corroborates the two. The role AWS reports must be the role the control plane said it dispatched, for the identity the metadata names.It issues certificates for that identity, scoped to the environment, stack and detail from the signed metadata.Step 3 is the point of the whole design. Neither statement is sufficient by itself, and they fail in complementary ways. The provider’s statement is unforgeable but under-specified: it gives you a role, and a role is not a workload. It says nothing about why this process exists or what it should be allowed to be. The platform’s statement is well-specified but unverifiable on its own, for the reasons above. Put together, the signature says the workload was legitimately dispatched and the pre-signed URL says this really is it. Neither party has to trust the workload’s own account of itself.There is a tension here that we settled deliberately, and other teams should expect to meet it. The provider’s answer identifies a role, not an application. Deriving an application name from a role name is fragile string matching, and the session identifiers the platform assigns are not under our control. We chose to take the application identity from the signed metadata and use the provider’s answer only for corroboration. That costs a network round trip and a stricter signing requirement, and it buys a much clearer trust story.The fan-out problemA Spark application is one driver and up to thousands of executors, created and destroyed throughout the life of a job. Attestation designed around one process per host does not survive that.There are two defensible answers.In the first, each executor attests independently. This is uniform, adds no new trust boundary, and grounds every process’s identity in the same provider-verified proof. The cost is amplification. One large job can produce thousands of STS calls and thousands of attestation requests in a burst, which looks a lot like an attack and creates a hard dependency on provider-side rate limits.In the second, executors inherit credentials from the driver. The driver attests once and distributes credentials to executors over Spark’s internal RPC, which we configure with authentication and AES-GCM encryption so that only processes in the same application can take part. Load on the identity service stays constant. The cost is a second trust boundary and a driver that is now a credential distribution point.Considering our scale at Netflix here we went with the second approach. The general lesson is that identity amplification is a capacity question, and it is better to answer it on purpose than to have it answered for you during an incident.LifecycleCertificates are short-lived by design, so bootstrapping is only half the work. On self-managed hosts a system service handles renewal. Inside managed compute there is no equivalent hook, so the driver JVM runs a timer that re-attests on a fixed interval, well ahead of expiry. Credential material is written to a temporary directory readable only by the process owner and removed when the process exits.Executors do not refresh. They are usually short-lived, and an executor that somehow outlives its credentials can exit and be replaced, which is cheaper than making every executor a renewal client.The rule worth carrying into any similar system is that attestation has to be a repeatable operation rather than a bootstrap step. Anything that can only happen once at process start will eventually be the reason a long-running job dies nine hours in.Why this generalizesIf you run your own identity system and are moving workloads onto managed compute, the specific services here matter less than the shape of the solution.Anchor on one exchangeable primitive. A single authoritative 1:1 mapping between your identity and a provider identity is what makes translation possible. The rest is plumbing.Require two independent claims and corroborate them. One from the provider, unforgeable and under-specified. One from your control plane, well-specified and replayable. Trust the intersection and never either alone.Keep the signer scarce. If exactly one service can make the platform’s claim, your trust story fits in a sentence.Hook the layer you still own. On managed compute you rarely control the host or its init system, but you almost always control the runtime: a plugin, an agent, an entrypoint. Attestation belongs there.Decide the amplification policy on purpose. Distributed engines multiply every per-process operation by their parallelism.Make attestation repeatable and credentials short-lived. Renewal is a requirement, not a follow-up.What makes the result trustworthy is that no single participant issues an identity by itself. Not the workload, not the control plane, not the provider. And the workload, which is the party in the weakest position to be trusted, is never asked to vouch for itself.AcknowledgementsThanks to Amer Hesson for the Data Project abstraction, Nick Siow for the Data Project IAM role sharding, and Doug Clark for Metatron attestation.Trading a Cloud Identity for Your Own: Workload Attestation on Managed Compute was originally published in Netflix TechBlog on Medium, where people are continuing the conversation by highlighting and responding to this story.