BRIEF DELIGHTS

Illustrating Reinforcement Learning from Human Feedback (RLHF)

Analyzes key technical developments and market implications surrounding Illustrating Reinforcement Learning from Human Feedback (RLHF). Provides essential context for enterprise engineering teams.

Amazon Aurora PostgreSQL now supports direct querying of Apache Iceberg and Parquet data in your data lake

Amazon Aurora PostgreSQL now lets you directly query Apache Iceberg and Parquet data stored in your data lake alongside live operational data—no ETL pipelines required. Powered by DuckDB embedded within Aurora, this capability enables single queries that join transactional and historical data using familiar PostgreSQL syntax. It supports AWS Glue Data Catalog, S3, and S3 Tables, with optimizations like predicate pushdown and caching for efficient performance.

Vercel Services: Run full stack on Vercel

To a user, an app with a Next.js frontend and a FastAPI backend feels like one product, and the same should be true for the engineers who build it. Instead, the two pieces are often deployed across different clouds with different development and deployment workflows.Today we're introducing Vercel Services in public beta, which lets you run multiple frameworks in one Vercel Project. This unlocks:Atomic deployments: Your frontend, backend, and other services stay in sync and deploy or roll back togetherShared preview deployments: See how any change affects all your servicesInternal service communication: Services can talk to each other without routing through the public InternetVercel handles the rest: routing, builds, deployments, and auto-scaling in production. The developer experience you already know from Vercel now covers your entire application.Compose applications with Vercel ServicesDeclare your services under the services key in vercel.json keeping the routing configuration explicit:Services can be mounted to a shared routing table without the need for a reverse proxy or CORS.The services configuration is recognized at multiple levels of the Vercel platform:The Deployments panel shows a visualization of services graphThe Logs UI allows filtering by individual serviceThe vercel dev CLI automatically runs all services giving you a production-like environment locallyService bindingsServices can talk to other services internally, without routing through the public Internet. Here's an example how the new bindings configuration key enables that:Now the JavaScript frontend code can talk to the Python service internally via the URL stored in the BACKEND_INTERNAL_URL environment variable:Service-to-service traffic stays on the Vercel network rather than egressing to the public internet. Many independent services can become one application connected by the same wiring, rather than separate deployments that you stitch together across hosts.Framework-defined infrastructureMost frameworks run on Vercel with zero configuration. Framework-defined infrastructure means that each service's framework is auto-detected and auto-provisioned, from FastAPI and Flask for Python to Express and Hono for TypeScript, with first-class support for Go and Rust servers.Services run on Fluid compute, autoscaling with traffic, while you pay only for active CPU time. But framework-defined optimizations go even deeper. With Django, for example, we automatically detect where static assets live and serve them from the CDN.The full stack platformVercel Services gives Vercel a structured way to build and run your application.Those services run on a platform that includes everything any backend could need, whether you're building APIs, agents, or background workers: compute, data, networking, background work, and secure connections to external services.Isolated compute for agent servicesAgents can read files, run commands, and write code. But that capability needs to be secure. Vercel Sandbox gives each agent its own Linux computer: a filesystem, a shell, Docker support, and its own kernel, completely isolated from your deployments. Agents can execute code, grep files, and spin up Redis or Postgres as dependencies. Nothing inside the sandbox can touch your production environment.With automatic persistence, state carries between sessions, and on Pro, sandboxes can run for up to 24 hours.Real-time backends with WebSocket supportVercel Functions handle persistent WebSocket connections in every Vercel runtime, whether your backend is written in Node.js, Python, or Go. They work with standard WebSocket libraries like Socket.IO, and because connections run on Fluid compute with Active CPU pricing, you only pay for the time spent processing messages, not the time a connection sits idle.Reach external services with Vercel ConnectAgents and backends need to reach services outside your project (Slack, GitHub, managed databases), which usually means storing a long-lived secret in your environment. Vercel Connect replaces that with a short-lived credential your app requests at runtime, scoped to the task in front of it. There is no long-lived secret left to leak.Databases and storage for your servicesYou can provision a database from the Vercel Marketplace in a few clicks, with credentials injected for you, from providers like Neon, Supabase, and from AWS with Aurora PostgreSQL, Aurora DSQL, and DynamoDB. Amazon OpenSearch Serverless adds full-text and vector search, and Vercel Blob handles object storage. You can query and manage any of it without leaving Vercel, and both Blob and the AWS integrations authenticate with short-lived OIDC tokens rather than stored secrets.Run durable workflows and background jobsVercel Queues processes background jobs off the request path. Vercel Workflow handles durable multi-step processes that survive crashes and redeploys. And Vercel Cron runs work on a schedule. This is the part of a backend that outlives a request, and it runs on the same platform as everything else, so the long-running side of your backend needs no separate infrastructure.Connect privately and run on the right computeWhen a backend needs to reach a private database or an internal network instead of a public service, Secure Compute, static IPs, and VPC peering open that path.Functions run for up to 30 minutes on Pro and Enterprise, and Python backends deploy with up to 500mb of dependencies.Underneath all of it, Fluid compute is the default for new projects, and Active CPU pricing bills for the time your code is actually running rather than the time it waits, the right shape for idle-heavy backend and AI work.A single platform for everything you shipVercel Services brings your frontend, backend, and supporting services into one project. They build together, preview together, deploy together, and communicate internally by default.Combined with Vercel's built-in primitives for compute, data, queues, workflows, cron, secure networking, and sandboxed agent environments, you can run the full stack without stitching together separate platforms.To start composing services, read the Vercel Services documentation and the routing and communication guide. If you are new to running a backend on Vercel, the zero-config backends post is the place to begin. Read more

Fly Kubernetes does more now

Eons ago, we announced we were working on Fly Kubernetes. It drummed up enough excitement to prove we were heading in the right direction. So, we got hard to work to get from barebones “early access” to a beta release. We’ll be onboarding customers to the closed beta over the next few weeks. Email us at sales@fly.io and we’ll hook you up. Fly Kubernetes is the “blessed path"™️ to using Kubernetes backed by Fly.io infrastructure. Or, in simpler terms, it is our managed Kubernetes service. We take care of the complexity of operating the Kubernetes control plane, leaving you with the unfettered joy of deploying your Kubernetes workloads. If you love Fly.io and K8s, this product is for you. What even is a Kubernete? So how did this all come to be—and what even is a Kubernete? You can see more fun details in Introducing Fly Kubernetes. If you wade through all the YAML and CNCF projects, what’s left is an API for declaring workloads and how it should be accessed.

But that’s not what people usually talk / groan about. It’s everything else that comes along with adopting Kubernetes: a container runtime (CRI), networking between workloads (CNI) which leads to DNS (CoreDNS). Then you layer on Prometheus for metrics and whatever the logging daemon du jour is at the time. Now you get to debate which Ingress—strike that—Gateway API to deploy and if the next thing is anything to do with a Service Mess, then as they like to say where I live, "bless your heart”.

Finally, there’s capacity planning. You’ve got to pick and choose where, how and what the Nodes will look like in order to configure and run the workloads.

When we began thinking about what a Fly Kubernetes Service could look like, we started from first principles, as we do with most everything here. The best way we can describe it is the scene from Iron Man 2 when Tony Stark discovers a new element. As he’s looking at the knowledge left behind by those that came before, he starts to imagine something entirely different and more capable than could have been accomplished previously. That’s what happened to JP, but with K3s and Virtual Kubelet. OK then, WTF (what’s the FKS)? We looked at what people need to get started—the API—and then started peeling away all the noise, filling in the gaps to connect things together to provide the power. Here’s how this looks currently:

Containerd/CRI → flyd + Firecracker + our init: our system transmogrifies Docker containers into Firecracker microVMs Networking/CNI → Our internal WireGuard mesh connects your pods together Pods → Fly Machines VMs Secrets → Secrets, only not the base64’d kind Services → The Fly Proxy CoreDNS → CoreDNS (to be replaced with our custom internal DNS) Persistent Volumes → Fly Volumes (coming soon)

Now…not everything is a one-to-one comparison, and we explicitly did not set out to support any and every configuration. We aren’t dealing with resources like Network Policy and init containers, though we’re also not completely ignoring them. By mapping many of the core primitives of Kubernetes to a Fly.io resource, we’re able to focus on continuing to build the primitives that make our cloud better for workloads of all shapes and sizes.

A key thing to notice above is that there’s no “Node”.

Virtual Kubelet plays a central role in FKS. It’s magic, really. A Virtual Kubelet acts as if it’s a standard Kubelet running on a Node, eager to run your workloads. However, there’s no Node backing it. It instead behaves like an API, receiving requests from Kubernetes and transforming them into requests to deploy on a cloud compute service. In our case, that’s Fly Machines.

So what we have is Kubernetes calling out to our Virtual Kubelet provider, a small Golang program we run alongside K3s, to create and run your pod. It creates your pod as a Fly Machine, via the Fly Machines API, deploying it to any underlying host within that region. This shifts the burden of managing hardware capacity from you to us. We think that’s a cool trick—thanks, Virtual Kubelet magic! Speedrun You can deploy your workloads (including GPUs) across any of our available regions using the Kubernetes API.

You create a cluster with flyctl:

Wrap text

Copy to clipboard fly ext k8s create --name hello --org personal --region iad When a cluster is created, it has the standard default namespace. You can inspect it:

Wrap text

Copy to clipboard kubectl get ns default --show-labels

Wrap text

Copy to clipboard NAME STATUS AGE LABELS default Active 20d fly.io/app=fks-default-7zyjm3ovpdxmd0ep,kubernetes.io/metadata.name=default The fly.io/app label shows the name of the Fly App that corresponds to your cluster.

It would seem appropriate to deploy the Kubernetes Up And Running demo here, but since your pods are connected over an IPv6 WireGuard mesh, we’re going to use a fork with support for IPv6 DNS.

Wrap text

Copy to clipboard kubectl run \ --image=ghcr.io/jipperinbham/kuard-amd64:blue \ --labels="app=kuard-fks" \ kuard And you can see its Machine representation via:

Wrap text

Copy to clipboard fly machine list --app fks-default-7zyjm3ovpdxmd0ep

Wrap text

Copy to clipboard ID NAME STATE REGION IMAGE IP ADDRESS VOLUME CREATED LAST UPDATED APP PLATFORM PROCESS GROUP SIZE 1852291c46ded8 kuard started iad jipperinbham/kuard-amd64:blue fdaa:0:48c8:a7b:228:4b6d:6e20:2 2024-03-05T18:54:41Z 2024-03-05T18:54:44Z shared-cpu-1x:256MB

This is important! Your pod is a Fly Machine! While we don’t yet support all kubectl features, Fly.io tooling will “just work” for cases where we don’t yet support the kubectl way. So, for example, we don’t have kubectl port-forward and kubectl exec, but you can use flyctl to forward ports and get a shell into a pod.

Expose it to your internal network using the standard ClusterIP Service:

Wrap text

Copy to clipboard kubectl expose pod kuard \ --name=kuard \ --port=8080 \ --target-port=8080 \ --selector='app=kuard-fks' ClusterIP Services work natively, and Fly.io internal DNS supports them. Within the cluster, CoreDNS works too.

Access this Service locally via flycast: Get connected to your org’s 6PN private WireGuard network. Get kubectl to describe the kuard Service:

Wrap text

Copy to clipboard kubectl describe svc kuard

Wrap text

Copy to clipboard Name: kuard Namespace: default Labels: app=kuard-fks Annotations: fly.io/clusterip-allocator: configured service.fly.io/sync-version: 11507529969321451315 Selector: app=kuard-fks Type: ClusterIP IP Family Policy: SingleStack IP Families: IPv6 IP: fdaa:0:48c8:0:1::1a IPs: fdaa:0:48c8:0:1::1a Port: 8080/TCP TargetPort: 8080/TCP Endpoints: [fdaa:0:48c8:a7b:228:4b6d:6e20:2]:8080 Session Affinity: None Events: You can pull out the Service’s IP address from the above output, and get at the KUARD UI using that: in this case, http://[fdaa:0:48c8:0:1::1a]:8080.

Using internal DNS: http://.svc..flycast:8080. Or, in our example: http://kuard.svc.fks-default-7zyjm3ovpdxmd0ep.flycast:8080.

And finally CoreDNS: ..svc.cluster.local resolves to the fdaa IP and is routable within the cluster.

Pricing The Fly Kubernetes Service is free during the beta. Fly Machines and Fly Volumes you create with it will cost the same as for your other Fly.io projects. It’ll be $75/mo per cluster after that, plus the cost of the other resources you create. Today and the future Today, Fly Kubernetes supports only a portion of the Kubernetes API. You can deploy pods using Deployments/ReplicaSets. Pods are able to communicate via Services using the standard K8s DNS format. Ephemeral and persistent volumes are supported.

The most notable absences are: multi-container pods, StatefulSets, network policies, horizontal pod autoscaling and emptyDir volumes. We’re working at supporting autoscaling and emptyDir volumes in the coming weeks and multi-container pods in the coming months.

If you’ve made it this far and are eagerly awaiting your chance to tell us and the rest of the internet “this isn’t Kubernetes!”, well, we agree! It’s not something we take lightly. We’re still building, and conformance tests may be in the future for FKS. We’ve made a deliberate decision to only care about fast launching VMs as the one and only way to run workloads on our cloud. And we also know enough of our customers would like to use the Kubernetes API to create a fast launching VM in the form of a Pod, and that’s where this story begins.

Vercel Services: Run full stack on Vercel

To a user, an app with a Next.js frontend and a FastAPI backend feels like one product, and the same should be true for the engineers who build it. Instead, the two pieces are often deployed across different clouds with different development and deployment workflows.Today we're introducing Vercel Services in public beta, which lets you run multiple frameworks in one Vercel Project. This unlocks:Atomic deployments: Your frontend, backend, and other services stay in sync and deploy or roll back togetherShared preview deployments: See how any change affects all your servicesInternal service communication: Services can talk to each other without routing through the public InternetVercel handles the rest: routing, builds, deployments, and auto-scaling in production. The developer experience you already know from Vercel now covers your entire application.Compose applications with Vercel ServicesDeclare your services under the services key in vercel.json keeping the routing configuration explicit:Services can be mounted to a shared routing table without the need for a reverse proxy or CORS.The services configuration is recognized at multiple levels of the Vercel platform:The Deployments panel shows a visualization of services graphThe Logs UI allows filtering by individual serviceThe vercel dev CLI automatically runs all services giving you a production-like environment locallyService bindingsServices can talk to other services internally, without routing through the public Internet. Here's an example how the new bindings configuration key enables that:Now the JavaScript frontend code can talk to the Python service internally via the URL stored in the BACKEND_INTERNAL_URL environment variable:Service-to-service traffic stays on the Vercel network rather than egressing to the public internet. Many independent services can become one application connected by the same wiring, rather than separate deployments that you stitch together across hosts.Framework-defined infrastructureMost frameworks run on Vercel with zero configuration. Framework-defined infrastructure means that each service's framework is auto-detected and auto-provisioned, from FastAPI and Flask for Python to Express and Hono for TypeScript, with first-class support for Go and Rust servers.Services run on Fluid compute, autoscaling with traffic, while you pay only for active CPU time. But framework-defined optimizations go even deeper. With Django, for example, we automatically detect where static assets live and serve them from the CDN.The full stack platformVercel Services gives Vercel a structured way to build and run your application.Those services run on a platform that includes everything any backend could need, whether you're building APIs, agents, or background workers: compute, data, networking, background work, and secure connections to external services.Isolated compute for agent servicesAgents can read files, run commands, and write code. But that capability needs to be secure. Vercel Sandbox gives each agent its own Linux computer: a filesystem, a shell, Docker support, and its own kernel, completely isolated from your deployments. Agents can execute code, grep files, and spin up Redis or Postgres as dependencies. Nothing inside the sandbox can touch your production environment.With automatic persistence, state carries between sessions, and on Pro, sandboxes can run for up to 24 hours.Real-time backends with WebSocket supportVercel Functions handle persistent WebSocket connections in every Vercel runtime, whether your backend is written in Node.js, Python, or Go. They work with standard WebSocket libraries like Socket.IO, and because connections run on Fluid compute with Active CPU pricing, you only pay for the time spent processing messages, not the time a connection sits idle.Reach external services with Vercel ConnectAgents and backends need to reach services outside your project (Slack, GitHub, managed databases), which usually means storing a long-lived secret in your environment. Vercel Connect replaces that with a short-lived credential your app requests at runtime, scoped to the task in front of it. There is no long-lived secret left to leak.Databases and storage for your servicesYou can provision a database from the Vercel Marketplace in a few clicks, with credentials injected for you, from providers like Neon, Supabase, and from AWS with Aurora PostgreSQL, Aurora DSQL, and DynamoDB. Amazon OpenSearch Serverless adds full-text and vector search, and Vercel Blob handles object storage. You can query and manage any of it without leaving Vercel, and both Blob and the AWS integrations authenticate with short-lived OIDC tokens rather than stored secrets.Run durable workflows and background jobsVercel Queues processes background jobs off the request path. Vercel Workflow handles durable multi-step processes that survive crashes and redeploys. And Vercel Cron runs work on a schedule. This is the part of a backend that outlives a request, and it runs on the same platform as everything else, so the long-running side of your backend needs no separate infrastructure.Connect privately and run on the right computeWhen a backend needs to reach a private database or an internal network instead of a public service, Secure Compute, static IPs, and VPC peering open that path.Functions run for up to 30 minutes on Pro and Enterprise, and Python backends deploy with up to 500mb of dependencies.Underneath all of it, Fluid compute is the default for new projects, and Active CPU pricing bills for the time your code is actually running rather than the time it waits, the right shape for idle-heavy backend and AI work.A single platform for everything you shipVercel Services brings your frontend, backend, and supporting services into one project. They build together, preview together, deploy together, and communicate internally by default.Combined with Vercel's built-in primitives for compute, data, queues, workflows, cron, secure networking, and sandboxed agent environments, you can run the full stack without stitching together separate platforms.To start composing services, read the Vercel Services documentation and the routing and communication guide. If you are new to running a backend on Vercel, the zero-config backends post is the place to begin. Read more