Tej Shah

Software engineer at Amazon Web Services in New York.

Interested in distributed systems, applied research, and data.

Figure 0. When the metric stops measuring what matters. A question behind my work on distributed systems and my interest in reinforcement learning: are we improving the system, or just its score? The proxy reward (red) keeps climbing while the outcome we actually want (black) eventually falls. The shaded gap is reward hacking; the dotted line marks where a careful run would have stopped.

01Explainers

Field guideSix parts · 29 figures

RL environments: a field guide

How to build, scale, verify, train on and measure RL environments, where each rollout's information leaks on its way into the weights, and the measured cost of every leak.

Read the explainer

BusinessSix parts · 24 figures

RL environments as a business

Who pays for RL environments and evals, where the number they paid for stops meaning what it says, and why nearly every business here ends up selling a measurement.

Read the explainer

02Writing

Coming soon.

03Work

Amazon Web Services
2024 to present
Seattle and New York
Software Development Engineer, Billing and Cost Management
Rutgers University
2021 to 2024
New Brunswick
Research Engineer, Assistant and Intern
Kleiner Perkins Fellows
2022
San Francisco
Product Fellow, Product Manager at Interos

All work