Episodi

  • Building the Network for AI Agents — Yan Avlasov, Google

    Building the Network for AI Agents — Yan Avlasov, Google
    Sep 15 2026

    AI agents bring new demands to the infrastructure connecting them to models, tools, and data. Each agent can have its own permissions and state, and spend much of its time asleep before waking up to make another request. Supporting that behavior at scale changes how the network routes traffic, loads policies, and manages resources.

    In this episode of Alexa’s Input (AI), I sit down with Yan Avlasov, Staff Software Engineer at Google and a senior maintainer of Envoy, to talk about building the networking stack for agents.

    We get into why Google is building on Envoy, what changes between serving inference and supporting agents, and why enforcing policies for individual agents requires a more dynamic control plane. Yan also shares what he’s seeing from AI-powered security scanning, including how models combine subtle bugs into serious vulnerabilities and why fixing them can mean changing behavior that production systems have relied on for years.

    From the episode:

    - Model selection, cost controls, and fallback across providers
    - Loading policies dynamically as agent traffic changes
    - Managing personalized state and agents that suspend and resume
    - Where Model Context Protocol (MCP) fits alongside the protocols agents already use
    - Semantic routing and choosing models based on what a request needs
    - Security findings involving policy enforcement, path normalization, and JSON parsing
    - The engineering work and cost of making security scanning continuous

    Chapters
    00:00 Introduction
    01:12 Welcome, Yan
    02:30 How AI is changing Google
    07:56 Yan’s networking background
    10:00 New networking layers for agents
    13:11 AI gateways, cost controls, and fallback
    16:38 Why Google builds on Envoy
    22:21 Per-agent policies and a dynamic control plane
    26:56 Inference versus stateful agents
    31:58 Project Substrate: running agents at scale
    33:31 MCP and the other protocols agents use
    37:32 Semantic routing and model costs
    40:40 AI security scanning and the work of fixing bugs
    43:55 Policy timing, path normalization, and JSON
    47:11 Making security scanning continuous
    50:40 What’s next for agentic networking
    53:32 Closing

    Read the episode article on Substack

    Watch on YouTube
    Mostra di più Mostra meno
    54 min
  • Personal Security with Alex Zenla, Founder and CTO of Edera

    Personal Security with Alex Zenla, Founder and CTO of Edera
    Jul 27 2026

    In this episode of Alexa's Input (AI), I sit down with Alex Zenla, founder and CTO of Edera.


    Alex grew up in a small town in Alabama, found a computer young, and started building. Her story is unlike many in tech. She taught herself to program and got a job in tech at 14 years old. Since then, she's been actively building and involved in open source. She's currently the founder and CTO of Edera, a company whose product integrates security into the lowest layers of the platform without sacrificing performance or velocity.


    In this episode, we get into where that path started, what it costs to be different in founder and venture rooms, and what breaks when infrastructure still ships with security off by default.


    From the episode:

    • Growing up in small-town Alabama without a path into tech
    • Southern niceness as theory versus practice
    • Full-time work at fourteen and presenting to executives as a teenager
    • Being one of very few trans founders in venture rooms, and the tension between visibility and being treated as a token
    • Elevator pitches that change with the audience
    • Detection and response after a problem has already occurred
    • Common Vulnerabilities and Exposures becoming untenable when tools like Mythos surface hundreds of findings per project per day
    • Kubernetes and vendors selling yet another layer while the foundations underneath are misaligned
    • Secure defaults as the path of least resistance for teams that just need a cluster that works


    Alex's mission is to make secure computing the default. Today you work hard to get a secure environment, and she's building Edera to invert that. What stays with you is how personal that work is for her. The path from a small Alabama town into those rooms is not separate from the product. It's why the default being broken bothers her enough to build a company around fixing it.


    GENERAL PODCAST LINKS
    Watch: https://www.youtube.com/@alexasinput
    Read: https://alexasinput.substack.com/
    Listen: https://creators.spotify.com/pod/profile/alexagriffith/
    More: https://linktr.ee/alexagriffith


    LEARN MORE ABOUT THE HOST
    Website: https://alexagriffith.com/
    LinkedIn: https://www.linkedin.com/in/alexa-griffith/


    FIND OUT MORE ABOUT THE GUEST
    LinkedIn: https://www.linkedin.com/in/azenla/
    Bluesky: https://bsky.app/profile/alex.zenla.io
    Edera: https://edera.dev/
    GitHub: https://github.com/edera-dev


    RESOURCES
    Edera docs: https://docs.edera.dev/
    Mostra di più Mostra meno
    51 min
  • Paying Attention in the Age of Agents

    Paying Attention in the Age of Agents
    Jul 20 2026

    AI agents have created more possibilities for engineers than ever before. But what does daily life actually look like for the builders who've gone all in?

    In my very first panel episode, I sit down with Adam Anzuoni from Cursor, Taylor Dolezal from Dosu, and Peter Bell from Gather.dev. Three builders running agents every day for real work, not demos. Peter runs nine API plans across three Mac Minis, with built-in adversarial review. Adam manages cloud agents from his phone. Taylor is building the context infrastructure that makes agent knowledge portable across teams.

    We get into deterministic pipelines, skill systems that become their own technical debt, and what all three kept coming back to: attention is now the bottleneck. When agents can do everything, deciding what deserves your focus is the actual hard problem.

    Three setups. One shared constraint. A conversation worth hearing.

    Topics discussed:

    • Attention as the real bottleneck when agents can do everything
    • Deterministic pipelines vs. agentic orchestration — when to use scripts and when to use agents
    • Peter's system: nine API plans, three Mac Minis, adversarial review, self-improving context
    • Adam on Cursor cloud agents and managing builds from his phone
    • Taylor on context distribution — making accumulated knowledge available to ephemeral agents
    • The skill maintenance problem and why agent systems become their own technical debt
    • The ADHD-like productivity loop that agent-driven work creates
    • Plans matter more than prompts — all three panelists converged on this
    • Intermediate artifacts as the key to quality output
    • Product mindset as the engineer's next high-value skill
    • Sandboxes, governance, and why "approve, approve, approve" puts your hard drive at risk

    General podcast links

    Watch: https://www.youtube.com/@alexasinput
    Read: https://alexasinput.substack.com/
    Listen: https://creators.spotify.com/pod/profile/alexagriffith/
    More: https://linktr.ee/alexagriffith

    Learn more about the host

    Website: https://alexagriffith.com/
    LinkedIn: https://www.linkedin.com/in/alexa-griffith/
    X: https://x.com/alexa_griffith_

    Find out more about the guests

    Adam Anzuoni
    LinkedIn: https://www.linkedin.com/in/adamanz/
    Website: https://www.adamanzuoni.com/
    Cursor: https://www.cursor.com/

    Taylor Dolezal
    LinkedIn: https://www.linkedin.com/in/onlydole/
    Website: https://onlydole.dev/
    Dosu: https://dosu.dev/

    Peter Bell
    LinkedIn: https://www.linkedin.com/in/peterfbell/
    Gather.dev: https://gather.dev/
    O'Reilly Book: Scaling AI Adoption in Engineering

    Resources mentioned in this episode

    Cursor: https://www.cursor.com/
    Dosu: https://dosu.dev/
    Gather.dev: https://gather.dev/
    Anthropic Claude: https://www.anthropic.com/
    The Phoenix Project (book reference by Taylor)
    Kelsey Hightower productivity survey (referenced by Taylor)
    Mostra di più Mostra meno
    1 ora e 4 min
  • Systems, Scale, and SRE with Vlad Leyberov

    Systems, Scale, and SRE with Vlad Leyberov
    Jun 29 2026

    Most engineers think reliability means avoiding outages. Vlad Leyberov learned the opposite lesson: sometimes you have to intentionally cause a 100% outage to fix the system faster.

    Vlad is a Site Reliability Engineer (SRE) at Google, running systems that handle billions of requests per second. Before Google, he kept critical infrastructure running at Meta (billions of events a day) and Amazon (millions of Alexa devices).

    In this conversation, we dig into cascading failures, incident responses, why consistency beats speed, how AI changes reliability engineering, and the philosophy behind running systems where downtime doesn't feel like an option.

    Topics Discussed:

    • How cascading failures propagate unpredictably in distributed systems (like nature, not machines)
    • Incident responses: virtual panic rooms, on-call, paging procedures, and how to narrow down failure points
    • The Alexa incident: why dropping an entire DynamoDB table was the right call
    • Critical User Journeys (CUJ): measuring end-to-end customer experience vs individual SLOs
    • Career journey from the USSR to maritime academy to business degree in Australia to SRE at Amazon, Meta, and Google
    • Why consistency in API response times beats raw speed
    • How AI makes it dangerously easy to create complex systems with poorly understood interactions
    • Science fiction, the Borg as a distributed system, and the Three Body Problem trilogy
    • Hot takes on reliability: all software development is maintenance, overrated 9s, underrated global failure modes


    General Podcast Links

    Watch: https://www.youtube.com/@alexasinput

    Read: https://alexasinput.substack.com/

    Listen: https://creators.spotify.com/pod/profile/alexagriffith/ More: https://linktr.ee/alexagriffith


    Learn more about the host

    Website: https://alexagriffith.com/

    LinkedIn: https://www.linkedin.com/in/alexa-griffith/


    Find out more about Vlad Leyberov

    LinkedIn: https://www.linkedin.com/in/vladleyberov/ Google SRE NYC Tech Talks

    Resources

    Google SRE Resources:

    • Google SRE Book: https://sre.google/books/
    • Google Cloud Platform: https://cloud.google.com/
    • Google Cloud Build: https://cloud.google.com/build (service discussed in outage story)
    • Google Cloud Pub/Sub: https://cloud.google.com/pubsub (Vlad's previous role, billions of requests/second)

    Sci-Fi Books Mentioned:

    • Three Body Problem trilogy by Liu Cixin (Vlad's current favorite)
    • Foundation series by Isaac Asimov
    • Left Hand of Darkness by Ursula K. Le Guin
    • Snow Crash by Neal Stephenson

    Internal Google Systems Referenced:

    • Borg: Google's internal cluster management system (Kubernetes predecessor), named after Star Trek Borg
    • DynamoDB: AWS distributed key-value store (used in Alexa poison pill incident)


    Intro Music:PR1BVOV7R4F1ASZC
    Mostra di più Mostra meno
    58 min
  • David Aronchick on Distributed Data Orchestration with Expanso

    David Aronchick on Distributed Data Orchestration with Expanso
    Jun 15 2026
    In this episode of Alexa's Input (AI), I sit down with David Aronchick, co-founder and CEO of Expanso and former product lead for Kubernetes at Google.Data is growing everywhere outside your data center. Solar panels in remote across a country. Security cameras at retail stores. IoT sensors across factory floors. And moving that data to the cloud for processing? It's expensive, slow, and often restricted by compliance.David is an expert when it comes to solving distribution problems. He led Kubernetes product at Google, co-founded Kubeflow to bring ML to production, and now he's building Expanso to tackle a difficult constraint: when your data can't move, how do you process it where it lives?We discuss:- The need for distributed data orchestration-Upstream data control: filtering and transforming at the source- Three forces making edge computing inevitable (physics, regulations, economics)- How to build successful open source infrastructure projects- Customer discovery and finding real pain points- His transition from Protocol Labs to founding Expanso- ETL pipelines: moving the first four steps closer to the data- Context loss and lineage in distributed systems- Processing 400,000 signals per second with 150MB agents- AI observability: attaching source metadata to training data- Running ML pipelines at the edge- Real-world deployment challenges (bandwidth, regulations, cost)Expanso is rethinking how we process data in an AI-native world—moving compute to data instead of data to compute. If you want to understand where distributed systems and edge computing are heading, this is a deep dive into the infrastructure layer beneath modern AI applications.General Podcast LinksWatch: https://www.youtube.com/@alexasinput Read: https://alexasinput.substack.com/ Listen: https://creators.spotify.com/pod/profile/alexagriffith/ More: https://linktr.ee/alexagriffithLearn more about the host atWebsite: https://alexagriffith.com/ LinkedIn: https://www.linkedin.com/in/alexa-griffith/Find out more about the guest atLinkedIn: https://www.linkedin.com/in/aronchick/ Twitter/X: https://x.com/aronchick GitHub: https://github.com/aronchick Expanso Website: https://expanso.io/ResourcesExpanso Website: https://expanso.io/ Kubernetes: https://kubernetes.io/ Kubeflow: https://www.kubeflow.org/ CNCF (Cloud Native Computing Foundation): https://www.cncf.io/ Protocol Labs: https://protocol.ai/KeywordsDavid Aronchick, Expanso, Kubernetes, Kubeflow, distributed systems, edge computing, data pipelines, ETL, upstream data control, Google Kubernetes Engine, open source, CNCF, observability, log processing, data lineage, provenance, schema enforcement, IoT, edge AI, distributed data, machine learning infrastructure, Protocol Labs, IPFS, Filecoin, data governance, compliance, GDPR, bandwidth optimization, data aggregation, AI infrastructure, multi-cloud, hybrid cloud, real-time processing
    Mostra di più Mostra meno
    1 ora e 18 min
  • How vLLM and llm-d Changed AI Inference with Rob Shaw

    How vLLM and llm-d Changed AI Inference with Rob Shaw
    Jun 3 2026
    In this episode of Alexa’s Input (AI), I sat down with Rob Shaw from Red Hat to talk about how AI inference evolved from a simple model serving problem into a large-scale distributed systems problem.We explored the infrastructure shifts behind modern LLM serving, including how vLLM and PagedAttention changed the economics and efficiency of inference, why KV cache management became one of the most important bottlenecks in production AI systems, and how orchestration layers like llm-d are emerging to coordinate distributed inference.We also discuss:how LLM inference differs from traditional model serving runtimesKV cache, prefix caching, and cache-aware routingwhy throughput and latency became major infrastructure challengeslong-context agents and repeated inference callsdistributed inference on Kubernetesintelligent routing, flow control, and load balancingprefill/decode disaggregationenterprise AI deployment realitiesvLLM has become one of the most important open-source projects in AI infrastructure, and llm-d represents a newer shift toward treating inference as a coordinated distributed system rather than just a single runtime problem.If you want to better understand the systems layer beneath modern AI applications, this episode is a deep dive into where inference infrastructure is heading next.General Podcast LinksWatch: ⁠⁠⁠⁠⁠⁠https://www.youtube.com/@alexasinput⁠⁠⁠⁠⁠⁠Read: ⁠⁠⁠⁠⁠⁠⁠⁠https://alexasinput.substack.com/⁠⁠⁠⁠⁠⁠⁠⁠Listen:⁠⁠ ⁠⁠https://creators.spotify.com/pod/profile/alexagriffith/⁠⁠⁠⁠More: ⁠⁠⁠⁠⁠⁠https://linktr.ee/alexagriffith⁠⁠⁠⁠⁠⁠Learn more about the host atWebsite: ⁠⁠⁠⁠⁠⁠https://alexagriffith.com/⁠⁠⁠⁠⁠⁠LinkedIn: ⁠⁠⁠⁠⁠⁠https://www.linkedin.com/in/alexa-griffith/⁠⁠⁠⁠⁠⁠Find out more about the guest at:LinkedIn: https://www.linkedin.com/in/robert-shaw-1a01399a/ Red Hat Articles: https://developers.redhat.com/author/robert-shawGithub: https://github.com/robertgshaw2-redhat ResourcesvLLM Website: https://vllm.ai/vLLM GitHub Repository: https://github.com/vllm-project/vllmllm-d Website: https://llm-d.ai/llm-d GitHub Repository - https://github.com/llm-d/llm-d KeywordsAI inference, VLLM, LMD, distributed inference, GPU optimization, open source AI, Kubernetes, multi-cluster deployment, AI infrastructure, enterprise AI AI infrastructure, Kubernetes, model optimization, speculative decoding, mixture of experts, AI deployment, performance tuning, AI systems, neural network scaling Key TopicsEvolution of vLLM and llm-dDistributed inference and routingGPU utilization and performance optimizationOpen source AI infrastructureEnterprise deployment challenges and solutions Standardization in Kubernetes for NIC exposurePerformance optimizations: quantization and speculative decodingMixture of experts architecture and parallelism strategiesFlow control and request scheduling in AI systemsEmerging hardware for AI inference, Cerebras processorReinforcement learning and AI system supportModular architecture of vLLM and ecosystem projects
    Mostra di più Mostra meno
    1 ora e 43 min
  • Intelligence Per Watt with Emilio Andere

    Intelligence Per Watt with Emilio Andere
    May 24 2026

    On this episode of Alexa’s Input (AI), I sit down with Emilio Andere, co-founder and CEO of Wafer, to talk about the future of AI infrastructure, inference optimization, and the economics driving the AI compute race.

    We discuss:

    • why “intelligence per watt” may become one of the defining metrics of the AI era
    • the current GPU and accelerator landscape across NVIDIA, AMD, TPUs, and emerging hardware startups
    • why software optimization is becoming just as important as hardware itself
    • inference optimization strategies
    • why AI infrastructure companies are racing up the stack
    • what it’s actually like building an AI infrastructure startup today

    and more!

    Emilio also shares lessons from founding Wafer, thoughts on the future of open-source AI infrastructure, and why he believes optimizing intelligence itself could become one of the most important engineering problems.


    General Podcast Links

    Watch: ⁠⁠⁠⁠⁠⁠https://www.youtube.com/@alexasinput⁠⁠⁠⁠⁠⁠

    Read: ⁠⁠⁠⁠⁠⁠⁠⁠https://alexasinput.substack.com/⁠⁠⁠⁠⁠⁠⁠⁠

    Listen:⁠⁠ ⁠⁠https://creators.spotify.com/pod/profile/alexagriffith/⁠⁠⁠⁠

    More: ⁠⁠⁠⁠⁠⁠https://linktr.ee/alexagriffith⁠⁠⁠⁠⁠⁠


    Learn more about the host at

    Website: ⁠⁠⁠⁠⁠⁠https://alexagriffith.com/⁠⁠⁠⁠⁠⁠

    LinkedIn: ⁠⁠⁠⁠⁠⁠https://www.linkedin.com/in/alexa-griffith/⁠⁠⁠⁠⁠⁠


    Find out more about the guest at:

    LinkedIn: https://www.linkedin.com/in/emi-andere/

    Wafer Website: https://www.wafer.ai/

    Wafer AI / Y Combinator Article: https://www.ycombinator.com/companies/wafer


    Chapters

    00:00 Exploring AI Conversations and Recent Podcasts

    02:14 Intelligence per Watt: A New Metric for AI

    07:35 The Manifesto: Efficiency in Civilization

    12:40 Founding Wafer: The Journey Begins

    18:08 The GPU Hardware Landscape and Market Dynamics

    23:07 AMD's Growing Presence in the GPU Market

    24:07 Emerging Competitors in the AI Hardware Space

    26:04 Comparing TPUs and GPUs

    27:21 Acquisition and Availability of TPUs

    28:33 Navigating the GPU Marketplace

    30:05 Understanding Neo Cloud Economics

    33:30 The AI Bubble Debate

    36:25 Optimizing AI Models for Performance

    44:46 Bottlenecks in AI Model Performance

    48:08 Future Directions in AI Hardware Optimization

    54:39 Balancing Speed and Cost in AI Performance

    56:54 Kernel Arena: Benchmarking AI Performance

    01:03:45 Lessons from Founding: Sales and Emotional Resilience

    01:07:38 The Future of AI: Trends and Predictions

    01:13:03 Outro


    Keywords

    AI hardware, inference optimization, intelligence per watt, GPU market, AI infrastructure, Wafer, AI bubble, TPU, GPU bottleneck, AI efficiency AI optimization, large language models, AI hardware, quantization, speculative decoding, benchmarking, AI infrastructure, model training, AI startups
    Mostra di più Mostra meno
    1 ora e 14 min
  • Building Reliable Systems at Bloomberg with Sal Furino

    Building Reliable Systems at Bloomberg with Sal Furino
    May 17 2026
    In this episode of Alexa’s Input (AI), I sit down with Sal Furino to explore the hidden engineering work that keeps modern systems reliable.We break down what Service Level Objectives, Indicators (SLOs/SLIs), and error budgets actually mean in practice, why reliability is as much a cultural problem as a technical one, and how teams can better measure real user experience instead of just infrastructure health.Sal also explains reliability engineering and the challenges of reliability at scale, like:Why latency and correctness become harder to measure with GenAIThe difference between a bad incident and a fundamentally bad systemHow observability and telemetry shape modern engineering organizationsWhy most teams focus too much on infrastructure metrics and not enough on user happiness Why “the best systems are the ones nobody notices.”If you work in AI infrastructure, distributed systems, platform engineering, observability, or SRE, this episode is a must listen!SRECon Talk Dashboards & Dragons: Reliability Magic for AI Platforms by Alexa Griffith and Sal Furino: https://youtu.be/aWMB_7ksbkc?si=S49nPyAl_hCUIH7yGeneral Podcast LinksWatch: ⁠⁠⁠⁠⁠https://www.youtube.com/@alexasinput⁠⁠⁠⁠⁠Read: ⁠⁠⁠⁠⁠⁠⁠https://alexasinput.substack.com/⁠⁠⁠⁠⁠⁠⁠Listen:⁠⁠ ⁠https://creators.spotify.com/pod/profile/alexagriffith/⁠⁠⁠More: ⁠⁠⁠⁠⁠https://linktr.ee/alexagriffith⁠⁠⁠⁠⁠Learn more about the host atWebsite: ⁠⁠⁠⁠⁠https://alexagriffith.com/⁠⁠⁠⁠⁠LinkedIn: ⁠⁠⁠⁠⁠https://www.linkedin.com/in/alexa-griffith/⁠⁠⁠⁠⁠Find out more about the guest at:LinkedIn: https://www.linkedin.com/in/salvatore-furino/Rootly Interview: https://rootly.com/humans-of-reliability/salvatore-furinoReliability at Scale Talk: https://youtu.be/J-VrU5JHPlk?si=8aV8acy57NWX30KABloomberg Careers: https://bloomberg.avature.net/careers/SearchJobsChapters00:00 - Introduction: Reliability in a world reshaped by generative AI02:22 - The importance of seamless, background system design04:41 - Becoming a Customer Reliability Engineer at Bloomberg05:17 - Clarifying the CRE role and its customer focus08:02 - The importance of observability and high-scale performance in finance09:00 - Balancing technical and cultural aspects of reliability10:19 - Coaching teams to be proactive using error budgets and SLIs12:21 - The social-technical system: People, processes, and tools13:06 - Mediation of differing opinions on reliability practices15:06 - The nuanced approach to alerting and incident response17:08 - The significance of tiered SLOs and the concept of error budgets21:08 - Using signals like latency, correctness, availability, saturation in system measurement22:53 - The impact of service level "nines" on system design and resilience28:00 - Handling non-determinism and trust in AI responses33:01 - Error budgets and their role in managing deployments34:10 - The challenge of achieving five nines and data durability considerations40:03 - Adapting SLOs for GenAI systems: core principles remain intact42:23 - Measuring non-deterministic AI responses and quality proxies44:41 - The ongoing importance of reliability even in AI/ML contexts47:25 - Reacting to error budget exhaustion and proactive mitigation50:42 - The significance of involving cross-functional teams during outages55:36 - Advocating reliability investment to leadership56:24 - The customer perspective: reliability as a fundamental feature58:42 - Connecting with Sal Furino: where to follow his work and learn more about Bloomberg's engineering culture59:20 - Final advice: Focus on user happiness to avoid common pitfalls in adopting SLOs
    Mostra di più Mostra meno
    54 min