Henrique Schechter Vera

I’ve been interviewed and interviewing people for 10 years for DS and MLE roles across Apple TikTok Uber Meta, etc. This is why it’s broken. 8 years ago: you wanted a “DS” role, you grind leetcode for a few months, read up on ML theory, get flown out to Google in Zurich to whiteboard a SWE problem typically leetcode medium, plenty of collaboration with the tech interviewer. You found out if you got the role by the time your plane lands back home. Today: you’re applicant 30/100 , you got selected from 10,000 applications by ATS, 95% of which were submitted by job hunt farms. One person submits 10-20 different CVs under different emails, A-B testing which one lands them in the cutoff threshold. You have 7 rounds, the majority of which will be leetcode medium/hard problems. The tech recruiter can’t be bothered, they want to get off the call quickly and are praying you just use AI to answer the problem so they can get back to debugging. The majority of people will use AI to cheat. I’ve seen them do it when their mirror stayed up behind them and saw how they use virtual environments to boot up blackbox.ai. I’ve seen one wearing with one AirPod longer than the other. If you ask questions whilst coding it demonstrates a lack of confidence, especially since the 5 people before you just vibe coded the solution in 5 minutes relative to your 20. You wasted 15 minutes of tech bros time, you’re not getting through. Recruiters treat job roles like sales trying to close, your 20 minutes is seen as a defective, you aren’t selling well, they will drop you, you’re a number to them before you’re a human. That’s why they’ll ignore your requests for feedback. I’ve literally seen recruiter teams laughing with one another over a senior dev’s frantic emails asking what he did wrong when he got rejected in round 5 of a 7 round process after writing a take home 2 times in a row. The game isn’t broken, everyone’s just on steroids now in an unregulated competition

152

15 Comments

Get faster and smarter MOE inference straight out of the box. 👇Deep dive on scaling expert parallelism with TensorRT-LLM. LLMs with MOE promise higher model capacity without linearly increasing compute costs - but they introduce new challenges -- more conditional computation, dynamic routing, and non-uniform GPU utilization -- solved by TensorRT-LLM. ✨ New in TensorRT-LLM has native support for expert parallelism—designed for fast, efficient inference with MoE models like Mixtral (Mistral AI) and DeepSeek (DeepSeek AI). This gives you: ✅ Dynamic expert routing: Automatically route tokens to the top-k experts with minimal overhead. ✅ Efficient expert scheduling: Balance expert loads across GPUs using smart sharding and token bucketization. ✅ Memory-aware execution: Maximize hardware utilization while respecting memory budgets. ✅ Drop-in support: Use @HuggingFace models with minimal code changes via TensorRT-LLM's #Python API. 🧠 How it works: MoE models activate only a subset of "experts" for each token. This dynamic nature is powerful—but hard to optimize. It’s all done under the hood using custom #CUDA kernels and NCCL-based communication primitives—giving you low latency, high throughput, and better GPU scaling. ✨ TensorRT-LLM handles: ✅Token-expert mapping using the gating network. ✅Token sorting to batch same-expert tokens together. ✅Expert parallel execution across GPUs. ✅Merging outputs for final predictions. 🛠️ Developer workflow - here is the code to get started. # Clone the repo git clone https://lnkd.in/g-GiDX23 # Use included examples to load and run a Mixtral model cd TensorRT-LLM/examples/mixtral From there, the Python API lets you load the model, convert it with TensorRT, and run expert parallel inference—all with a few lines of code. Results? 📈 Performance at scale. Tests show up to 2.3x faster inference throughput compared to standard tensor parallelism when using 8 GPUs and top-2 experts per token. Even better—TensorRT-LLM keeps efficiency high across increasing batch sizes. Want to see it in action or contribute? 👉 Read the full tech blog: https://lnkd.in/g_7YV3vV 👉 Explore the code on GitHub: https://lnkd.in/gNjQ5W2U 👉 Follow updates in the TensorRT-LLM repo: https://lnkd.in/gqSHYQ4u Share your experiences with us.

78

3 Comments

i've been diving into llm agents lately, and after going through a lot of material—books, blog posts, research papers, and countless conversations with claude—i wanted to recommend few resources that really clicked for me. 1. Cameron R. Wolfe, Ph.D. https://lnkd.in/gJjD_3Pk he breaks down concepts from the ground up. no buzzwords—just solid fundamentals that actually make sense. his agents from first principles really sets the tone. you should definitely read it. 2. Chip Huyen https://lnkd.in/gCi8UrqM she has this amazing ability to cut through the hype and explain what's actually happening in the field. her perspectives have shaped how i think about ai. in this article she breaks down what makes an agent effective, tool selection, planning capabilities, and failure modes. 3. Anthropic https://lnkd.in/gEPqzyGJ their technical posts are incredible. they show you what's under the hood without drowning you in jargons. their building effective agents https://lnkd.in/g-XYme8Q is amazing. 4. Hugging Face's smolagents https://lnkd.in/gPbgwFZp is a minimalist library for implementing multi-step agentic workflows. 5. and if you want to go deeper, Berkeley RDI's llm agents mooc is worth checking out. they go really in depth. part 1 https://lnkd.in/gp89pK97 part 2 https://lnkd.in/giqYg74S image is about a "high-level flow of a coding agent" by anthropic

223

6 Comments

There's a lot of talk of LLMs "saturating all the evals" but there's plenty of evals people could make where LLMs would do poorly: -Beat a Zelda game -Make a profit in a prediction market -Write a stand-up set that's original and funny I'm bullish on AI, but we're far from done.

198

17 Comments

Excited to share about two manuscripts to be presented at ICASSP 2025! 🎉 1️⃣ Speech-N-LlaMA: Solving the Exposure Bias Problem in Speech LLMs 🤖 🤝 Research done with LlaMA Speech team, AI at Meta The challenge: LLMs are sequence-to-sequence networks that use teacher forcing (golden output) during training, but inference happens auto-regressively → creating an "exposure bias" gap! Our solution: N-pass decoding during training to bridge this gap Results: Up to 18% relative WER improvement with zero additional inference compute! 2️⃣ DiffSSD: Catching Up to Diffusion-Based Speech Generators 🤝 Done during and w/ PhD collaborators at Purdue University and ISPL - Politecnico di Milano The problem: Showed current synthetic speech detectors are falling behind! They don't generalize to recent diffusion-based generators Our solution: DiffSSD - a new 200-hours dataset featuring speech from both open-source and commercial diffusion-based generators Impact: Detectors trained with this dataset can counter synthetic speech from the latest speech generator tools ICASSP proceedings can be found in comment. Thanks to collaborators Gil, Desh, Ozlem, and all peers from LlaMA Speech team, Kratika, Prof. Delp, and Prof. Bestagini. #AI #SpeechLLMs #GenerativeAI #Deepfake #LLMs

177

5 Comments

The SF Chronicle just quoted my hot take on the AI bubble. Turns out my wildest prediction was spot-on (down to the exact number). I told them how I meet first-time founders (non researchers) with no revenue, no product, no team, and no deck and yet they are easily raising $5M+ on $25M+ caps for oversubscribed initial rounds. This was impossible 5 years ago. Back then, only proven founders with exits could raise large seed rounds. But now, engineers leaving OpenAI or hot AI startups get $100M+ for their first startup with zero proof of concept. I’ll be the first to claim: “Yes, it's a bubble”, but it may NOT matter because VCs are okay with having 1000 failures if one of them becomes a trillion-dollar company. A decade ago, one 10x exit could justify 10 failures. Then came the decacorns. One could justify 100 failures. Now, in the trillion-dollar era, one winner can justify 1000 failures. And, this is happening in real time. OpenAI is now preparing for an IPO targeting a $1 trillion valuation. They are doing $10B in annualized revenue but lost $12B just last quarter. This proves my point about why investors keep believing despite the exorbitant losses or sky–high valuations. In fact, Sam Altman himself warned that someone would lose a phenomenal amount of money. And the Bank of England and the IMF both warned recently of a market correction. So yes, it’s a bubble. But in the meantime, we’re all living in it and benefiting from the trillion-dollar potential of AI companies and the thousands of overpriced startups that it can support. And my trillion-dollar analogy is playing out right now with OpenAI's IPO plans. What do you think? Will OpenAI be that one massive winner that justifies the 1000 failures, or will this end like the dot-com crash?

149

29 Comments

We didn’t go to Ivy Leagues. Weren’t 22. Didn’t live in San Francisco. No big tech on our resumes. Yet Raffi Sarkissian and I got into YC before Lago had revenue, funding or even a working product. Today, “How do I get into YC?” is the most common question founders ask me. Here’s what I tell them: 1/ YC invests more into the founding team than into ideas. Make clear why you have an unfair advantage others don’t have. What experience or information do you have? 2/ Execution is everything. We already had 70+ startups show interest before we had a website. Basically, YC wants to see you’d build a startup with YC or without it. 3/ The application is simple, not easy. It’s a few questions that look like a Google Form. Don’t treat it that way. Workshop your application. Trim every unnecessary word. Make sure your writing is clear. 4/ Leverage the network (even from the outside). The YC community is one of the most supportive professional networks out there. And with thousands of YC founders out there on LinkedIn, reach out to see if they can give you feedback or even refer you. And even if you don’t get in, applying to YC forces you to answer many questions you probably should be asking yourself anyway.

88

6 Comments

As a Machine Learning (ML) engineer, I’ve come across the reality that system design interviews (in addition to leetcode, ML system designs, behavior, and ML algorithms) are becoming increasingly common in our field. IMO understanding system design concepts is essential, especially when scaling ML systems or handling production-level challenges. I found these System Design walkthrough videos incredibly helpful. The interviews are conducted by ex-Meta staff engineers, and they walk you through designing various complex systems. The videos focus on designing Ticketmaster, Uber, Dropbox/Google Drive, Twitter, an Ad Click Aggregator, a Web Crawler, Top-K YouTube Videos, LeetCode, Tinder, Bitly, WhatsApp, Live Comments. These videos go in-depth into designing large-scale systems and share key design principles, trade-offs, and challenges that can help you prepare for these types of questions in interviews. Video Playlist: https://lnkd.in/gxGqCmTW Additionally, I came across another highly recommended resource – the System Design Masterclass on Udemy. Although I haven’t personally tried it, it has received great reviews, and it seems like a great option to check out for anyone diving deeper into system design. [I've noticed the engineers who made these videos share coupons for a free course from time to time] Udemy link: https://lnkd.in/gkce_YzS If you're an ML engineer preparing for interviews or just want to sharpen your system design skills, I highly recommend checking out these resources.

50

NVIDIA Research just published an intersting paper stating that SLMs aren't just "budget alternatives" to LLMs, but they're often superior for real-world applications. Their analysis of popular agents like MetaGPT, Open Operator, and Cradle shows that 40-70% of current LLM calls could be effectively replaced by specialized SLMs. Meanwhile, Andrej Karpathy envisions an even more radical future: a small "cognitive core" model that runs locally by default, with always-on capabilities, multimodal input/output, and the ability to dial up reasoning power when needed while delegating complex tasks to cloud-based oracles. The convergence is clear: we're moving from monolithic, cloud-dependent LLMs toward modular, efficient architectures. SLMs offer 10-30x better cost efficiency, lower latency, and can be fine-tuned faster and at a reasonable cost. For agentic systems performing repetitive and specialized task are better handled with careful context engineering and specialized SLMs. The future isn't about choosing between big and small models, but it's about intelligent orchestration of both. Paper: https://lnkd.in/gZAgH4Y6 Andrej Karpathy's X post: https://lnkd.in/g23777Mc

114

2 Comments

Interpretable LLM-based Table Question Answering published in TMLR (June 2025), work led by Giang Nguyen over his internship at JPMC (Summer '24). This work decomposes queries on tabular data using a sequence of SQL transformations. The key aspect, we found, is that rendering the query plan in SQL is much more *interpretable* to the user, allowing faster and more accurate verification of the steps to produce the answer. Great work, Giang. He drove this project to its TMLR acceptance. https://lnkd.in/ge8_uWp4

38

RL for LLMs is getting popular -- it's not too expensive or too complex to try. Example: You can now train OpenAI gpt-oss with Reinforcement Learning (RL) for free on a Colab notebook. A few things that stood out to me: - The notebook auto-creates faster kernels via RL. - They also explain how to counteract reward-hacking (one of the biggest RL pitfalls). - Unsloth AI gets 3x faster inference, 50% less VRAM use, and 8x longer context windows — all without accuracy loss. Link to Colab notebook: https://lnkd.in/gueqnKRj Link to their guide: https://lnkd.in/drB756KX ♻️ Share it with anyone who’s curious about RL for LLMs :) I share tutorials on how to build + improve AI apps and agents, on my newsletter 𝑨𝑰 𝑬𝒏𝒈𝒊𝒏𝒆𝒆𝒓𝒊𝒏𝒈 𝑾𝒊𝒕𝒉 𝑺𝒂𝒓𝒕𝒉𝒂𝒌: https://lnkd.in/gaJTcZBR #AI #LLMs #GenAI

34

1 Comment