Topic: Inference

The conversation shifts to the financial industry's realization that AI inference requires significant guardrail systems to be reliable. Experts note that depending on raw chatbot output often leads to a 50% error rate in document analysis. The hosts emphasize that software must be built to perform specific tasks, using inference only at certain points in the pipeline to prevent hallucinations.

The vision of "appless" computing is discussed, where users "vibe code" custom tools for specific tasks rather than using general applications. The DGX Spark server is currently running inference for a 35-billion parameter model and Whisper Turbo, with the ultimate goal of fine-tuning a custom model on the entire Podcast Index database.