BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Palaner//palaner Single Event//EN
METHOD:PUBLISH
CALSCALE:GREGORIAN
BEGIN:VEVENT
UID:meetup-e51f6408ea@palaner.app
DTSTAMP:20260802T151000Z
DTSTART:20260805T000000Z
DTEND:20260805T013000Z
SUMMARY:Benchmarks Are Lying to You: How to Evaluate Models and Agents in t
 he Real World
LOCATION:STATION Austin\, Voltron Room (1st floor)
DESCRIPTION:Saved in Palaner as: saved\n\nPalaner match: 35% · why: lots of
  people going\n\nDetails\nBenchmarks Are Lying to You: How to Tell Whether
  Models and Agents Actually Work\n\nDescription\nAI benchmarks are everywh
 ere\, but what do they actually measure\, and how much should we trust the
 m?\n\nIn this talk\, we’ll break down the major benchmarks used to evaluat
 e language models\, coding systems\, multimodal models\, and autonomous ag
 ents\, from MMLU and GPQA to SWE bench\, GAIA\, and OSWorld.\n\nWe’ll expl
 ore common pitfalls such as benchmark contamination\, judge bias\, agent s
 caffolding\, hidden test-time compute\, and misleading leaderboard compari
 sons\, then discuss how to design practical evaluations for real world AI 
 products.\n\nAttendees will leave with a clearer framework for interpretin
 g benchmark results\, comparing models fairly\, and measuring whether an A
 I system actually works in production.\n\nSpeaker\nRakshak Talwar\n\nInfo\
 nAustin Deep Learning Journal Club is group for committed machine learning
  practitioners and researchers alike. The group typically meets every firs
 t Tuesday of each month to discuss research publications. The publications
  are usually the ones that laid foundation to ML/DL or explore novel promi
 sing ideas and are selected by a vote. Participants are expected to read t
 he publications to be able to contribute to discussion and learn from othe
 rs. This is also a great opportunity to showcase your implementations to g
 et feedback from other experts.\n\nSponsors:\nSTATION Austin is the center
  of gravity for entrepreneurs in Texas. Day and night\, in-person and onli
 ne\, we gather the best founders\, programmers\, and designers outside of 
 Silicon Valley and introduce them to investors\, employees\, and customers
  who help their ideas launch. STATION Austin is powered by Capital Factory
 \, whose investments and leadership have helped power the Texas startup ec
 osystem for more than a decade. To sign up for a STATION Austin membership
 \, click here.
URL:https://www.meetup.com/austin-deep-learning/events/312687537/
CATEGORIES:palaner,saved
STATUS:TENTATIVE
END:VEVENT
END:VCALENDAR