BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Memento EPFL//
BEGIN:VEVENT
SUMMARY:IC Colloquium: A new framework for large-scale sequential A/B test
 ing
DTSTART:20180206T101500
DTEND:20180206T113000
DTSTAMP:20260406T103730Z
UID:d28bce06811c63812c746679e3049122c3a4c506d0226c6a0b59466d
CATEGORIES:Conferences - Seminars
DESCRIPTION:By: Aaditya Ramdas - UC Berkeley\nIC Faculty candidate\n\nAbst
 ract:\nThe current framework of large-scale A/B testing in the tech indust
 ry has several drawbacks. Firstly\, the tests are often continuously moni
 tored without correcting for the resulting inflation in the false alarm ra
 te. Secondly\, the number of samples used in the experiment grows linearly
  with the number of options being tested\, independent of the quality of t
 he options. Lastly\, running hundreds or thousands of such tests artificia
 lly inflates the apparent number of significant discoveries\, and companie
 s have no idea what proportion of their discoveries are spurious.\n \nWe 
 propose a new framework as an alternative to existing setups for controlli
 ng false alarms across multiple A/B tests\, and tackles all three aforemen
 tioned issues. It combines ideas from pure exploration for best-arm identi
 fication in multi-armed bandits (MAB)\, with online false discovery rate (
 FDR) control. This framework has various applications\, including pharmace
 utical companies testing a control pill against a few treatment options\, 
 to internet companies testing their current default webpage (control) vers
 us many alternatives (treatment). Our setup allows running a (possibly inf
 inite) sequence of best-arm MAB instances\, and controlling the overall FD
 R of the process in a fully online manner. We adapt existing theory from b
 oth the MAB and online FDR literature to ensure that our framework comes w
 ith strong sample-optimality guarantees\, as well as control of the power 
 and (a modified) FDR at any time. \n \nThis talk will feature joint work
  with (alphabetically) Fanny Yang\, Kevin Jamieson\, Michael Jordan\, and 
 Martin Wainwright.\n\nBio:\nAaditya Ramdas is a postdoctoral researcher i
 n Statistics and EECS at UC Berkeley\, advised by Michael Jordan and Marti
 n Wainwright. He finished his PhD in Statistics and Machine Learning at CM
 U\, advised by Larry Wasserman and Aarti Singh\, winning the Best Thesis A
 ward in Statistics. A lot of his research focuses on modern aspects of rep
 roducibility in science and technology — involving statistical testing
  and false discovery rate control in static and dynamic settings. \n\nMor
 e information\n\n 
LOCATION:BC 420 https://plan.epfl.ch/?room==BC%20420
STATUS:CONFIRMED
END:VEVENT
END:VCALENDAR