Learning with Online Constraints: Shifting Concepts and Active Learning
Many practical problems such as forecasting, real-time decisionmaking, streaming data applications, and resource-constrainedlearning, can be modeled as learning with online constraints. Thisthesis is concerned with analyzing and designing algorithms forlearning under the following online constraints: 1) The algorithm hasonly sequential, or one-at-time, access to data. 2) The time andspace complexity of the algorithm must not scale with the number ofobservations. We analyze learning with online constraints in avariety of settings, including active learning. The active learningmodel is applicable to any domain in which unlabeled data is easy tocome by and there exists a (potentially difficult or expensive)mechanism by which to attain labels.First, we analyze a supervised learning framework in which nostatistical assumptions are made about the sequence of observations,and algorithms are evaluated based on their regret, i.e. theirrelative prediction loss with respect to the hindsight-optimalalgorithm in a comparator class. We derive a lower bound on regretfor a class of online learning algorithms designed to track shiftingconcepts in this framework. We apply an algorithm we provided inprevious work, that avoids this lower bound, to an energy-managementproblem in wireless networks, and demonstrate this application in anetwork simulation. Second, we analyze a supervised learning frameworkin which the observations are assumed to be iid, and algorithms arecompared by the number of prediction mistakes made in reaching atarget generalization error. We provide a lower bound on mistakes forPerceptron, a standard online learning algorithm, for this framework.We introduce a modification to Perceptron and show that it avoids thislower bound, and in fact attains the optimal mistake-complexity forthis setting.Third, we motivate and analyze an online active learning framework.The observations are assumed to be iid, and algorithms are judged bythe number of label queries to reach a target generalizationerror. Our lower bound applies to the active learning setting as well,as a lower bound on labels for Perceptron paired with any activelearning rule. We provide a new online active learning algorithm thatavoids the lower bound, and we upper bound its label-complexity. Theupper bound is optimal and also bounds the algorithm's total errors(labeled and unlabeled). We analyze the algorithm further, yielding alabel-complexity bound under relaxed assumptions. Using opticalcharacter recognition data, we empirically compare the new algorithmto an online active learning algorithm with data-dependent performanceguarantees, as well as to the combined variants of these twoalgorithms.