Learning in an Echo Chamber:
Online Learning with Replay Adversary

Daniil Dmitriev, Harald Eskelund Franck, Carolin Heinzler, Amartya Sanyal (- order)

University of Pennsylvania, University of Copenhagen

Echo Chamber: A Definition

According to the Cambridge Dictionary:

echo chamber (situation):

[/ˈek.əʊ ˌtʃeɪm.bər/]
A situation in which people only hear opinions of one type,
or opinions that are similar to their own.

Motivation

Source: Report from Graphite.io, Plot: Article by Axios.com

Online Learning

For :

  • Learner outputs a hypothesis ,
  • Nature (adversarial or stochastic) produces and reveals to where for some ,
  • Learner suffers loss .

Online Learning with a Replay Adversary

For :

  • Learner outputs a hypothesis ,

  • Nature (adversarial or stochastic) produces

  • Replay adversary reveals to , where either

  • Learner suffers loss not revealed to the learner.

Online Learning with a Replay Adversary

Number of mistakes:

Replay adversary can pick consistent with all rounds for which .

Define the reliable version space as the set of hypotheses consistent with all labels that could not have been produced by replay.

Related Work

  • Online Learning: proper and improper learning, realisable and agnostic (different noise models)
  • Closure Algorithm and Intersection Closed Classes
  • Performative Prediction: learner's predictions induce concept drift

Example: Learning Thresholds

Consider finite domain and threshold functions on .

Halving-based Algorithm



Correct prediction

Example: Learning Thresholds

Consider finite domain and threshold functions on .

Halving-based Algorithm



Correct prediction

Halving-based Algorithm



Correct prediction

Define a Trap Region as a region where the leaner has predicted with both 0s and 1s in previous rounds, without being certain of the label.

Halving-based Algorithm



Correct prediction

Define a Trap Region as a region where the leaner has predicted with both 0s and 1s in previous rounds, without being certain of the label.

Halving-based Algorithm



Mistake?

Halving-based Algorithm



Mistake?

Closure-based Algorithm



Correct prediction

Closure-based Algorithm



Correct prediction

Closure-based Algorithm



Incorrect prediction: mistake

Closure-based Algorithm



Incorrect prediction: mistake

Closure-based Algorithm



Incorrect prediction: mistake

Insight: A closure-based algorithm never creates a Trap Region.
We show: For replay, only the closure-based algorithm achieves sublinear mistakes in .

Closure-based Algorithm



Incorrect prediction: mistake

Insight: A closure-based algorithm never creates a Trap Region.
We show: For replay, only the closure-based algorithm achieves sublinear mistakes in .

General Results: Prerequisites

  • Define the -closure as for any .
    is intersection-closed (over arbitrary intersections) if .
  • For any define the -representation .
    The -representation of is .
  • Define the Threshold dimension as the largest , such that
    and with .

Definition

We define the Extended Threshold dimension of :

where .

Results on Extended Threshold dimension

For any hypothesis class , the Extended Threshold dimension satisfies

If is intersection-closed, then .

Furthermore, for every , there exists a hypothesis class over , such that , but .

Closure-Algorithm

Result I


For general convex bodies () in the stochastic adversarial case, we also establish matching upper and lower bounds sublinear in .

Result II

is properly learnable in the replay setting, if and only if there exists a -representation of the class such that is intersection-closed.

Replay is generalisation of mistake bound model no access to the loss recall stochastic and adversarial setting (also proper and improper)

Note that it is possible to have M(A)=0 (if only replay) but learner does not learn anything about f^* (reliable version space=H)

online learning improper is little dim, and proper is k*Little dim (Helly number), different noise models closure algorithm: conistent algo and achieves optimal sample comp for PAC learnin performative pred: note that data distribtuion is not affected, just labels. and possibly adversarial

The learner halves the *reliable version space* every time with their prediction explain concept of exploring the space vs constervative

The learner halves the *reliable version space* every time with their prediction explain concept of exploring the space vs constervative

Then for all $Y\subseteq \mathcal{X}$, it holds that $\mathrm{clos}_\mathcal{H}(Y)\in\mathcal{H}$. Talk about that we often use the supp(h) and h interchangeably

talk about flipping! and results on VC dim 1

Used in lower bounds for dp [Noga Alon, Roi Livni, Maryanthe Malliaris, and Shay Moran. Private pac learning implies finite littlestone dimension.] explain intuition of upper triangular matrix or longest chain w.r.t. subset relation It holds that the Treshold dimension is exponentially related to the Littlestone dimension of a hypothesis class: $\tdim{\HC} = \Omega(\log(\Ldim(\HC))) \text{ and } \Ldim(\HC) = \Omega(\log(\tdim{\HC}))$ \cite{shelah1978ClassificationTheoryNumbers,hodges1997ShorterModelTheory}.

Explain the Flipping!

note that for thresholds N/2=ExThresh for intersection closed ThD= ExThresh up to constants general classes upper bound scales with VC(clos(H)) which can be arbitrarily big Note: VC dim 1 classes are nice Stochastic convex case: T^(d-1)/(d+1)

Int-closed is sufficient and necessary for proper importance of choosing f to make learning possible and to improve example of 2 intervals: a proper learner can be forced to make mistakes