Crowdsourcing the Perception of Machine Teaching

  • 2020-02-05 03:20:25
  • Jonggi Hong, Kyungjun Lee, June Xu, Hernisa Kacorri
  • 18


Teachable interfaces can empower end-users to attune machine learning systemsto their idiosyncratic characteristics and environment by explicitly providingpertinent training examples. While facilitating control, their effectivenesscan be hindered by the lack of expertise or misconceptions. We investigate howusers may conceptualize, experience, and reflect on their engagement in machineteaching by deploying a mobile teachable testbed in Amazon Mechanical Turk.Using a performance-based payment scheme, Mechanical Turkers (N = 100) arecalled to train, test, and re-train a robust recognition model in real-timewith a few snapshots taken in their environment. We find that participantsincorporate diversity in their examples drawing from parallels to how humansrecognize objects independent of size, viewpoint, location, and illumination.Many of their misconceptions relate to consistency and model capabilities forreasoning. With limited variation and edge cases in testing, the majority ofthem do not change strategies on a second training attempt.


Quick Read (beta)

loading the full paper ...