Natural language processing is heavily Anglo-centric, while the demand formodels that work in languages other than English is greater than ever. Yet, thetask of transferring a model from one language to another can be expensive interms of annotation costs, engineering time and effort. In this paper, wepresent a general framework for easily and effectively transferring neuralmodels from English to other languages. The framework, which relies on taskrepresentations as a form of weak supervision, is model and task agnostic,meaning that many existing neural architectures can be ported to otherlanguages with minimal effort. The only requirement is unlabeled parallel data,and a loss defined over task representations. We evaluate our framework bytransferring an English sentiment classifier to three different languages. On abattery of tests, we show that our models outperform a number of strongbaselines and rival state-of-the-art results, which rely on more complexapproaches and significantly more resources and data. Additionally, we findthat the framework proposed in this paper is able to capture semantically richand meaningful representations across languages, despite the lack of directsupervision.