RIKEN Center for AIP: Human Computation Team

• Hisashi Kashima

• Jiyi Li (Yamanashi University)
• Junichiro Mori (University of Tokyo)
• Satoshi Oyama (Hokkaido University)
• Shigeru Saito (SIGNATE Inc.)
• Kazuki Yamamoto (University of Tokyo)
• Yuya Yoshikawa (Chiba Institute of Technology)

• Jill-Jênn Vie (INRIA)

1. Foundations of human computation:
One of the most major technical issues in human computation is quality assurance because human computation is mainly executed on crowdsourcing platforms with unspecified people having various levels of skills, knowledges, or diligence, and therefore the quality of results is unstable. We focus on statistical aggregation methods for various question types as well as mechanism design to motivate people to participate into human computation. For example, we focus on knowledge intensive questions that only a small number of experts can correctly answer, such as ones asking medical knowledge, where the majority of members fail.
2. Human-in-the-loop artificial intelligence:
Most of the real-world problems are hard to address only with machine intelligence, and we develop general methods to involve humans into machine learning systems. For example, we develop human-in-the-loop machine learning frameworks in which humans adaptively design new features useful for prediction, feature values are extracted by humans, and deep neural networks learn from crowd-labeled data. We also study a more general problem solving framework based on crowdsourcing that consists of collecting solution ideas, and organizing and prioritizing them for supporting decision making.
3. Learning analytics:
Not only exploiting human intelligence to develop intelligent systems, we also focus on helping human more intelligent. We study learning analytics that exploits artificial intelligence technology to boost human education at scale. One of our specific target areas is data science education through data science competitions.
4. Advanced machine learning algorithms and applications:
We delvelop new machine learning algorithms as well as exploring new machine learning applications collaborating with industry collaborators.

1. Jiyi Li. Budget Cost Reduction for Label Collection with Confusability-based Exploration. In Proceedings of the 26th International Conference on Neural Information Processing (ICONIP), 2019.
2. Jiyi Li, 馬場 雪乃, 鹿島 久嗣. 超問題：専門知識を要するクラウドソーシングタスクの回答統合法. 日本データベース学会和文論文誌, Vol. 17-J, 2019.
3. Jiyi Li, Hisashi Kashima. Incorporating Worker Similarity for Label Aggregation in Crowdsourcing. In Proceedings of the 27th International Conference on Artificial Neural Networks (ICANN), 2018.
4. Jiyi Li, Yukino Baba, Hisashi Kashima. Simultaneous Clustering and Ranking from Pairwise Comparisons. In Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI), pp.XX-XX, 2018.
5. Yuko Sakurai, Jun Kawahara, Satoshi Oyama. Aggregating Crowd Opinions Using Shapley Value Regression. In Proceedings of the 12th Multi-Disciplinary International Conference on Artificial Intelligence (MIWAI8), pp. 151-160, 2018.
6. Jiyi Li, Yukino Baba, Hisashi Kashima. Hyper Questions: Unsupervised Targeting of a Few Experts in Crowdsourcing. In Proceeding of the 26th ACM International Conference on Information and Knowledge Management (CIKM), 2017.
7. Kosuke Yoshimura, Yukino Baba, Hisashi Kashima. Quality Control for Crowdsourced Multi-Label Classification using RAkEL. In Proceeding of the 24th International Conference on Neural Information Processing (ICONIP), 2017.
8. Hiroki Morise, Satoshi Oyama, Masahito Kurihara. Collaborative Filtering and Rating Aggregation Based on Multicriteria Rating. In Proceedings of the First IEEE Workshop on Human-Machine Collaboration in Big Data (HMData) pp. 4417-4422, 2017.
9. Yuko Sakurai, Masafumi Matsuda, Masato Shinoda, Satoshi Oyama. Crowdsourcing Mechanism Design. In Proceedings of the 20th International Conference on Principles and Practice of Multi-Agent Systems (PRIMA) pp. 495-503, 2017.

1. Shun Ito, Yukino Baba, Tetsu Isomura, Hisashi Kashima. Synthetic Accessibility Assessment Using Auxiliary Responses. Expert Systems with Applications (ESWA), 2020.
2. Jiyi Li, Fumiyo Fukumoto. A Dataset of Crowdsourced Word Sequences: Collections and Answer Aggregation for Ground Truth Creation. In Proceedings of the First Workshop on Aggregating and analysing crowdsourced annotations for NLP (AnnoNLP 2019), pp. 24-28, 2019.
3. Jing Song, Satoshi Oyama, Masahito Kurihara. Exploratory Causal Analysis of Open Data: Explanation Generation and Confounder Identification. Journal of Advanced Computational Intelligence and Intelligent Informatics, Vol.24, No.1, 2020.
4. Takafumi Suzuki, Satoshi Oyama, Masahito Kurihara. Explainable Recommendation Using Review Text and a Knowledge Graph. In Proceedings of the 3rd IEEE Workshop on Human-in-the-loop Methods and Human Machine Collaboration in BigData (HMData), 2019.
5. Kyohei Atarashi, Akimi Moriyama, Satoshi Oyama, Masahito Kurihara. A Personalized Affect Response Model for Online News Articles. In Proceedings of the 5th Workshop on Linguistic and Cognitive Approaches to Dialog Agents (LaCATODA), pp. 5-10, 2019.
6. Yusuke Sakata, Yukino Baba, Hisashi Kashima, Hisashi Kashima. CrowNN: Human-in-the-loop Network with Crowd Crowd-generated Inputs. In Proceedings of the 44th International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2019.
7. Yukino Baba, Tetsu Isomura, Hisashi Kashima. Wisdom of Crowds for Synthetic Accessibility Evaluation. Journal of Molecular Graphics and Modelling, Vol.80, pp.217-223, 2018.
8. Jing Song, Satoshi Oyama, Masahito Kurihara. A Framework for Crowd-based Causal Analysis of Open Data. In Proceedings of the 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2018.
9. Kyohei Atarashi, Satoshi Oyama, Masahito Kurihara. Semi-supervised Learning from Crowds Using Deep Generative Models. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI), 2018.
10. Ryusuke Takahama, Yukino Baba, Nobuyuki Shimizu, Sumio Fujita, Hisashi Kashima. AdaFlock: Adaptive Feature Discovery for Human-in-the-loop Predictive Modeling. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI), 2018.
11. Takafumi Suzuki, Satoshi Oyama, Masahito Kurihara. Toward Explainable Recommendations: Generating Review Text from Multicriteria Evaluation Data. In Proceedings of the Second IEEE Workshop on Human-in-the-loop Methods and Human Machine Collaboration in Big Data (HMData), 2018.

1. Benoît Choffin, Fabrice Popineau, Yolaine Bourda, Jill-Jênn Vie. DAS3H: a new student learning and forgetting model for optimally scheduling distributed practice of skills. In Proceedings of the 12th International Conference on Educational Data Mining (EDM), 2019. [Best Paper Award]
2. Takeru Sunahase, Yukino Baba, Hisashi Kashima. Probabilistic Modeling of Peer Correction and Peer Assessment. In Proceedings of the 12th International Conference on Educational Data Mining (EDM), 2019.
3. Ryota Sekiya, Satoshi Oyama, Masahito Kurihara User-Adaptive Preparation of Mathematical Puzzles Using Item Response Theory and Deep Learning. In Proceedings of the 32nd International Conference on Industrial, Engineering & Other Applications of Applied Intelligent Systems (IEA/AIE), 2019.
4. Jill-Jênn Vie, Hisashi Kashima. Factorization Machines for Knowledge Tracing. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI), 2019.
5. Jill-Jênn Vie, Fabrice Popineau, Éric Bruillard, Yolaine Bourda. Automated Test Assembly for Handling Learner Cold-Start in Large-Scale Assessments. International Journal of Artificial Intelligence in Education, pp.1–16, 2018.
6. Sein Minn, Yi Yu, Michel Desmarais, Feida Zhu, Jill-Jênn Vie. Deep Knowledge Tracing and Dynamic Student Classification for Knowledge Tracing. In Proceedings of the 18th IEEE International Conference on Data Mining (ICDM), pp.1182–1187, 2018.
7. Yukino Baba, Tomoumi Takase, Kyohei Atarashi, Satoshi Oyama, Hisashi Kashima. Data Analysis Competition Platform for Educational Purposes: Lessons Learned and Future Challenges. In Proceedings of the 8th Symposium on Educational Advances in Artificial Intelligence (EAAI) 2018.
8. Jill-Jênn Vie, Fabrice Popineau, Françoise Tort, Benjamin Marteau, Nathalie Denos. A Heuristic Method for Large-Scale Cognitive-Diagnostic Computerized Adaptive Testing. In Proceedings of the Fourth ACM Conference on Learning @ Scale (L@S), pp.323–26, 2017.

1. Masaru Isonuma, Junichiro Mori, Ichiro Sakata. Unsupervised Neural Single-Document Summarization of Reviews via Learning Latent Discourse Structure and its Ranking. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019), pp.2142–2152, 2019.
2. Kazuya Shimura, Jiyi Li, Fumiyo Fukumoto. Text Categorization by Learning Predominant Sense of Words as Auxiliary Task. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019), pp. 1109-1119, 2019.
3. Yuko Sakurai, Satoshi Oyama, Mingyu Guo, Makoto Yokoo. Deep False-Name-Proof Auction Mechanisms. In Proceedings of the 22nd International Conference on Principles and Practice of Multi-Agent Systems (PRIMA), pp. 594-601, 2019.
4. Shonosuke Harada, Hirotaka Akita, Masashi Tsubaki, Yukino Baba, Ichigaku Takigawa, Yoshihiro Yamanishi, Hisashi Kashima. Dual Graph Convolutional Neural Network for Predicting Chemical Networks. BMC Bioinformatics (presented at GIW/ABACBS 2019), 2019.
5. Ryoma Sato, Makoto Yamada, Hisashi Kashima. Approximation Ratios of Graph Neural Networks for Combinatorial Problems. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
6. Rafael Pinot, Laurent Meunier, Alexandre Araujo, Hisashi Kashima, Florian Yger, Cédric Gouy-Pailler, Jamal Atif. Theoretical Evidence for Adversarial Robustness Through Randomization. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
7. Shogo Hayashi, Yoshinobu Kawahara, Hisashi Kashima. Active Change-Point Detection. In Proceedings of the 11th Asian Conference on Machine Learning (ACML), 2019.
8. Ryoma Sato, Makoto Yamada, Hisashi Kashima. Learning to Sample Hard Instances for Graph Algorithms. In Proceedings of the 11th Asian Conference on Machine Learning (ACML), 2019.
9. Daiki Tanaka, Yukino Baba, Kashima Hisashi, Yuta Okubo. Large-scale Driver Identification Using Automobile Driving Data. In Proceedings of 2019 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2019.
10. Shonosuke Harada, Kazuki Taniguchi, Makoto Yamada, Hisashi Kashima. In-app Purchase Prediction Using Bayesian Personalized Dwell Day Ranking. In Proceedings of AdKDD 2019 Workshop (AdKDD), 2019.
11. Kosuke Yoshimura, Tomoaki Iwase, Yukino Baba, Hisashi Kashima. Interdependence Model for Multi-label Classification. In Proceedings of the 28th International Conference on Artificial Neural Networks (ICANN), 2019.
12. Hiroki Morise, Satoshi Oyama, and Masahito Kurihara. Bayesian probabilistic tensor factorization for recommendation and rating aggregation with multicriteria evaluation data. Expert Systems with Applications, Vol.131, pp.1-8, 2019.
13. Kyohei Atarashi, Subhransu Maji, Satoshi Oyama. Random Feature Maps for the Itemset Kernel. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI), 2019.
14. Guoxi Zhang, Tomoharu Iwata, Hisashi Kashima. On Reducing Dimensionality of Labeled Data Efficiently. In Proceedings of the 22nd Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD), 2018.
15. Koh Takeuchi, Hisashi Kashima, Naonori Ueda. Autoregressive Tensor Factorization for Spatio-temporal Predictions. In Proceedings of the 2017 IEEE International Conference on Data Mining (ICDM), 2017.