Amir-massoud Farahmand



Research Goal

I help machines learn how to live a rewarding life! A machine can be an industrial controller whose job is to optimize the production of a chemical factory, or it might be a recommendation app that suggests products on the web. If the suggestions are appreciated, people will continue using the service. This would be a rewarding life for this app.

These are two examples of sequential decision-making problems under uncertainty (also known as reinforcement learning (RL) problems). The goal of an RL agent is finding a policy based on interaction with the environment such that the long-term performance is maximized. This abstract framework includes a wide-range of real-world applications such as problems in robotics, healthcare decision support systems (e.g., adaptive treatment strategies for chronic diseases, epilepsy management and suppression), decision making in financial markets, smart video games, etc.

My research goal is to develop and analyze algorithms to solve high-dimensional reinforcement learning problems based on solid mathematical principles. I also work on other areas of machine learning. See my publications for more information.

Research Interests

  1. BulletMachine Learning and Statistics: statistical learning theory, nonparametric algorithms, regularization, manifold learning, non-i.i.d. processes, online learning

  2. BulletReinforcement Learning and Sequential Decision Making: high-dimensional problems, regularized algorithms

  3. BulletRobotics: uncalibrated visual servoing, learning from demonstration, behavior-based architecture for robot control

  4. BulletLarge-scale Optimization

  5. BulletEvolutionary Computation: cooperative co-evolution, interaction of evolution and learning


  1. BulletI left McGill University after more than two years of working on many interesting projects with Doina Precup, Joelle Pineau and other wonderful members of the Reasoning and Learning Lab. The good news is that I am joining the Robotics Institute, Carnegie Mellon University to work with Drew Bagnell.

  2. BulletWe are organizing Sequential Decision-Making with Big Data workshop at AAAI-2014.

  3. BulletTwo papers at the Neural Information Processing Systems (NIPS) conference, 2013. Learning from Limited Demonstrations and Bellman Error Based Feature Generation using Random Projections on Sparse Spaces. Joint work with Beomjoon, Doina, Joelle, Mahdi, and Yuri.

  4. BulletTwo papers at the 1st Multidisciplinary Conference on Reinforcement Learning and Decision Making: Approximate Policy Iteration with Demonstration Data and CAPI: Generalized Classification-based Approximate Policy Iteration. Joint work with Andre, Beomjoon, Doina, Joelle, and Mohammad.

  5. BulletPhD Outstanding Thesis Award for the period of 2011-2012, Department of Computing Science, University of Alberta, 2013.

Academic blog: ThesiLog

Persian blog: ضدخاطرات

Google Scholar Profile

CV (To be updated!)

Academic and Non-academic Tweets