---
title: Game Playing meets Game Theory with Professor Michael Wellman
description: Game Playing meets Game Theory with Professor Michael Wellman
image: https://blog.grainstonelee.com/hubfs/GrainstoneLee_September2020/js/dbee24b5723389bd078f20b665959098.png
---
- [About](https://www.grainstonelee.com/about-us)
- [Candidates](https://www.grainstonelee.com/candidates)
- [CLIENTS](https://www.grainstonelee.com/clients)
- [Insights](https://blog.grainstonelee.com/insight)
- [Contact Us](https://www.grainstonelee.com/contact-us)
- [About](https://www.grainstonelee.com/about-us)
- [Candidates](https://www.grainstonelee.com/candidates)
- [CLIENTS](https://www.grainstonelee.com/clients)
- [Insights](https://blog.grainstonelee.com/insight)
- [Contact Us](https://www.grainstonelee.com/contact-us)
Game Playing meets Game Theory with Professor Michael Wellman
[BACK TO BLOG](https://blog.grainstonelee.com/insight)
In this edition of views from Academia I had the pleasure of talking to Professor Michael Wellman, Professor of Computer science & Engineering and Head of the Strategic Reasoning Group at the University of Michigan, about the exciting progress AI is making due to combining deep reinforcement learning with game-theoretic modelling. Below are some of the highlights.
**Simon Grainger:** For those of you that have missed Michael’s work on Game Theory and Agent-Based Modelling, can you give us a quick overview of the SRG?
**Professor Michael Wellman:** The Strategic Reason Group is part of the Artificial Intelligence Lab at the University of Michigan Computer Science Department.
## *"What we focus on is AI - automatic decision making in strategic domains".*
What strategic means technically is that you're in a world where there are other agents and the outcome to you depends on their actions as well as your own. Your audience, of course, is extremely familiar with strategic domains. It's an everyday occurrence where we go into the market and you can't say what the best strategy is for us to play without thinking about how everyone else is going to react, what everyone else is going to do. That poses special challenges for AI because you're not just reasoning about the world but your reasoning about a special world where there are other entities that are also reasoning about the world, namely the other agents.
What we do is develop technology for this kind of decision making and it generally involves Game Theory, which is the fundamental science of predicting the behaviour of other rational agents, or “approximately” rational agents.
## *"If you believe that the other parties making decisions have a good basis for making those decisions, they may not be perfectly smart but they're probably as close to smart as you are. Then that's an appropriate tool to use".*
**Simon Grainger:** Well it’s another topic whether the markets are rational or not. You gave a presentation recently on “Game Playing Meets Game Theory” referencing the AlphaGo project at Deep Mind and your own groups work on Empirical Game-Theoretic Analysis (EGTA). How excited are you by the potential of combining deep reinforcement learning with EGTA and why?
**Professor Michael Wellman: **
## *"This is a really exciting time for Artificial Intelligence. I've been working this field for a long time and the level of Interest has never been as great".*
That's partly because of some breakthroughs that have recently come to pass and new applications that are getting a lot of attention. Some of these breakthroughs have been by groups like Google Deepmind on games like ‘Go’ and actually more recently ‘Starcraft’.
Game playing and AI have been together really since the beginning of AI with some of the first work on playing chess or checkers. AI has had milestones in game-playing over the past decades. IBM's Deep Blue beat Kasparov. That was really interesting and was kind of cool but it really didn't change AI much.
## *"No one really thought that the way that IBM beat Kasparov was going to be the technique with the key to AI. It might be different now".*
The groups like Deepmind, Open AI and others are pursuing games as a challenging problem for AI not just arbitrarily because games are fun and people are interested in them because it's the multi-agent quality. The fact that you're pitting an intelligence against another intelligence, they view as a really effective way to drive AI because if you try to solve a problem that doesn't involve other agents usually with enough engineering you can eventually crack it, and then you're done.
But if there are other intelligent agents around, they're matching every step you take with a new idea that you have to respond to. The view in some of this really fundamental General AI work is that you need other agents to drive the development of intelligence further. So, I think that’s a really exciting new development.
Now, I mentioned my Strategic Reasoning Group because we're interested in Games, not necessarily because we want to solve the AI problem overall, but because we want to be good at games. I don't mean recreational games like ‘Go’ or ‘Poker’. They're kind of fun. But I mean games like trading in financial markets or games like cybersecurity.
## *"Real-world situations where there are other agents and being good at it has important consequences".*
We've been following these developments with great interest. A really interesting twist and what was the impetus for the presentation you mentioned, was seeing that in a lot of the recent work out of Deepmind, and others, game-theoretic methods are playing a bigger role.
For example, they reached a significant milestone in the very complex video game Starcraft 2. They used game-theoretical methods to help decide which other agent strategies to focus on in their learning. A good strategy depends on anticipating what the others are going to do and it's a very large space. The techniques that they use are very similar to the Empirical Game Theoretical Analysis techniques that we've been using in our lab for solving real-world games. This is a way of combining agent-based simulation. Simulation modelling where you have entities called agents that are making decisions along the way with Game Theory.
Some of you may be familiar with building a game theory matrix. Real-world situations play out over time, have lots of uncertainty, maybe have many players and a lot of complex dynamics...
## *"...in those situations, we can’t even build the Matrix".*
So we need more complex descriptions of the game situation and that's why we resort to agent-based simulation. We use the simulations to generate data about what happens when different combinations of strategies are played. Then we use this data with machine learning methods to learn a game model from that data.
It's a combination of Game Theory, Agent-Based Simulation and Machine Learning. That's where the empirical in EGTA comes from, its using empirical methods like statistics and machine learning to do game reasoning. That's been the bread and butter technique that we've been applying to a lot of the domains of interest and we're also finding it’s really on the forefront of some of AI work pushing the envelope on games that can be solved.
**Simon Grainger:** Where does deep reinforcement learning come into this?
**Professor Michael Wellman:** Of course, deep learning in neural networks have been a big part of the recent resurgence of success in AI. Deep reinforcement learning is specifically the use of deep neural networks for learning policies, learning strategies to play in a complex environment including a game environment. Basically, you can think of deep RL as a technology for deriving a policy to respond to a situation and in a game or in a situation with other agents, that environment includes what the other agents do.
So in our EGTA methods, we have a step where we have to think of new strategies. We've tried a bunch of strategies. We understand the game among those strategies. Now, let's think of some additional strategies may be that we haven't evaluated yet.
## 
## *"We use deep RL as a technique for generating those new strategies, and really that's how Alphago uses deep RL to come up with a new twist on a ‘Go’ playing strategy".*
So, Deep RL is a technology for coming up with a new strategy, and we use it in conjunction with game theory as a kind of a strategy jet. It's a generator of strategy ideas which we can then evaluate using the game-theoretic methods.
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
Click here to view the entire interview and find out more about the practical domains that Michael is applying his research to or alternatively you can visit Michael at [www.strategicreasoning.org](http://www.strategicreasoning.org) and uncover a treasure trove of papers and resources dedicated to Game Theory and Agent-Based Modelling.
### Don’t miss an update.
SUBSCRIBE TO OUR NEWSLETTER
### Subscribe To Our Newsletter
Unlock your potential
[Get in touch](https://www.grainstonelee.com/contact-us)
### address
Grainstone Lee Limited, 20 St Thomas Street, London, SE1 9RS
### number
+44 (0) 203 478 1898
### email
info@grainstonelee.com
### Social media
© 2020 Grainstone Lee Ltd. All rights reserved.
- Privacy Policy
### Privacy Policy
###### Grainstone Lee Limited Privacy Policy
###### This privacy policy will explain how our organisation uses the personal data we collect from you when you use our website.
Topics:
- What data do we collect?
- How do we collect your data?
- How will we use your data?
- How do we store your data?
- Marketing
- What are your data protection rights?
- What are cookies?
- How do we use cookies?
- What types of cookies do we use?
- How to manage your cookies
- Privacy policies of other websites
- Changes to our privacy policy
- How to contact us
- How to contact the appropriate authorities
**What data do we collect?**
Grainstone Lee Limited collects the following data:
- Personal identification information (Name, email address, phone number, etc.)
**How do we collect your data?**
You directly provide Grainstone Lee Limited with most of the data we collect. We collect data and process data when you:
- Register online, apply for a job or submit any other forms
- Voluntarily complete a customer survey or provide feedback on any of our message boards or via email.
- Use or view our website via your browser’s cookies.
**How will we use your data?**
Grainstone Lee Limited collects your data so that we can:
- Provide recruitment services and manage your account
- Email you with special offers, content and services we think you might like.
**How do we store your data?**
Grainstone Lee Limited securely stores your data at Suite 3, Third Floor, Roxby House, 20 – 22 Station Road, Sidcup DA15 7EJ.
Grainstone Lee Limited will keep your data for 6 months unless it is in your best interest for us to keep this longer, for example if we are processing an application for you. Once this time period has expired, we will delete your data from our systems.
**Marketing**
Grainstone Lee Limited would like to send you information about products and services of ours that we think you might like.
If you have agreed to receive marketing, you may always opt-out at a later date.
You have the right at any time to stop Grainstone Lee Limited from contacting you for marketing purposes.
If you no longer wish to be contacted for marketing purposes, please click [here](https://hs-4648117.s.hubspotemail.net/hs/manage-preferences/unsubscribe-test?d=VndxG37Zm3_CVWbyV-3zgDK_W3SYM003P6jZDW3K1LY33Qz6XDW3_fymH15gBJzW6R2ZdT6r9pWdW6G5VBg59p51gN89K80zWCp1QN7QpczPXk_7QVbw6sK1zSn3PW4CqGtw4l6bgVw6PMmFtmDQ2&v=3&utm_source=hs_email&utm_medium=email&utm_content=2&_hsenc=p2ANqtz-__cRcA7VOacblos76RUspmQkPU4wlz6nM1Ymxc_LM1NNM2QPTgrCKjn4rakxwhB_c89-B91hMjsPqG-kSD6SdWXiKRow&_hsmi=2).
**What are your data protection rights?**
Grainstone Lee Limited would like to make sure you are fully aware of all of your data protection rights. Every user is entitled to the following:
**The right to access** – You have the right to request Grainstone Lee Limited for copies of your personal data.
**The right to rectification** – You have the right to request that Grainstone Lee Limited correct any information you believe is inaccurate. You also have the right to request Grainstone Lee Limited to complete the information you believe is incomplete.
**The right to erasure** – You have the right to request that Grainstone Lee Limited erase your personal data, under certain conditions.
**The right to restrict processing** – You have the right to request that Grainstone Lee Limited restrict the processing of your personal data, under certain conditions.
**The right to object to processing** – You have the right to object to Grainstone Lee Limited’s processing of your personal data, under certain conditions.
**The right to data portability** – You have the right to request that Grainstone Lee Limited transfer the data that we have collected to another organization, or directly to you, under certain conditions.
If you make a request, we have one month to respond to you. If you would like to exercise any of these rights, please contact us at our email: [s.grainger@gslsearch.com](mailto:s.grainger@gslsearch.com)
Call us at: +44 20 3478 1898
Or write to us: Suite 3, Third Floor Roxby House, 20-22 Station Road, Sidcup, DA15 7EJ.
**Cookies**
Cookies are text files placed on your computer to collect standard Internet log information and visitor behaviour information. When you visit our websites, we may collect information from you automatically through cookies or similar technology
For further information, visit allaboutcookies.org.
**How do we use cookies?**
Grainstone Lee Limited uses cookies in a range of ways to improve your experience on our website, including:
- Keeping you signed in
- Understanding how you use our website
**What types of cookies do we use?**
There are a number of different types of cookies, however, our website uses:
- Functionality – Grainstone Lee Limited uses these cookies so that we recognize you on our website and remember your previously selected preferences. These could include what language you prefer and location you are in. A mix of first-party and second-party cookies are used.
- Advertising – Grainstone Lee Limited uses these cookies to collect information about your visit to our website, the content you viewed, the links you followed and information about your browser, device, and your IP address.
**How to manage cookies**
You can set your browser not to accept cookies, and the above website tells you how to remove cookies from your browser. However, in a few cases, some of our website features may not function as a result.
**Privacy policies of other websites**
The Grainstone Lee Limited website contains links to other websites. Our privacy policy applies only to our website, so if you click on a link to another website, you should read their privacy policy.
**Changes to our privacy policy**
Grainstone Lee Limited keeps its privacy policy under regular review and places any updates on this web page. This privacy policy was last updated on 30 March 2020.
**How to contact us**
If you have any questions about Grainstone Lee Limited’s privacy policy, the data we hold on you, or you would like to exercise one of your data protection rights, please do not hesitate to contact us.
Email us: [s.grainger@gslsearch.com](mailto:s.grainger@gslsearch.com)
Call us at: +44 20 3478 1898
Or write to us: Suite 3, Third Floor Roxby House, 20-22 Station Road, Sidcup, DA15 7EJ.
**How to contact the appropriate authority**
Should you wish to report a complaint or if you feel that Grainstone Lee Limited has not addressed your concern in a satisfactory manner, you may contact the Information Commissioner’s Office.
Email us: [s.grainger@gslsearch.com](mailto:s.grainger@gslsearch.com)
Or write to us: Suite 3, Third Floor Roxby House, 20-22 Station Road, Sidcup, DA15 7EJ.