There are many applications that one can build by applying Reinforcement Learning (RL). We have chosen to develop a system, popularly called “Village Game” that will teach a computer agent on how to play this educational game to maximize the reward which in this case the total bank balance for the agent.
In developing countries there are serious problems in various areas mainly due to lack of education. We think providing practical education in a game environment can promote social equality and can have a profound effect especially when done at a young age. This game when developed completely can teach the children the responsibilities of a family and how various decisions can affect their livelihood.
The solution of this problem uses Reinforcement Learning using Q-Learning technique in unknown environments where the rewards are time delayed. The game also raises awareness of the importance of decision-making, based on objective data and the assessment of the consequences of various short, medium and long-term action alternatives.
In this problem the agent doesn’t know how the real world works. The agent will have to act in the real world, learn how it works from experiencing it and trying to take actions that yield high rewards. This problem combines our dual objective of an area that we want to understand the various aspects of Q-Learning technique and at the same time create a positive social impact.
Our intention is to implement the game using Pygame, a popular Python library for coding video games. And then implement the AI solution in Python. We intend to publish the details of our implementation, testing and analysis through repeated gameplay.
For more information read here
For our first version read here
First Install Gym
pip install --user gym
If you already have it then upgrade to latest version
pip install --upgrade pip
Next Install Village game gym env
Go to directory where setup.py is located.
cd Village-Gym/gym-village
pip install -e .
Go to directory Village-Gym/Tests
cd Village-Gym/Tests
python VillageTest.py
Model 1: Q-Learning (Q Tables)
Learn: python td_agent2.py learn
Play: python td_agent2.py play
Model 2: Function Approximation (state, action)
Learn: python td_agent3.py learn
Play: python td_agent3.py play
Model 3: Function Approximation (features)
Learn: python td_agent4.py learn
Play: python td_agent4.py play
Model 4: Deep Q Learning (DQN)
Preequisites:
Keras-rl -
pip install keras-rl
Tensorflow 1.13 -
pip install tensorflow=1.13.1
Jupyter Notebook
pip install jupyterlab
Next Install Village game Jupyter gym env
cd Village-Gym/Village2-Gym/gym-village2
pip install -e .
cd …/…/Tests-Jupyter
jupyter notebook
Run the model one step at a time
To do a test run, reduce the time it takes. In step 14, change the parameter as follows:
nb_steps=2000
dqn_only_embedding.fit(env, nb_steps=2000, visualize=False, verbose=1, nb_max_episode_steps=99, log_interval=500)
DQN Models
python -m pip install -U pip
python -m pip install -U matplotlib
cd …/…/Village-DQN
jupyter notebook
DQN State Based Model
open DQN_2_(state)_Learn_Play
To Learn
In step 12
num_episodes = 10000
mode = ‘train’
run the notebook
To Play
After training, to predict, change
mode = ‘test’
run the notebook