README.md

@@ -9,22 +9,22 @@ The package has been written to be used in the master course of AI taught at the
 You can download the source code from <https://gitlab.inf.unibz.it/tessaris/wumpus> and use `pip install .`, or install directly from the repository using
 
 ```
-pip install https://gitlab.inf.unibz.it/tessaris/wumpus/-/archive/master/wumpus-master.tar.gz
+pip install git+https://gitlab.inf.unibz.it/tessaris/wumpus.git@master
 ```
 
 ## Usage
 
-To write your own player you should create a subclass of `Player` (defined in [gridworld.py](https://gitlab.inf.unibz.it/tessaris/wumpus/blob/master/wumpus/gridworld.py)) and then use an instance as a parameter of the `run_episode` method of `GridWorld` class (defined in [gridworld.py](https://gitlab.inf.unibz.it/tessaris/wumpus/blob/master/wumpus/gridworld.py)).
+To write your own player you should create a subclass of `OnlinePlayer` or `OfflinePlayer` (defined in [player.py](https://gitlab.inf.unibz.it/tessaris/wumpus/blob/master/wumpus/player.py)) and then use an instance as a parameter of the `run_episode` function (defined in [runner.py](https://gitlab.inf.unibz.it/tessaris/wumpus/-/blob/master/wumpus/runner.py)).
 
-Examples of the usage of the package can be found in the implementation of two players `RandomPlayer` and `UserPlayer` from [gridworld.py](https://gitlab.inf.unibz.it/tessaris/wumpus/blob/master/wumpus/gridworld.py), and in the files [`wumpus-usage.py`](https://gitlab.inf.unibz.it/tessaris/wumpus/blob/master/examples/wumpus-usage.py), [`eater-usage.py`](https://gitlab.inf.unibz.it/tessaris/wumpus/blob/master/examples/eater-usage.py) in the [`examples`](https://gitlab.inf.unibz.it/tessaris/wumpus/blob/master/examples) directory of the repository.
+Examples of the usage of the package can be found in the implementation of two players `RandomPlayer` and `UserPlayer` in [player.py](https://gitlab.inf.unibz.it/tessaris/wumpus/blob/master/wumpus/player.py), and in the files [`wumpus_usage.py`](https://gitlab.inf.unibz.it/tessaris/wumpus/blob/master/examples/wumpus_usage.py), [`eater_usage.py`](https://gitlab.inf.unibz.it/tessaris/wumpus/blob/master/examples/eater_usage.py) in the [`examples`](https://gitlab.inf.unibz.it/tessaris/wumpus/blob/master/examples) directory of the repository.
 
-Your player could be also run using the script `gridrunner` script (in the repository is the `runner.py` file) and it'll be available once the package is installed (in alternative could be executed using `python -m wumpus.runner`):
+Your player could be also run using the script `gridrunner` script (in the repository is the [`cli.py`](https://gitlab.inf.unibz.it/tessaris/wumpus/-/blob/master/wumpus/cli.py) file) and it'll be available once the package is installed (in alternative could be executed using `python -m wumpus.cli`):
 
-```
+``` bash
 $ gridrunner --help
 usage: gridrunner [-h] [--name NAME] [--path PATH] --entry ENTRY
                   [--world {EaterWorld,WumpusWorld}] [--horizon HORIZON]
-                  [--noshow] [--out OUT] [--version]
+                  [--noshow] [--out OUT] [--version] [--log LOG]
                   [infiles [infiles ...]]
 
 Run episodes on worlds using the specified player.
@@ -54,31 +54,32 @@ optional arguments:
   --out OUT, -o OUT     write output to file (default: <_io.TextIOWrapper
                         name='<stdout>' mode='w' encoding='UTF-8'>)
   --version             show program's version number and exit
+  --log LOG, -l LOG     write the log of the games to file (JSON) (default:
+                        None)
 ```
 
 For example:
 
 ``` bash
-$ gridrunner --world EaterWorld --entry wumpus:RandomPlayer --noshow --horizon 5 --path examples ./examples/eater-world.json
-┌────────┐
-│．．．．．│
-│．██．．．│
-│．．．．．│
-│．．．．．│
-│🍌🐒．🍌．│
-└────────┘
-Step 0: agent Eater_df9919cd executing W -> reward 9
-Step 1: agent Eater_df9919cd executing S -> reward -1
-Step 2: agent Eater_df9919cd executing N -> reward -1
-Step 3: agent Eater_df9919cd executing W -> reward -1
-Step 4: agent Eater_df9919cd executing N -> reward -1
+$ gridrunner --world EaterWorld --entry wumpus:RandomPlayer --noshow --horizon 5 ./examples/eater-world.json
+Step 0: agent Eater_c881e1b0 executing N -> reward -1
+Step 1: agent Eater_c881e1b0 executing W -> reward -1
+Step 2: agent Eater_c881e1b0 executing E -> reward -1
+Step 3: agent Eater_c881e1b0 executing W -> reward -1
+Step 4: agent Eater_c881e1b0 executing E -> reward -1
 Episode terminated by maximum number of steps (5).
-┌─────────┐
+┌──────────┐
 │．．．．．│
 │．██．．．│
-│🐒．．．．│
 │．．．．．│
-│．．．🍌．│
-└─────────┘
-Episode terminated with a reward of 5 for agent Eater_df9919cd
+│．🐒．．．│
+│🍌．．🍌．│
+└──────────┘
+Episode terminated with a reward of -5 for agent Eater_c881e1b0
 ```
+
+You can also use a player defined in a script; e.g., if the player class `GooPlayer` is defined in the `eater_usage.py` you can use the `eater_usage:GooPlayer` entry. Remember Python rules for finding modules, where the current directory is added to the search path. If the script is in a different directory, you can use the `--path` argument to tell the script where to find it:
+
+```bash
+gridrunner --world EaterWorld --entry eater_usage:GooPlayer --path examples --noshow --horizon 5 ./examples/eater-world.json
+```
\ No newline at end of file

examples/eater_usage.py

@@ -11,7 +11,7 @@ import sys
 from wumpus import OfflinePlayer, run_episode, Eater, EaterWorld, Food
 
 
-class MyPlayer(OfflinePlayer):
+class GooPlayer(OfflinePlayer):
     """
     Offline player demonstrating the use of the start episode method to inspect the world.
     """
@@ -19,7 +19,7 @@ class MyPlayer(OfflinePlayer):
     def _say(self, text: str):
         print(self.name + ' says: ' + text)
 
-    def start_episode(self, world: EaterWorld):
+    def start_episode(self, world: EaterWorld) -> Iterable[Eater.Actions]:
         """
         Print the description of the world before starting.
         """
@@ -34,6 +34,7 @@ class MyPlayer(OfflinePlayer):
         for o in world.objects:
             if isinstance(o, Eater):
                 eater_location = (o.location.x, o.location.y)
+                all_actions = list(o.Actions)
             elif isinstance(o, Food):
                 food_locations.append((o.location.x, o.location.y))
 
@@ -46,24 +47,21 @@ class MyPlayer(OfflinePlayer):
         self._say('Food in {}'.format(sorted(food_locations)))
         self._say('Blocks in {}'.format(block_locations))
 
-        self._say('Available actions: {}'.format({a.name: a.value for a in Eater.Actions}))
+        self._say('Available actions: {}'.format({a.name: a.value for a in all_actions}))
+
+        # creates an iterator that returns a sequence of random actions
+        def random_actions():
+            # prevent unbounded iterations
+            for _ in range(10000):
+                yield all_actions[random.randint(0, len(all_actions) - 1)]
+
+        return random_actions()
+
 
     def end_episode(self, outcome: int, alive: bool, success: bool):
         """Method called at the when an episode is completed."""
         self._say('Episode completed, my reward is {}'.format(outcome))
 
-    # random player
-    def play(self, turn: int, percept: Eater.Percept, actions: Iterable[Eater.Actions]) -> Eater.Actions:
-        actions_lst = list(actions)
-        next_move = actions_lst[random.randint(0, len(actions) - 1)]
-        self._say('I see {}, my next move is {}'.format(percept, next_move.name))
-        return next_move
-
-    def feedback(self, action: Eater.Actions, reward: int, percept: Eater.Percept):
-        """Receive in input the reward of the last action and the resulting state. The function is called right after the execution of the action."""
-        self._say('Moved to {} with reward {}'.format(percept.position, reward))
-        self.reward += reward
-
 
 MAP_STR = """
 ################
@@ -80,7 +78,7 @@ def main(*args):
     Play a random EaterWorld episode using the default player
     """
 
-    player_class = MyPlayer
+    player_class = GooPlayer
 
     world = EaterWorld.random(map_desc=MAP_STR)
     player = player_class('Hungry.Monkey')

examples/environment.yml

 name: wumpus
 channels:
   - conda-forge
+  - nodefaults
 dependencies:
-  - python>=3.6
-  - gym
+  - python>=3.8
+  - gym<=0.22
   - pip
   - pip:
-    - wumpus[gym] @ git+https://gitlab.inf.unibz.it/tessaris/wumpus.git
\ No newline at end of file
+    - wumpus @ https://gitlab.inf.unibz.it/tessaris/wumpus/-/archive/dev/wumpus-dev.zip
\ No newline at end of file

examples/wumpus_usage.py

@@ -5,14 +5,15 @@
 import argparse
 import random
 import sys
+from typing import Iterable
 
 import wumpus as wws
 
 
-class MyPlayer(wws.OfflinePlayer, wws.UserPlayer):
+class GooPlayer(wws.OfflinePlayer):
     """Offline player demonstrating the use of the start episode method to inspect the world."""
 
-    def start_episode(self, world: wws.WumpusWorld):
+    def start_episode(self, world: wws.WumpusWorld) -> Iterable[wws.Hunter.Actions]:
         """Print the description of the world before starting."""
 
         world_info = {k: [] for k in ('Hunter', 'Pits', 'Wumpus', 'Gold', 'Exits')}
@@ -22,6 +23,7 @@ class MyPlayer(wws.OfflinePlayer, wws.UserPlayer):
         for obj in world.objects:
             if isinstance(obj, wws.Hunter):
                 world_info['Hunter'].append((obj.location.x, obj.location.y))
+                all_actions = list(obj.Actions)
             elif isinstance(obj, wws.Pit):
                 world_info['Pits'].append((obj.location.x, obj.location.y))
             elif isinstance(obj, wws.Wumpus):
@@ -35,6 +37,14 @@ class MyPlayer(wws.OfflinePlayer, wws.UserPlayer):
         for k in ('Size', 'Pits', 'Wumpus', 'Gold', 'Exits', 'Blocks'):
             print('  {}: {}'.format(k, world_info.get(k, None)))
 
+        # creates an iterator that returns a sequence of random actions
+        def random_actions():
+            # prevent unbounded iterations
+            for _ in range(10000):
+                yield all_actions[random.randint(0, len(all_actions) - 1)]
+
+        return random_actions()
+
 
 def classic(size: int = 0):
     """Play the classic version of the wumpus."""
@@ -51,7 +61,7 @@ def classic_offline(size: int = 0):
     world = wws.WumpusWorld.classic(size=size if size > 3 else random.randint(4, 8))
 
     # Run a player with knowledge about the world
-    wws.run_episode(world, MyPlayer())
+    wws.run_episode(world, GooPlayer())
 
 
 WUMPUS_WORLD = '''
@@ -74,7 +84,7 @@ def fixed_offline(world_json: str = WUMPUS_WORLD):
     world = wws.WumpusWorld.from_JSON(world_json)
 
     # Run a player with knowledge about the world
-    wws.run_episode(world, MyPlayer())
+    wws.run_episode(world, GooPlayer())
 
 
 def real_deal(size: int = 0):

setup.cfg

+[metadata]
+name = wumpus
+version = attr:wumpus.__version__
+description = This package implements a Python version of the Hunt the Wumpus game as described in the book Artificial Intelligence: A Modern Approach by Russell and Norvig.
+long_description = file: README.md
+license = MIT
+author = Sergio Tessaris
+author_email = tessaris@inf.unibz.it
+
+[options]
+zip_safe = False
+include_package_data = True
+packages = find:
+python_requires = >=3.8
+install_requires =
+    gym < 0.24
+
+[options.entry_points]
+console_scripts =
+  gridrunner = wumpus.cli:main

setup.py

-from setuptools import setup, find_packages
-import re
-import codecs
-import os
-import subprocess
+#!/usr/bin/env python
+import setuptools
 
-# Hack to allow non-normalised versions
-#   see <https://github.com/pypa/setuptools/issues/308>
-from setuptools.extern.packaging import version
-version.Version = version.LegacyVersion
-
-_INCLUDE_GIT_REV_ = False
-
-
-# see <https://stackoverflow.com/a/39671214> and 
-#     <https://packaging.python.org/guides/single-sourcing-package-version>
-def find_version(*pkg_path):
-    pkg_dir = os.path.join(os.path.abspath(os.path.dirname(__file__)), *pkg_path)
-    version_file = codecs.open(os.path.join(pkg_dir, '__init__.py'), 'r').read()
-    version_match = re.search(r"^__version__ = ['\"]([^'\"]*)['\"]",
-                              version_file, re.M)
-    _git_revision_ = None
-    if _INCLUDE_GIT_REV_:
-        try:
-            _git_revision_ = subprocess.check_output(['git', 'describe', '--always', '--dirty'], encoding='utf-8').strip()
-        except subprocess.CalledProcessError:
-            pass
-    if version_match:
-        return version_match.group(1) + ('' if _git_revision_ is None else '+' + _git_revision_)
-    elif _git_revision_:
-        return _git_revision_
-    raise RuntimeError("Unable to find version string.")
-
-
-def long_description_md(fname='README.md'):
-    this_directory = os.path.abspath(os.path.dirname(__file__))
-    with open(os.path.join(this_directory, fname), encoding='utf-8') as f:
-        long_description = f.read()
-    return long_description
-
-
-setup(
-    name='wumpus',
-    version=find_version('wumpus'),
-    description='Wumpus world simulator',
-    long_description=long_description_md(),
-    long_description_content_type='text/markdown',
-    author='Sergio Tessaris',
-    author_email='tessaris@inf.unibz.it',
-    packages=find_packages(),
-    include_package_data=True,
-    entry_points= {
-        'console_scripts': ['gridrunner=wumpus.cli:main']
-    },
-    install_requires=[
-    ],
-    extras_require={
-        'gym': ['gym']
-    },
-    exclude_package_data={'': ['.gitignore']},
-)
+if __name__ == "__main__":
+    setuptools.setup()
\ No newline at end of file

wumpus/__init__.py

 from .gridworld import Agent, Percept, Coordinate, coord, GridWorld, EaterWorld, GridWorldException, Eater, Food
-from .player import OfflinePlayer, Player, UserPlayer, RandomPlayer
+from .player import OfflinePlayer, OnlinePlayer, UserPlayer, RandomPlayer
 from .wumpus import WumpusWorld, Hunter, Wumpus, Pit, Gold, Exit
 from .runner import run_episode
 
-__version__ = '0.4.0'
+__version__ = '1.1.0'

wumpus/player.py

 import random
 import textwrap
-from typing import Iterable
+from typing import Iterable, Union
 
 from .gridworld import Agent, Percept, object_id, GridWorld
 
 
-class Player(object):
+class OnlinePlayer:
     """A player for a given agent. It implements the play method which should
     return one of the actions for the agent or None to give up.
     """
     def __init__(self, name: str = None):
         """
-        Initialise the name of the agent if provided.
+        Initialise the name of the player if provided.
         """
-        if name is not None:
-            self.name = str(name)
+        self._name = str(name) if name is not None else object_id(self)
+
+    @property
+    def name(self) -> str:
+        """The name of the player or a default value based on its type and hash."""
+        return self._name
 
     def start_episode(self):
         """Method called at the beginning of the episode."""
@@ -25,40 +29,34 @@ class Player(object):
         """
         pass
 
-    def play(self, turn: int, percept: Percept, actions: Iterable[Agent.Actions]) -> Agent.Actions:
-        """Given a turn (integer) and a percept, which might differ according to the specific problem, returns an action, among the given list of possible actions, to play at the given turn or None to stop the episode."""
+    def play(self, percept: Percept, actions: Iterable[Agent.Actions], reward: Union[int, None]) -> Agent.Actions:
+        """Given a percept, which might differ according to the specific problem, and the list of valid actions, returns an action to play at the given turn or None to stop the episode. The reward is the one obtained in the previous action, on the first turn its value is None."""
         raise NotImplementedError
 
-    def feedback(self, action: Agent.Actions, reward: int, percept: Percept):
-        """Receive in input the reward of the last action and the resulting state. The function is called right after the execution of the action."""
-        pass
 
-    @property
-    def name(self) -> str:
-        """Return the name of the player or a default value based on its type and hash."""
-        try:
-            return self._name
-        except AttributeError:
-            return object_id(self)
-
-    @name.setter
-    def name(self, value: str):
-        """Set the name of the player.
-
-        Args:
-            value (str): the name
+class OfflinePlayer:
+    """
+    A player that receives the configuration of the world at the beginning of the episode and returns the sequence of actions to play.
+    """
+    def __init__(self, name: str = None):
+        """
+        Initialise the name of the player if provided.
         """
-        self._name = value
+        self._name = str(name) if name is not None else object_id(self)
 
+    @property
+    def name(self) -> str:
+        """The name of the player or a default value based on its type and hash."""
+        return self._name
 
-class OfflinePlayer(Player):
-    """
-    A player that receives the configuration of the world at the beginning of the episode.
-    """
-    def start_episode(self, word: GridWorld):
+    def start_episode(self, word: GridWorld) -> Iterable[Agent.Actions]:
         """Method called at the beginning of the episode. Receives the current world status."""
-        pass
+        raise NotImplementedError
 
+    def end_episode(self, outcome: int, alive: bool, success: bool):
+        """Method called at the when an episode is completed with the outcome of the game and whether the agent was still alive and successfull.
+        """
+        pass
 
 ############################################################################
 #
@@ -70,16 +68,16 @@ class OfflinePlayer(Player):
 #
 
 
-class RandomPlayer(Player):
+class RandomPlayer(OnlinePlayer):
     """This player selects randomly one of the available actions."""
-    def play(self, turn: int, percept: Percept, actions: Iterable[Agent.Actions]) -> Agent.Actions:
+    def play(self, percept: Percept, actions: Iterable[Agent.Actions], reward: int) -> Agent.Actions:
         actions_lst = list(actions)
         return actions_lst[random.randint(0, len(actions) - 1)]
 
 
-class UserPlayer(Player):
+class UserPlayer(OnlinePlayer):
     """This player asks the user for the next move, if it's not ambiguous it accepts also commands initials and ignores the case."""
-    def play(self, turn: int, percept: Percept, actions: Iterable[Agent.Actions]) -> Agent.Actions:
+    def play(self, percept: Percept, actions: Iterable[Agent.Actions], reward: int) -> Agent.Actions:
         actions_dict = {a.name: a for a in actions}
         print('{} percept:'.format(self.name))
         print(textwrap.indent(str(percept), '  '))

wumpus/runner.py

@@ -7,15 +7,16 @@ Functions to run the players
 import argparse
 import copy
 import importlib
+import inspect
 import io
 import json
 import os
 import re
 import sys
-from typing import Any, Dict, Iterator, Type
+from typing import Any, Dict, Iterator, Type, Union
 
 from .gridworld import Agent, GridWorld, GridWorldException, EaterWorld
-from .player import Player, OfflinePlayer
+from .player import OnlinePlayer, OfflinePlayer
 from .wumpus import WumpusWorld
 
 
@@ -43,7 +44,7 @@ def check_entrypoint(arg_value: str, pattern: re.Pattern = re.compile(r"^[\w.-]+
     return arg_value
 
 
-def get_player_class(object_ref: str, path: os.PathLike = None) -> Type[Player]:
+def get_player_class(object_ref: str, path: os.PathLike = None) -> Union[Type[OnlinePlayer], Type[OfflinePlayer]]:
     if path is not None and path not in sys.path:
         if not os.path.isdir(path):
             raise FileNotFoundError('Directory <{}> not found'.format(path))
@@ -51,17 +52,20 @@ def get_player_class(object_ref: str, path: os.PathLike = None) -> Type[Player]:
 
     # see <https://packaging.python.org/specifications/entry-points/#data-model>
     modname, qualname_separator, qualname = object_ref.partition(':')
-    obj = importlib.import_module(modname)
+    try:
+        obj = importlib.import_module(modname)
+    except ModuleNotFoundError as e:
+        raise ImportError(f'Cannot find entrypoint {object_ref}: {e}')
     if qualname_separator:
         for attr in qualname.split('.'):
             try:
                 obj = getattr(obj, attr)
             except AttributeError as e:
-                raise ImportError('Cannot import {} object: {}'.format(object_ref, e))
+                raise ImportError(f'Cannot find entrypoint {object_ref}: {e}')
 
     player_class = obj
-    if not issubclass(player_class, Player):
-        raise NotImplementedError('class {} is not a subclass of Player'.format(player_class))
+    if not inspect.isclass(player_class) or not issubclass(player_class, (OnlinePlayer, OfflinePlayer)):
+        raise RuntimeError(f'{player_class} is not a subclass of OnlinePlayer or OfflinePlayer')
     return player_class
 
 
@@ -101,7 +105,7 @@ def worlds(files, world_class: Type[GridWorld]) -> Iterator[GridWorld]:
             continue
 
 
-def run_episode(world: GridWorld, player: Player, agent: Agent = None, horizon: int = 0, show=True, outf: io.TextIOBase = None) -> Dict[str, Any]:
+def run_episode(world: GridWorld, player: Union[OnlinePlayer, OfflinePlayer], agent: Agent = None, horizon: int = 0, show=True, outf: io.TextIOBase = None) -> Dict[str, Any]:
     """Run an episode on the world using the player to control the agent. The horizon specifies the maximum number of steps, 0 or None means no limit. If show is true then the world is printed ad each iteration before the player's turn.
 
         Raise the exception GridWorldException is the agent is not in the world.
@@ -138,15 +142,21 @@ def run_episode(world: GridWorld, player: Player, agent: Agent = None, horizon:
         'maxsteps': False
     }
     # inform the player of the start of the episode
-    try:
-        if isinstance(player, OfflinePlayer):
-            player.start_episode(copy.deepcopy(world))
-        else:
+    if isinstance(player, OfflinePlayer):
+        try:
+            plan = iter(player.start_episode(copy.deepcopy(world)))
+        except Exception as e:
+            plan = iter([])
+            print(f'Exception in Player.start_episode: {e}', file=outf)
+            game_log['exceptions'].append(f'Player.start_episode: {e}')
+    else:
+        try:
             player.start_episode()
-    except Exception as e:
-        print(f'Exception in Player.start_episode: {e}', file=outf)
-        game_log['exceptions'].append(f'Player.start_episode: {e}')
+        except Exception as e:
+            print(f'Exception in Player.start_episode: {e}', file=outf)
+            game_log['exceptions'].append(f'Player.start_episode: {e}')
     step = 0
+    reward = None
     while not horizon or step < horizon:
         if agent.success():
             print('The agent {} succeeded!'.format(agent.name), file=outf)
@@ -157,12 +167,17 @@ def run_episode(world: GridWorld, player: Player, agent: Agent = None, horizon:
 
         if show:
             print(world, file=outf)
-        try:
-            action = player.play(step, agent.percept(), agent.actions())
-        except Exception as e:
-            action = None
-            print(f'Exception in Player.play: {e}', file=outf)
-            game_log['exceptions'].append(f'Player.play: {e}')
+
+        if isinstance(player, OfflinePlayer):
+            action = next(plan, None)
+        else:
+            try:
+                action = player.play(agent.percept(), agent.actions(), reward)
+            except Exception as e:
+                action = None
+                print(f'Exception in Player.play: {e}', file=outf)
+                game_log['exceptions'].append(f'Player.play: {e}')
+
         if action is None:
             game_log['actions'].append(action)
             agent.suicide()
@@ -171,11 +186,6 @@ def run_episode(world: GridWorld, player: Player, agent: Agent = None, horizon:
         game_log['actions'].append(action.name)
         reward = agent.do(action)
         print('Step {}: agent {} executing {} -> reward {}'.format(step, agent.name, action.name, reward), file=outf)
-        try:
-            player.feedback(action, reward, agent.percept())
-        except Exception as e:
-            print(f'Exception in Player.feedback: {e}', file=outf)
-            game_log['exceptions'].append(f'Player.feedback: {e}')
 
         step += 1
     else:

wumpus/wumpus.py

@@ -272,7 +272,7 @@ class WumpusWorld(GridWorld):
             else:
                 return [getCoord(data)]
 
-        size = next(coordLst('size'))
+        size = getCoord(desc.get('size', [8, 8]))
         blocks = coordLst('blocks')
         hunters = desc.get('hunters', [])
         pits = coordLst('pits')