Stop agent walking off the grid (grid world example of RL)

Question

My Problem is when my agent gets to 0,0 , 0,9 etc i am trying to stop it from going to 9,0, or 9,9 etc like this in this example

   (0,0) N -1.0 (9,0)
   (9,0) N -1.0 (8,0)
   (8,0) W -1.0 (8,9)

I want it to move in-between the gird and not take short by going back around the grid. Eg, instead of goind form (0,0) to (9,0) go to (0,1) 0r (1 ,0) etc.

I tried something simple like this code, to check if the x and y values are greater than the x and y values for number of rows and columns or if x and y is less than 0

  public boolean Notvalid(int x, int y) {

    return (x > cr.NUM_ROWS || y > cr.NUM_COLUMNS || x < 0 || y < 0);

     }

and call this method when setting the x and y co-ordinates

 public GridState(int xpos, int ypos) {

    if (!Notvalid(x, y)) {

        x = xpos;
        y = ypos;

    } else {
        x = cr.START_ROW;
        y = cr.START_COL;
    }
}

Anybody know an easier way to handle a rule like this?

Answer 1

It seems like there are three problems with your code:

• Assuming that row indices are 0 to 9 and cr.NUM_ROWS is 10, you should use x >= cr.NUM_ROWS or x > cr.NUM_ROWS - 1 instead of x > cr.NUM_ROWS. (same for columns)
• You are checking whether the current x/y position is valid (x and y), and not the position where the agent is about to go (xpos, ypos).
• By setting x = cr.START_ROW, it seems like you are exactly doing what you do not want to do, i.e. set the agent to 0 when it goes to 10.

Try this:

public boolean notValid(int x, int y) {
    return x >= cr.NUM_ROWS || y >= cr.NUM_COLUMNS || x < 0 || y < 0;
}

public gridState(int xpos, int ypos) {
    if (! notvalid(xpos, ypos)) {
        x = xpos;
        y = ypos;
    } else {
        // new position not valid -> just stay where you are
    }
}