This project utilizes reinforcement learning (RL) to solve a navigational task within a defined environment, specifically, a "Lawnmower Grid World". The environment consists of 16 states with a goal to maximize collected rewards while avoiding penalties.The primary tasks involve moving from the initial state at the top-left corner of the grid to the goal state at the bottom-right corner, collecting batteries (positive rewards) and avoiding pebbles (negative rewards).
![](https://private-user-images.githubusercontent.com/44944748/323046267-e28881a3-14f3-4bdd-bf49-3e2c4a9a95bf.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjM4NDk1MzIsIm5iZiI6MTcyMzg0OTIzMiwicGF0aCI6Ii80NDk0NDc0OC8zMjMwNDYyNjctZTI4ODgxYTMtMTRmMy00YmRkLWJmNDktM2UyYzRhOWE5NWJmLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE2VDIzMDAzMlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTExYmYwMjIwYjExMTYwMDAwODI0MTgyZGVlOTNiOWNiNjc1ZjVlOTQ3ZjgyZjI4ZDUyM2UwMWJlNTAzZWZjNzQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.yATwGtn7htXiLZDsh7ov71eQmRM-B0LeyRwIHIlvQVo)
- States: The states are defined as tuples representing the row and column indices on a 4x4 grid, ranging from (0, 0) to (3, 3).
- Actions: Four possible actions include moving Up, Down, Right, and Left.
- Rewards: Rocks represent negative rewards (-5, -6), batteries positive rewards (+5, +6), and the goal state offers the highest reward (+10).
- Objective: Navigate through the grid to reach the goal state at (3, 3) by maximizing the total rewards.
![](https://private-user-images.githubusercontent.com/44944748/323046489-0c9807c2-844e-4169-aa53-a26925b38fa3.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjM4NDk1MzIsIm5iZiI6MTcyMzg0OTIzMiwicGF0aCI6Ii80NDk0NDc0OC8zMjMwNDY0ODktMGM5ODA3YzItODQ0ZS00MTY5LWFhNTMtYTI2OTI1YjM4ZmEzLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE2VDIzMDAzMlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWI5YmVlZDcwYjM4ZmY0OWM3ZTAwNjQzZTA4NTFiOTZhZWQyZmM5ZDVmZThhMTZhNDBhZmRmMzVhNDljNzI2ZjcmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.bJwnT8KfuNkDHs4s4biRblD76Y8sSwhtRSkOM5j2Bds)
![](https://private-user-images.githubusercontent.com/44944748/323046797-912526f6-521e-43e2-bbf6-0e8e30cac7b9.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjM4NDk1MzIsIm5iZiI6MTcyMzg0OTIzMiwicGF0aCI6Ii80NDk0NDc0OC8zMjMwNDY3OTctOTEyNTI2ZjYtNTIxZS00M2UyLWJiZjYtMGU4ZTMwY2FjN2I5LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE2VDIzMDAzMlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTA1NWJhODQ0Y2EyZDMxODRlZTUwZDFiNTFjOGE2Nzk4MmRhNjc0ZmU0MzUzMjZhNWFjMDY4N2NjZjhmYTU2NTkmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.8YCbSfePlPRUwyScGgfpHAm1zvqZ6cFRvBkOj1EziFQ)
- Approach: On-policy learning algorithm where the agent learns from the action taken based on the current policy.
- Update Formula: Q(s, a) = Q(s, a) + α (r + γ * Q(s', a') - Q(s, a)), with parameters for learning rate (α), reward (r), discount factor (γ), and Q-values of the current and next state-action pairs.
- Features: Employs a direct method from actual experiences with a balance between exploration and exploitation.
- Approach: Off-policy algorithm that learns the optimal policy indirectly using the greedy method.
- Update Formula: Q(s, a) = Q(s, a) + α * (r + γ * max(Q(s', a')) - Q(s, a)).
- Features: It tends to overestimate Q-values due to its maximization step but is robust in large state spaces.
The project showcases the application of SARSA and Q-Learning to a simple yet illustrative RL problem, emphasizing the importance of hyperparameter tuning and algorithm selection based on the specific characteristics of the environment and task objectives.