This article delves into the realm of distributional reinforcement learning, specifically focusing on the QR-DQN (Quantile Regression - Deep Q-Network) algorithm. Unlike conventional reinforcement learning approaches that estimate the mean value function, QR- DQN explicitly models the distribution over returns. The article examines the effects of testing the QR-DQN algorithm in complex environments, such as Atari games, instead of the commonly used Cartpole environment. It further investigates the influence of the number of quantiles employed in the learning process. Additionally, a trick is employed to explore the impact of altering the action selection for the next state during learning. By exploring these effects, this article contributes to the development and comprehension of distributional reinforcement learning techniques.