First, we must determine the optimal strategy which maximises our payoff of playing the game. Let us begin by considering the simpler subgame where we can only reroll once. The optimal strategy is to reroll if our die value is lower than the expected value of the final die roll. Since the expected value of a single die roll is $3.5$, then we reroll if we roll $\{1,2,3\}$, and we stick if we roll $\{4,5,6\}$. Our expected payoff if we stick is the average of $\{4,5,6\}$ which is 5, and the expected payoff if we reroll is the expected value of a single die roll, 3.5. It is straightforward to see that we stick or reroll with equal probability. So, the expected value of playing this subgame is $$ \text{E}[\text{Payoff}] = \frac{1}{2} (5) + \frac{1}{2} (3.5) = 4.25. $$ Likewise, if we have two rerolls then we will reroll if the first die roll gives us a value less than the expected value of the subgame, that is, we reroll if we roll $\{1,2,3,4\}$. So, we will stick if we roll $\{5,6\}$. Therefore, there is a $\frac{1}{3}$ probability that we roll $\{5,6\}$ and stick. The average payoff if we stick on our first die roll is $5.5$. Since we know the expected value of rerolling is the expected value of playing the subgame, we can now calculate the expected payoff of the full game. $$ \text{E}[\text{Payoff}] = \frac{1}{3} (5.5) + \frac{2}{3} (4.25) = \frac{14}{3} \approx 4.67. $$