Interactive learning environments often provide help facilities because they are shown to influence learning positively. Hints, for example, help students remember concepts relevant to learning activities, identify their mistakes, make inferences and so forth. However, several studies show cases of ineffective help use. In this project we observed that in the context of informal learning over the summer, students who were given access to on-demand hints while answering math problem sets took longer to complete them and did not perform better than other students who did not have access to on-demand hints on the first three problems of the problem set. In the context of formal learning during the regular school year, students given access to on-demand hints while answering math problem sets had significant performance improvements and had similar completion times as students without access to on-demand hints on the first three problems of the problem set. Furthermore, students who learned during the regular school year dropped out more than students who learned during summer, which indicates the need for learning support facilities that do not only address cognitive challenges, but also metacognitive issues. Hints’ effectiveness are largely influenced by the context in which they are situated. A deeper understanding of the learning context informs the design of better learning support facilities, such as those intended for learning over the summer. Developing appropriate support for learning over the summer is particularly interesting because *summer slide* or *summer loss* impacts student learning during the regular school year significantly. There are several challenges in current summer programs, which may be addressed with the help of learning systems.