Robot learning requires large amounts of training data that is often expensive and time-consuming to obtain. This is commonly sought to be addressed by designing more sample efficient learning methods. However, there is a large variation in the cost of acquiring different types of data in robotics. For example, human demonstrations are substantially more expensive than simulation time. In this talk, I will discuss our recently proposed algorithms that explicitly reason about the cost of acquiring data during learning so as to optimize the task performance while keeping costs down. Importantly, our algorithms choose not to train certain parts of the system if they are redundant or if the cost of training is not justified by the improvement in performance. I will discuss resource optimization in the context of collaborative manufacturing, where we show that it can bring down the cost of manufacturing and sequential manipulation under uncertainty where we are able to learn more robust and effective recovery skills using a given training budget.