In many real-world domains, e.g., driving, the state space of offline planning is rather different from the state space of online execution. Inspired by such sequential decision-making problems, we propose a bi-level framework called RePReL, an integrated planning and RL framework.  In RePReL, planning occurs offline, at the level of deciding the route, while execution occurs online and needs to take into account dynamic conditions on the road. The agent typically does not have access to the dynamic part of the state at the planning time nor does it have the computational resources to plan an optimal policy that works for all possible traffic events. The key principle that enables agents to deal with these informational and computational challenges is abstraction.  In this talk, I argue that domain-specific knowledge can be leveraged to construct appropriate abstractions and propose a dynamic Statistical Relational Learning (SRL) language for the specification of task-specific abstraction. I present empirical results in various grid world domains and a robotic task to underline the significance of the proposed language for efficient learning and effective transfer across tasks.