Recently, there is a vast interest in developing unsupervised methods that are independent of the feature learning from the training data, e.g., deep image prior, zero-shot learning, and internal learning. These methods are based on the common goal of maximizing the quality of image features learned from a single image despite inherent technical diversity. In this work, we bridge the gap between the various unsupervised approaches above and propose a general framework for image restoration and image retargeting. We use contextual feature learning and internal learning to improvise the structure similarity between the source and the target images. We perform image resizing application in the following setups: classical image resizing using super-resolution, a challenging image resizing where the low-resolution image contains noise, and content-aware image resizing using image retargeting. We also compare our framework with relevant state of-the-art methods.