The accuracy of Machine Learning models is going up by the day with advances in Deep Learning. But this comes at a cost of explainability of these models. There is a need to uncover these black boxes for the Business users. This is very essential especially for heavily regulated industries like Finance, Medicine, Defence and the likes. A lot of research is going on to make ML models interpretable and explainable. In this talk we will be going through the various approaches taken to unravel machine learning models and explain the reason behind their predictions. We’ll see the different approaches being taken by discussing the latest research literature, the behind the scenes view of what is happening inside these approaches with enough mathematical depth and intuition. Finally, the aim is to leave the audience with the practical know-how on how to use these approaches in understanding deep learning and classical machine learning models using open source tools in Python.