This paper focuses on the study of an original combination of the Euler Multilevel Monte Carlo introduced by M.Giles and the popular importance sampling technique. To compute the optimal choice of the parameter involved in the importance sampling method, we rely on Robbins-Monro type stochastic algorithms. On the one hand, we extend our previous work on adaptive Statistical Romberg method to the Multilevel Monte Carlo setting. On the other hand, we improve it by providing a new adaptive algorithm avoiding the discretization of any additional process. Furthermore, from a technical point of view, the use of the same stochastic algorithms as in our previous work appears to be problematic and we are reduced to employ a specific version of stochastic algorithms with projection (see e.g. Laruelle, Lehalle and Pagès). We firstly prove the almost sure convergence of this stochastic algorithm towards the optimal parameter. Then, we prove a central limit theorem of type Lindeberg-Feller for the new adaptive Euler Multilevel Monte Carlo algorithm together with non-asymptotic Berry-Essen bounds. Finally, we illustrate the efficiency of our method through applications in option pricing for the Heston model.