I had a similar dilemma recently and was agonising over my moral quandary- go for a new job that actually was better for me in terms of experience and pay, or be loyal to a previous employer and have the safety net of familiarity! I felt that if I accepted the other job I would in some way be "letting down" my old employer as I know it would have made their lives easier, taking on someone who knew the system/team/patients, however ultimately this is a long hard road and we need to get the best experiences we can.
You need to think about the long term and where you want to be- services are always going to be able to recruit but you may not necessarily get the job opportunity again.
Also unfortunately high turnover is quite common, and on a more practical level if you're leaving fairly early they may well have a bank of other interviewees who they could offer it to without having to interview again.
I think as long as you convey that you genuinely appreciate the time they have given you you'll be fine. It's not ideal but managers are managers and have to deal with these things all the time!
Best of luck for your interviews