I think it depends from person to person. I also had difficulty in the beginning. Either I would always fall asleep because of how relaxed I was or my head would end up thinking about something else. I simply practiced, lots of practice, and I also practiced emptiness meditation, with breathing exercises included. It doesn't matter if you get it wrong the first few times but the important thing is that you try and try again.
Never lose hope, I am sure you will succeed.