Quick answer: the in and out of it is that it's harder. Inhale and exhale, that is.
It's been a long time in between harmonica and the other instruments that it's hard to remember the first struggles thirty and almost 60 years ago for my other two main instruments (dulcimer, flute) I was just a kid of 10 when I started flute; my progress has continued, not in finger speed but in musicianship and expression, but that comes from general musical growth.
Dulcimer, from 1975, was (reportedly) painful in the beginning to my (then) wife until I got better. I got pretty good after awhile.
Harmonica! I didn't start (restart) until 2004, and admittedly haven't devoted the time to it to get up to par. The breath patterns still give me fits, which means I need to spend more time with it to get any good. Seven years puts me behind where I should be.
I think all the other things are there; expression and musical ability and sensitivity, which gives me a leg up, thanks to my previous experience. But I've plateaued.
Tom