) As a result maybe you haven't put in the hours and focused thought that a lot of people with the "it" factor have. They may have started off playing with different things, maybe even for a long time, but at some point they shifted into buckling down and narrowing their playing field and they started tons of more focused work, a lot that they might not even show anyone. Lot's and lot's of sketches and/or finished piece. Tons. They really hash out ideas, subjects, and mediums. They probably have made a lot of not so great art in that process. Yes they may explore with different things and may not exclusively stick to the same things, but they have put in the hours to know how to manipulate their work in certain ways, and can focus more on emotional impact.
That's a big question you're asking! The "it" factor is so hard to explain or describe, that's why we say just "it" to describe it isn't it? In French we say a "je ne sais quoi" qhich literally translates to a "I don't know what".
To me it just means you capture the emotional essence, personality or atmostphere of what you're drawing. I think it's so hard to explain because it's different from piece to piece! Let's say you're drawing a house on a random street. Maybe you'll draw it very technically, clinically. To you, it's just a random house. The drawing will likely end up pretty boring... But let's say the owner draws his own house, well he may add.. a cat snoozing in the window, or maybe a child's toy forgotten on the lawn, or whatever little detail that makes it a home to him. Suddenly, the drawing has something more to it, it has the it factor!
For portraits it's the same thing, if you know the person or their story, you may interject something of that in their expression, or maybe even subconsciously in the colors or shading that will suggest something subtly different. I will wager that's what happened in the portrait of your friend's deceased brother. Maybe the drawing had a aura of sadness, or melancholic happiness, or something. Without that something, it's just the portrait of a guy right?
This it factor tend to be captured most successfully when people draw something they know, something close to their heart, because they can capture just what it is about it that makes it special. Even realistic drawing isn't exactly like photography, and in each of the choices you make you're attempting to show something. Maybe the photography of a house you find creepy would look to someone else like a perfectly normal looking house, but your drawing of it might subtly show that creepy vibe that YOU are feeling when you look at it...
That's my take on it anyway!