Grammar or pronunciation or vocabulary?
You can express yourself with faulty grammar and less than perfect pronunciation. If you do not have the words you cannot express yourself. The constant battle to acquire enough vocabulary to read what you want to read, to say what you want to say and to understand what you want to understand, that is the hardest part.
The grammar comes as you need it with more and more exposure. Imperfect grammar and pronunciation do not prevent communication and enjoyment of the language. Lack of vocabulary does.
When I correct writing, it is overwhelmingly vocabulary, improper use of words and phrases that is the biggest problem, not grammar.
How do you accumulate words and phrases? You do so from input, from reading, and from listening to content that is of interest to you. You have to see the words and phrases often in different contexts. But then you have to use them in writing and speaking. Writing can really help because you can analyze what you are doing and which words you are using wrong.
It was the lack of a systematic method for accumulating words and phrases that motivated me to develop The Linguist.