You have to work at it
I am constantly amazed at the number of people who are surrounded by a language but do not learn it properly. Here in Vancouver there are countless people who live in English, work in English, and even watch TV and read the newpaper in English and yet their English is poor. Why?
In my view, you have to work at it. You have to have a deliberate approach of imitating the pronounciation and wording of native speakers. You have to work to increase your vocabulary. You cannot be lazy and stay with the turns of phrase and pronunciation of your native language.
As I often say, if your life depended on it, if you were a spy for example, you would learn. Or, beter still, if I offered these people one million dollars to really improve their English, they would improve quite quickly. It does not just come naturally.