Consider a photography career as your dream house and the only way for you to be able to maintain it is to establish a foundation strong enough with a job that doesn't necessarily have to relate to photography directly. The point is for you to be able to cover you expenses with your day job while pursuing your passion on the side. You're able to experiment more with you style and quite frankly refine it because there's no pressure in thinking that every frame you take has to produce money for the rent.
Overall, great advice by Christopher. I invariably get asked by close friends why haven't I gone full-time with what I still consider a hobby (photography) and my honest response is "I don't need to" or at least that's my state of mind now but who knows in another year.