During the Old Covenant it was the Temple. Jesus points this out when he flips the tables in the outer courts of the Temple. "My house shall be a house of prayer..." was the cry as coins flew through the air and livestock made way for the Prince of Righteousness. God's house was the temple and his presence was in the Holy of Holies. Previous to this God went camping with the travelling desert dwellers. God's desire however has not been to dwell in buildings that are constructed by men.
An easy way to answer this is to simply say that God lives in heaven. That would be a right answer but not right in totality. In the New Covenant God lives in a believer. 1 Corinthians 3:16 and 1 Peter 2:5 are clear that since Jesus finished work at Calvary God dwells in Believers! So, if a number of believers meet together in a physical building, does that building collectively become God's house? Ephesians 2:22 tells us we are being built together as a dwelling place in the spirit. So the building is the people and is never the bricks and mortar, even when meeting together.
The greek word that has been translated 'church' is 'ekklesia.' This described 'the called out assembly' is the common understanding. So the people are exclusively God's house, one He is building together. People have often said to me, a house-church advocate, that it doesn't matter where people meet, the house of God is being built. The whole idea of people meeting in homes rather than auditoriums is not to define where God lives, but to rather define the nature of the building. We obviously don't stack on top of each other like bricks so what does it mean when God is 'building' His house?
The nature of the building
If I designed a house it would be to my specific requirements. It would have elements that I see as being useful for my purposes. The same is true of God. He is building in the sense that he is making His people the way He wants them. Firstly believers are being built up in the Fathers love. This creates belonging and a deep sense of son-ship. Secondly, God wants to have us issue forth into this world and be salt and light. My own house would have rooms for my family because I want it to be a house where love resides. Then I would be looking at 'function.' God is building believers together to firstly be a people who love and are loved. Then a people who are inwardly met can go and meet the needs in this dark world.
The building site
The process of believers basking in the Fathers love and becoming fulfilled in this way rises and falls on the nature of the gathering. If the gathering is all about serving and saving then it greatly neglects the foundation of God's unconditional love forwarded to our hearts. Knowing that He loves you is not not the same as walking in the reality of the love, and that is where many gatherings of believers miss. God has instructed that the family is the place where we can experience the love of the Father as agenda-less people come together is a place where families meet: the home. Auditoriums, halls, cathedrals start off on an institutional / fraternity footing and more often than not bypass the relational flow of authentic family.
God's genius in living in each believer is clear. Only together can we be build up to fulfill the plans and purposes of God. I'm a brick and so are you! Let's facilitate the building of God's house by clearing the way for the foundations and walking with the plan to be bound together with cords that cannot be broken.