Can I start small and add on later?

Yes. Solar is quite unlike a computer. If you start with a couple of good base components it is easy to add to your system later. First is to start with a good charge controller. Lets say you want to start with one solar module now. Don’t buy a charge controller that can handle one module. Buy a 20 or a 30 amp charge controller that can handle several modules. This will keep you from throwing away a small controller because you outgrew it. Most larger charge controllers cost less then two smaller charge controllers, so you will also save money. 

The same principal goes for inverters. An inverter is a one time purchase. So think about what your needs will be in the future and buy something you can grow into. For example if you are looking at a 2500 watt 12 volt inverter. And you are using 80% of its power rating now you are not leaving yourself any room to grow. You may want to think about a 4000 watt 24 volt or 48 volt inverter. The higher voltages will save you in wiring cost but also mean you will have to add solar panels in 4 modules increments to make 48 volt.