When should I upgrade my electrical panel

It's Too small

Many older homes still have their original 100 amp service. You may be looking to remodel the kitchen or to add a room, often there is no room in the distribution panel to install the required additional circuits.

It's Unsafe

Your circuit breakers are overloading and tripping off, you notice lights dimming when you turn on a vacuum indicating your are maxing out on available power.
If you live in an older home your electrical panel might use fuses. If so, you should contact an electrician and have the electrical panel replaced to use circuit breakers instead of fuses. Fuses are considered a fire hazard and as a result insurance companies may not insure such a home.