Maintaining the grout for brick walls (repointing or tucking as sometimes is called) is important not only an aesthetic reasons but because it makes sure your building structure is safe.

"Over time, weathering and decay cause voids in the joints between masonry units, allowing the undesirable entrance of water. Water entering through these voids can cause significant damage through frost weathering and from salt dissolution and deposition."

Wikipedia