By gum is an expression commonly used to express surprise or excitement. However, there are several other synonyms that can be used instead of by gum. For instance, "by golly," "by jove," "by heavens," or "by George." All of these phrases are similar in meaning to "by gum" and can be used to express surprise or excitement in a less offensive manner. Other synonyms include "by my stars," "by my troth," and "by the holy rood." All of these expressions add a touch of antiquated charm to conversations and writing, and allow for the speaker to express their emotions without having to resort to harsh language.