Even orthodox cavalry carried firearms, especially the pistol, which they used in a tactic known as the caracole.
From the en.wikipedia.org
The pistol was specifically developed to try to bring cavalry back into the conflict, together with manoeuvres such as the caracole.
From the en.wikipedia.org
Charles IX was, however, able to implement the Dutch system for fighting in caracole among the cavalry, with unfortunate results.
From the en.wikipedia.org
Reiters specialised in manoeuvering in deep, close formations and practiced a tactic known as the caracole where successive ranks of men rode forward, shot and retired to reload.
From the en.wikipedia.org
More examples
Make a half turn on a horse, in dressage
The caracole or caracol (from the Spanish caracol - "snail") is a turning manoeuvre on horseback in dressage and, previously, in military tactics.
(Caracoles) The Caracol people are an English-speaking people who have been established in Northern Honduras (specifically, the Bay Islands) since the early 1800s and are mainly of European British-Caribbean descent. ...
Elaborate cavalry manoeuvre in which the front rank fired its pistols, trotted to the rear to reload and was replaced by the second and successive ranks. First used by German reiters in the sixteenth century, but made obsolete by Swedish cavalry tactics intitiated by Gustavus Adolphus.