Metamathematics is mathematics used to study mathematics. It was originally differentiated from ordinary mathematics in the 19th century to focus on what was then called the foundations problem in mathematics.
Many issues regarding the foundations of mathematics (there is no longer necessarily considered to be any one "problem") and the philosophy of mathematics touch on or use ideas from metamathematics. The working assumption of metamathematics is that mathematical content can be captured in a formal system.
On the other hand, quasi-empiricism in mathematics, the cognitive science of mathematics, and ethno-cultural studies of mathematics, which focus on scientific method, quasi-empirical methods or other empirical methods used to study mathematics and mathematical practice by which such ideas become accepted, are non-mathematical ways to study mathematics.