Noah Smith argues the case for using mathematical models. He does acknowledge some downsides witb such models.

There are advantages to telling your stories in English [or other verbal languages, my correction] instead of in Math. One is that you can be more flexible - you can add stuff to your story, or be ecumenical in your view of the world, without being worried about pesky internal contradictions. The other advantage is that you can explain your ideas to a wider class of people, who either don't understand Math or can't be bothered to read through it.

Smith however fails to note that the use of Math

also means that you have to make theories wrong deliberately, such as how the requirement in Lagrange multiplier that the partial derivative in equilibrium must always be zero, something that for examples excludes the existence of entrepreneurs since their action presupposes that there are unexploited profit opportunities.

Smith then argues the case for mathematical models

Of course, there are disadvantages as well. One is that you can't give quantitative predictions: If we deviate from NGDP Level Targeting by X%, how much will our incomes go down? And so forth. This problem can be solved by using simple "ad-hoc" or aggregate-only models, which are fairly easy for most people to grasp and can be modified without much effort.

Well, first of all, it should be emphasized with regard to quantitative predictions, that first of all even if they can be given correctly in a certain case, it won't have theoretical significance since all quantitative relationships only apply to a specific time and place. Remember, economic theory deals with universal relationships, while statistical relationships are really just quantitative forms of economic history.

And secondly, while some econometricians may claim otherwise it is impossible to exactly pinpoint the ceteris paribus causal effect of certain policies even in a specific time and place. One can however make rough estimates of the causal effects by analysing changes in various relevant statistics. But that doesn't require and is probably not even aided by econometric analysis, and it is definitely not even aided by stating economic theory mathematically.

Smith then uses a second argument:

Then there's the second disadvantage of English storytelling, which is internal consistency. Formal models rely on their assumptions, but once you choose your assumptions, your conclusion is forced by the Math. With English storytelling, you can state conclusions that don't follow from your own assumptions, and often no one will be the wiser.

I don't want to sink to the lows of certain advocates of mathematical economics, who hint that opponents oppose it because they are too stupid or lazy to learn advanced mathematics, but the fact is that it is possible to detect logical flaws in verbal logical reasoning if you really think through things.

Since economics like other sciences should be as easy to understand as possible, we should at any case strive to make it (including logical flaws in arguments) as easy to comprehend as possible, so that we can avoid the distraction from the issue of how the economy works, that the way that economics is expressed represented (and of course the insulting hints about your opponents intelligence).

Trying to express the theories in formal terms is indeed probably a good way to better and quicker detect possible logical flaws in economic reasoning. However, I don't really see why formal terms need to be expressed in advanced mathematics, rather than normal verbal languages. Aristotles' classical formal syllogistic logic was expressed in verbal terms, such as

All humans are mortal

Aristotle was a human

Therefore, Aristotle was mortal

Here the premisses of the argument that Aristotle are explicitly stated, and we can clearly see that assuming that the two premises are correct, and they clearly are, then the conclusion is also correct. This kind of formal reasoning can of course be made in economic contexts as well, for example.

People respond positively to economic incentives

Tax cuts on labor improve economic incentives to work

Tax cuts on labor will make people respond [by working more]

Noah's third argument is the most incomprehensible:

If Sumner used formal models, I could at least go and do a statistical test of the model (for example, a Full Information Maximum Likelihood test) against the available macro data. Sure, no one would pay attention if the test rejected the model - the data are crappy and the econ profession lacks generally agreed standards for scientific disproof - but at least I could do it.

So, statistical tests of mathematical models can't really settle economic theoretical disputes, but it should still be done, because it can be done. Sort of arguing that while astrological forecasts won't give you any hints of the future you will actually face, you should still pay attention to them because it is possible to do so.