We started using Square before it was as common as it is now. Didn't have any issues using it, worked on all our devices and rates were similar to paypal which was an option at the same time (2011) when we first launched.
We get paid mostly with checks and like yourself I'd prefer it most of the time, however I will say some folks just like my wife and I, may have bonus rewards or cashback and prefer to use the card for those rewards reasons and will be much quicker to pay often, though not always.
After some time I was doing work for a federal client and found their purchasing folks hollered at them as Square and Paypal are "third party" and not a direct bank.
This caused me to commit to a bank at the time and I went with Bank of America. Most banks charge either an annual fee or monthly fees to have their version of a swiper and account of this nature and a percentage rate per transaction and "batch" rate as well, but ultimately there are other options I've seen folks talk about on here maybe 2 years ago or further back, including some other banks that aren't the main banks we here about and have good options.
I'm looking at my local branch of my local bank now as they will give you a check if they can't beat the competitor, so might as well look. I don't have anything bad to say about BOA but don't like the monthly when I don't use CC that often.
You can deduct the fees to extent allowable by law and you can pass on the fees if you want to your client and say so in your contracts.
Ultimately we've used square and still do for non govt. work and paypal for our white-nose syndrome non profit grant raising booths, but Bank of America for our official govt. contracts and for any other client if its what I have handy. Have all 3 apps on my phone and all three swipers in my rig.
Hope that helps, we have had no issues, I'll add since we all like to see money appear quickly once paid, the bank is by far the longest wait for the money in my experience with my bank. Square is fastest at 2-3 days, B.O.A. can take 4-5 days.
Also should add, we started using quickbooks online for this 2018 year and they offer ability to take credit cards on the platform as well as bank transfers, you basically create the invoice, select which boxes you are willing to allow types of payment wise and then email to your client. They can log on and use whatever method or mail you a check if they don't want the others.
The first client I allowed to bank transfer, from the time they submitted it, took nearly 7 days! No more of that nonsense for me, but I would use their credit card app or portal and feel good about it just the same as the others I have, they all run similar rates and breaks in rate are usually going to be for amount of processing you do, etc.. and I don't do much as most pay with check.