I thought much the same as you about catfish until an accidental meeting many moons ago. Was driving the Skyline Drive not far from Asheville, North Carolina and found a lovely lake for fishing. Took a break for a bit of fishing (in the USA and Europe, I always drive with my fishing gear). Caught several catfish at first and gave them to a young boy and his father who were fishing next to me when they saw I wanted to throw them back. Later, as we got to chatting, the boy asked why I was going to throw them back and I told him because of the "muddy taste". He and his father broke into laughter and the father told me that like most "Yankee White boys" I was wrong about catfish.
Between us we started a nice wood fire, he took a small grill holder from his car, showed me precisely how to clean the fish (puncture the innards and you get that muddy taste or worse..... learn to clean without puncturing and you're in fine shape) and then how to grill them. No wine available on that day but we did thoroughly enjoy cold beer.
Agreed that farmed catfish are not the most desirable. Here in Israel there are two-three places where one can catch them wild (the Jordan River where there is a crossing from the Galilee to the Golan Heights; the Yarkon river near its source; and the unnamed river that empties near the Caesaria power plant). Good fishing at the first for trout, bass, catfish, talipia (known here as St. Peter's Fish) and in the other two fine catfish.