I always burn my fudge. Why?

I'm trying to learn how to make fudge. Usually, I have no problems following recipes, but this one seems to be particularly difficult:

1 can of condensed milk (397 g)
115 grams of butter
450 grams of sugar
1,5 deciliter of milk

The process is as follows:

1. Mix everything together in a pot and slowly heat up, making sure all sugar is dissolved before the mixture boils.
2. Once boiling, leave the mixture alone, not stirring it, until it reaches soft ballstage (I use a candy thermometer to determine the soft ball stage).
3. When it reaches soft ball, remove pot from heat and leave it alone until it reaches 43 degrees celcius.
4. Finally beat the mixture with a wooden spoon until it sets (glossy surface turns dull).

The problems I face are these:

1. No matter what I do, the sugar burns at the bottom of the pot. I have tried a large pot, a small pot and a low and wide pot. I set the burner to very low (2) on a 1-9 scale on a normal ceramic plate cooker so the mixture simmers (not a rolling boil).
2. I have tried to prevent the burning by stirring but then the mixture crystallizes and the fudge turns grainy. If I don't stir, the fudge on top turns out okay, but there's a very bad burn on the bottom. Sometimes the fudge on top is dark brown, though. (see problem 4).
3. The boiling time is extremely long. More than an hour on the low/wide pot and up to one hour and 20 minutes in a regular pot. Almost all recipes says about 20 minutes cooking time. How can this be?
4. The fudge almost always turn a dark brown (approx. at the 110 degree celsius stage). This happens even if I stir, and I guess it's due to the burn on the bottom of the pot.

My candy thermometer is properly calibrated and I cook at sea level.

Can the problem be my pots? They have quite a thick bottom, but I guess I could find a thicker one. But would this help against burning? After all, at more than one hour at the same burner setting, I expect the bottom to be just as hot in a thick pot? What type of pot do you use, if you're able to make fudge.

Could it be humidity? It was quite humid some of the days I tried, but as I understand this should affect the setting of the fudge only (my problems turn up before cooling). Possibly boiling time too? But still, why does it take so much longer that the recipes state?

Please, please help me, so that I can make delicious, smooth, light brown fudge. I've googled everything, it seems and watched a ton of videos, but it just won't work. Please note that I'm not looking for alternative recipes. I really just want this one to work.

Update (nov 8th, 2017)

I've tried to follow the recipe a couple of times more but still to no luck. I did the following, based on the advice given in comments and answers to this topic:

1. Used a heavier pot.
2. Swirled instead of stirring.
3. Delay adding of butter until after cooling.

... but the mixture still burns. Burning is not an issue in the first half hour of cooking, so maybe I'll try letting it boil until it colors and then start stirring constantly until it is finished. Otherwise I just have to give up, but I really hate that.

Update (nov 9th, 2017)

I tried stirring constantly during the cooking and this time I avoided burning the mixture. Also, cooking time was reduced to approx 30 mins.

It was clear that my stirring made the mixture boil harder on the surface, and thus it seems that the lower part of the mixture (closest to the heat) is getting hotter faster than the surface. This might explain the prolonged cooking time when burning the bottom: A burnt layer on the bottom of the pan would make it harder for heat to pass through.

Even though I didn't burn the mixture and thus made some progress, the fudge turned out hard and gritty. I think that I'm not careful enough avoiding stray crystals dropping into the mixture (from the sides of the pot or whereever).

I think I'll need to try another recipe with some syrup to reduce the crystalization. Next time I think I will try this one which contains 200 grams of golden syrup. The author of that recipe also complains about gritty fudge from other recipes.

Update (nov 15th, 2017)

I successfully made a very delicious batch of fudge using the recipe referenced in the previous update. For reference, here's the recipe:

100 grams of butter
550 grams of demerara sugar
200 grams of golden syrup
350 ml double cream (replaced with 38% cream)
1 tsp vanilla extract (replaced with 1 table spoon of vanilla sugar)
¼-½ tsp sea salt flakes

Mix all ingredients and bring to simmer on medium-low heat after ensuring all the sugar is dissolved. Boil until softball stage, stirring occasionally to ensure that the mixture does not burn. When softball stage is hit (after about 1 hour), remove from heat and mix in the vanilla extract and salt. Beat for approx. 10 minutes until the mixture is setting and losing its gloss. Pour into a 23x23cm container and let cool for about 1 hour before cutting into squares.

I'm sure that it is the syrup that allows me to stir the mixture without risk of crystalizing, as I sometimes make toffees with a recipe with glocuse sirup that I can also stir without problems.

By the way, I discovered, making those toffees that my candy thermometer was not able to measure correctly. It's quite embarrassing, since I was sure that the thermometer was giving the right readings. I hope I haven't wasted everybody's time because of my stupid mistake. I think the misreadings were caused by the size of the batch as the thermometer was able to make correct readings in the recipe listed in this update which give a slightly larger batch of fudge.

Thus, some of my problems with the original recipe might be due to me simply cooking the batch too much. I might try the original recipe again and rely on a test in a glass of cold water instead of the thermometer readings. In that case, I'll update this topic again.