Homemade chocolate turns crumbly

GdD gives a good answer, but I am going to give a recommendation type answer based on your ingredients because although the quantities are different the ingredients are exactly the same. Go to Hershey.com and search for their fudge recipe. Follow this recipe.

When your "fudge" reaches nearly the desired consistency from stirring, start stirring in the peanuts, and quickly spoon out your clusters onto wax paper.

I think that will give you the desired results and give you a silky fudge chocolate peanut cluster.

Chocolate gets its texture from cocoa butter, which is substituted by regular butter in this recipe, so what you get from this recipe is not really chocolate, but a chocolate flavored mix. You are never going to get as silky a texture with regular butter as opposed to cocoa butter.

There's quite a bit that could be going wrong with the recipe and method you posted. It does not give exact measurements (heaping tbsp is not a good measurement), or give much details of what you are looking for texture-wise during the cooking process. It also calls for boiling it for 6 minutes, which I would never do myself. If I were using that recipe I would add add more butter, and I would not boil it. I would mix the cocoa and sugar with a bit of the milk to make a thick paste, then I would add the rest of the milk and the butter and cook gently for 5 minutes, say about 170F, stirring to keep it from burning. Then I'd add the vanilla (unspecified amount, maybe 1/2 tsp at most) and nuts, then let it cool some until it gets a bit more viscous before spooning it out.

My gut tells me this recipe and method are pretty flawed though, so I would try a different recipe first or just melt some chocolate and add nuts.