You probably overdone it a little in the garden, so you might have temporal symptoms of what is called "Tunnel Syndrom" in German.
It's about a pathway through the "tunnel" between your thumb and your palm, where all the nerves you need to move your fingers go through.
When it's swollen it causes the fingers to get numb. When it's getting smaller due to arthritis the sydrome is getting pathologic and has to be treated, before the nerves are cut off for good.
My mom's thumb was getting so thin in the end, because she couldn't move it. After the "Tunnel" surgery, she had to build up new muscle there.
She had that surgery on her right hand. Her left hand is untreated and she can't make a fist, now.
I don't mean to scare you. I really think you just overworked your poor hand, as I imagine you applied a lot of pressure to your palm during garden work.
Try a warm bath for your hand and bandage your palm. The problem is there, not in your pinkie.
Get better soon!
Good luck :-)