Definitely sounds like mould, and the trick is to get rid of it now, before it worsens.
The most natural remedy to use is a mixture of vinegar/lemon juice and bicarbonate of soda. Mix the wet ingredients into the dry until they form a paste, then spread this over the affected areas.
Leave for a few hours, preferably overnight, then scrub the paste into the rubber, which will take the mould with it. You may need to repeat this process, but when finished each time, remember to rinse with clean water, because the vinegar will continue to eat at the rubber otherwise.
And to prevent mould build-up in future, each time you finish doing your washing, leave the door open for a few hours to let water left in the machine evaporate. It also helps to run the machine once a month, with a more liquid mix of these ingredients and with the machine empty.