One of the earliest meanings of 'tosh' came from cricket in the late 19th century and referred to bowling that was contemptibly easy to deal with...ie what we today would probably call 'rubbish bowling'. It then came to mean 'rubbish' in any circumstances.