The Rehabilitation of Offenders Act isn't recognised by the US authorities so, as far as they're concerned, a conviction can never become 'spent'.
EU law prevents the UK from making UK criminal records generally available to the US immigration service. The UK is allowed to pass on information about specific individuals (e.g. people convicted, or suspected of, drug-trafficking, people-trafficking, child-sex tourism or terrorism). So the usual advice (up until recently) here on AB has always been to simply 'forget' about your criminal past and lie on the Visa Waiver form.
However, under anti-terror agreements, the UK is also allowed to pass on information when they receive a specific request for it from the US authorities. Now that all visitors to the USA (travelling under the Visa Waiver Program) have to provide advance information (using the ESTA system) it's possible that the USA is requesting information on every traveller. Because 'anti-terror' matters are, naturally, shrouded in secrecy, nobody really knows for sure exactly what is now passed on. Because of that, the standard advice, here on this site, has changed from "Go for it" to "Don't take the risk".
Prior to the introduction of the ESTA, there were plenty of contributors to this site who stated that they'd got into the USA by 'forgetting' about their convictions. I've not seen any recent similar posts, but there have been several press reports regarding travellers who've had very unpleasant experiences because they'd arrived in the USA with undeclared criminal convictions.
Chris