People are falling into a trap of greater inactivity during middle age, according to new research.