Looking at your current situation your gross income is £19,976pa (£10,888 salary, £4,512 CS pension; £4,576 State pension). Tax due on this is (19976-10600) x 20% which is £1,875. So your current tax payments of £160 per month (£1,920 pa) is a little more than you need to pay.
Assuming you will only receive half a year’s salary this year as you are stopping work in October your gross income for the year will be £14,532 (£19,976 - £5,444). Tax due on this will be £786 but you will have by then paid £960 (£160 x 6). So you will be due a tax rebate of £174 which should be returned via PAYE on your CS pension.
As I said, next year you should pay no tax. Your tax allowance should be around £6200 (maybe a little less depending on how much your state pension is increased by) but this should more than cover your CS pension. Finally, as bhg has pointed out, you will have a small amount of tax-free interest on your savings (about £500) and you will have to reclaim this at the end of the current tax year (using form R40). But this will increase to about £5,000 next year and you should then declare yourself a non-taxpayer and have your interest paid net of tax (using form R85).