At its core, a battery consists of two different electrodes sitting in a conductive medium (the electrolyte). The two electrode materials give up electrons with different ease.
At one electrode, electrons are released (oxidation); at the other, they're taken up (reduction). Since the electrolyte itself doesn't conduct electrons well, they have to take a detour through the external circuit – this exact flow of electrons is the electric current that, for example, lights up a bulb.
The reaction continues until the starting materials at one of the two electrodes are used up – then the battery is empty. In rechargeable batteries, external energy can force the reaction to run in the opposite direction, regenerating the original starting materials.