Your avatar will automatically start swimming in any water that is set up for swimming.
To just continue swimming in a specific direction, you can use the general movement keys (W,A,S,D, or Up, Down, Left, Right), or hold down the Left and Right Mouse Buttons at the same time.
'CTRL' will make the character dive under water. Releasing 'CTRL' will release the dive.
To swim back up, press 'SPACE.' If you are at the surface and press 'SPACE,' your persona will jump slightly out of the water.
You can swim faster by holding down 'SHIFT.'